THE DEVELOPMENT OF 'PROJECT 1'- FORMATIVE ASSESSMENT Project 1: Formative Assessment 1 Volume 13, Number 3 ISSN 1099-839X The Development of 'Project 1': Formative Assessment Strategies in UK Schools Ian Clark University of Washington Citation Clark, I. (2011). The Development of 'Project 1': Formative Assessment Strategies in UK Schools. Current Issues in Education, 13(3). Retrieved from http://cie.asu.edu/ Abstract In recent years, a growing political emphasis has been placed upon the development of transformative assessment-driven reform in schools. There is global consensus on the value of assessment activities that are carefully designed to be consistent with desired learning outcomes, and which coherently connect learning theory, the curriculum, classroom activities and assessment. The consensus ends when there is debate between those who believe that schools produce more able students when they are faced with summative assessments and those who propose the implementation of formative assessment strategies in classrooms. This article presents a literature review which investigates the large-scale transformation of classroom practice in the Scottish region of the UK, known as Project1. The article begins with a summary overview of Project 1 and continues with a conceptual discussion of formative and summative assessment. The latter sections of the article delineate the key architectural principles underpinning formative assessment in practical settings before going on to present the professional evaluations of Project 1 by participating teachers. Keywords: formative, assessment, curriculum, summative, Scotland, UK C urrent Issues in E ducation V ol. 13 N o. 3 2 About the Author(s) Author: Ian Clark Affiliation: University of Washington Email: clarki@u.washington.edu Biographical information: Ian Clark is a research assistant at the University of Washington (Seattle) specializing in formative assessment and in the latter stages of his doctoral studies. He is a nationally qualified high-school and college level teacher in Business Studies and Economics and has worked in seven different nations where his focus has been the development and delivery of formative assessment programs in ESL/EFL settings. Project 1: Formative Assessment 3 In the past 30 years, educational researchers have given the significance of emphasis to various investigations on assessment practices and how they impact the quality of student learning and motivation, (Bloom et al, 1971; Crooks, 1988; Black & Wiliam 1998a, b; ARG, 1999, 2002; George Street Research, 2007). Arising from the proliferation of historical interest into assessment and learning the development of formative assessment strategies has gathered momentum on an international scale. One example of the remarkable expansion in awareness regarding the benefits of formative assessment is a 2005 Organisation for Economic Co-operation and Development (OECD) study which features exemplary cases from secondary schools in Canada, Denmark, England, Finland, Italy, New Zealand, Queensland in Australia, and Scotland. Despite widening global awareness regarding the impact of assessment strategies on learning, one review of assessment practices (in US schools) remarked that transformative ‘assessment practices are not common, even though these kinds of approaches are now widely promoted in the professional literature’ (Neill, 1997, p. 35-6). Project 1: Summary Overview There were two distinct phases to assessment reform in Scotland: Firstly, the development phase (2002-2004) of the program, which was strategically directed by the Assessment Action Group (AAG) and operationally managed by the AifL Program Management Group (APMG). Secondly, the implementation phase (2005-2008) was overseen by the APMG, which served as the main forum for liaison, encouraged cooperation and emphasized the importance of professionals working together in order to build communities of practice (SEED, Sept 2005). This article examines the crucial first phase which took place between the beginning of 2002 and the end of 2004. C urrent Issues in E ducation V ol. 13 N o. 3 4 The development of formative assessment strategies in the Scotland is connected by two contiguous assessment themes: assessment for learning (AfL) and assessment as learning (AaL). AfL focuses on the progress of the learner toward a desired goal, seeking to close the gap between a learner’s current status and the desired outcome. ‘This can be achieved through processes such as sharing criteria with learners, effective questioning and feedback’ (AAG/APMG 2002-2008). AaL is about reflecting on evidence of learning. It is a process ‘where pupils and staff set learning goals, share learning intentions and success criteria, and evaluate their learning through dialogue and self and peer assessment’ (AAG/APMG, 2002-2008). Both themes are integrated into a nation-wide project classified by the Scottish Executive Education Department (SEED) as 'Project 1' and stand together as the cornerstone of a wider programme known as Assessment is for Learning (AifL). 'An AifL school is a place where everyone is learning together. It is a place where assessment is part of learning and teaching without dominating them ... is about supporting classroom learning and teaching. It connects assessment and learning/teaching' (AAG/APMG, 2002-2008). The emphasis of this article is placed upon the early experiences of participating schools, which in partnership with the British Government began to develop the use of Project 1 formative assessment strategies across many hundreds of primary, special and secondary schools. 195 schools were involved in the initial phase of AifL (2002-2003). A key feature of developments during 2003-2004 was to increase the number of schools involved in the program, with a particular emphasis on building collaborative partnerships between schools called Associated Schools Groups (ASGs). ASGs range from individual schools to large groups of schools (for instance a secondary school and feeder primary schools) that work together on practitioner-based action-research projects that support the development of professional practice in assessment. By December 2004, local authorities' reports on the number of schools involved in AifL through ASGs had risen from Project 1: Formative Assessment 5 195 to 1,581 schools (approximately 56% of all schools in Scotland). The Scottish Executive Education Department (SEED) expressed the intention of making every possible effort to meet the target to 'ensure all schools are part of the assessment is for learning programme, by 2007' (June, 2005). Formative and Summative Assessment: Concepts and Issues New Ways of Seeing Formative assessment is described as: 'all those activities undertaken by teachers, and by their students in assessing themselves, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged. Such assessment becomes 'formative assessment' when the evidence is used to adapt the teaching work to meet the needs.' (Black & Wiliam, 1998b, p. 2). SEED advocate a re-thinking of the 20th century Vygotskian perspective because traditional teacher centered instructional methods designed to prepare students for frequent high-stakes summative tests were found to be ineffective when compared with collaborative learning systems which emphasize high quality interactions between teachers, students and parents/carers (AAG/APMG 2002-2008). The notion of the zone of proximal development (ZPD) (Vygotsky, 1978) is of central relevance to the formative assessment classroom entailing a significant change in classroom culture from one that is teacher-centered (expert/novice) to one that encourages equality and mutuality between students. Goos, Galbraith and Renshaw (2002) use the term ‘collaborative ZPD’ to denote equal status interactions between peers. However, high-quality interactions between students are rarely found in classrooms. In an oft-cited study from 1980 conducted by Galton, Simon and Croll, it was noted that the students frequently work in groups but not as groups. Some 20 years later Galton, Hargreaves, Comber, Wall and Pell (1999) conducted a replication study which found only a slight C urrent Issues in E ducation V ol. 13 N o. 3 6 increase in peer interaction; a finding more recently reiterated by Black (2005). Baines, Blatchford and Chowne (2007) observe, ‘teachers typically plan for their interactions with pupils, but not for interactions between pupils’ (p. 664). What is a 'Project 1' Classroom? A Project 1 AfL/AaL classroom is founded upon eight key formative assessment principles (AAG/APMG, 2002-2008; OECD, 2005) which demand high quality interactions between students, teachers and parents/carers. There are six key principles underpinning the AfL aspects of Project 1: students need to 1) be able to understand clearly what they are trying to learn, and what is expected of them; 2) be given immediate feedback about the quality of their work and what they can do to make it better; 3) be given advice about how to sustain improvement; 4) be fully involved in deciding what needs to be done next; 5) be aware of who can give them help if they need it and have full access to such help, and 6) to engage parents and carers in the learning process. The six principles of AfL combine with two further AaL assessment principles: students need to 7) be able to build knowledge of themselves as learners, and become meta-cognitive; 8) take more responsibility for their learning and participate more in the process of learning. When taken together AfL/AaL strategies breathe a new vitality into the concept of 'student centered learning' by re-positioning the students at the center of the learning process by building a network of constructive interactions which seek to link students directly to systems, teachers, parents/carers and their peers. In a Project 1 classroom, students can develop a deeper understanding of their learning when they are given opportunities to discuss the learning process with their teacher as their advisor and with their peers in a climate of equality and mutuality (AAG/APMG, 2002-2008). The quality of classroom interaction is a matter for grave consideration. Askew (2000) observes that: 'the characteristics of dialogue are equality, sharing, spontaneity, collaboration and Project 1: Formative Assessment 7 reciprocity. What I found interesting is that young people do not think such experiences are appropriate for the classroom where a particular view of behaviour is perceived' (p. 47). The issue of ineffective classroom interaction is comprehensively addressed by Black & Wiliam (1998b) who believe that, 'in terms of systems engineering, present policies in the U.S. and in many other countries seem to treat the classroom as a black box' (p.1). The 'black box' is an object for vital criticism because it functions primarily as a receptive system where, 'certain inputs from the outside--pupils, teachers, other resources, management rules and requirements, parental anxieties, standards, tests with high stakes, and so on--are fed into the box,' (Black & Wiliam, 1998b, p. 1). As proponents of the constructivist classroom, they are persuasive in voicing their concerns about such a system which is primarily designed to receive and decode external signals. Those who advocate AfL/AaL seek to go beyond a merely passive congregation of students (i.e. the notion of the pejorative 'black box') by encouraging co-operative and collaborative activities in the classroom. However, traditional learning arrangements continue to prevail, consequently pupils are often seen working in groups but not working as groups (Galton et al, 1980; Black, 2005). At present, the majority of classrooms exhibit a superficial culture of cooperative interaction (Black, 2005). However, research undertaken by the AAG (2002-2004) and the subsequent in-service experiences reported by the APMG (2005-2008) found compelling evidence which indicates the conditions for successful cooperation exist in classrooms. It was found that students are very responsive to the idea of co-operating with different people in their assessment. Of particular interest among their findings is confirmation that students liked to help each other with schoolwork, while either actually working and learning or in terms of assessment. It is therefore unsurprising that a key technique of the AfL/AaL program is effective dialogue among the central participants--the students. Furthermore, AfL/AaL necessarily entails cooperation C urrent Issues in E ducation V ol. 13 N o. 3 8 between all the participants in the learning process including teachers and parents/carers. The APMG found that students appreciate in-depth communication with teachers and parents/carers about their progress, and consider it to be a valuable part of the process of learning. McCroskey and Richmond (1992) state that 'all aspects of what happens in the classroom are contingent upon eliciting cooperation from every class member by employing sound principles of classroom management' (p. 44). The term 'management' in a AfL/AaL classroom context takes on a unique meaning because it entails the effective management of the learning process by developing activities which support learning through high quality interaction such as, discussion, feedback and goal setting. Formative Assessment Strategies in Practical Settings Participants in Project 1 schools were trained in the deployment of 16 formative assessment teaching strategies: Higher order questioning techniques; use of problem solving techniques; jot time (students are afforded an extended period of time to commit their ideas in writing before the beginning of an interaction); use of wrong answers; wait time; traffic lighting; group work and pair work; discussions; feedback as comments and not grades; oral feedback; sharing assessment criteria; peer assessment; redrafting of work; developing peer-peer communication skills (relational skills training); collaborative goal setting; reflective learning. A strategy worthy of some extended emphasis from the list provided here is that of traffic lighting. It may be used in various ways and is a powerful tool for self and peer assessment. For peer assessment, students may be required to traffic light each other’s explanations and justifications: green indicates work that exhibits explanations which surpass their own current level; orange indicates work that is comparable to their own, and red is used for work which is of lesser quality than their own. All students are then required to explain why they assigned those levels of recognition and provide constructive critique Project 1: Formative Assessment 9 and praise to each other. For example, a student may recognize that his/her partner has included an example or explanation from beyond the worksheet and award a ‘green light’. Traffic lighting also provides an opportunity for self-assessment and reflection upon one’s current status. For example, if students are required to learn certain science terms or language vocabulary they may clarify their own knowledge by assigning green to words they know and can use with confidence, orange for words in which they have some understanding and red for terms/words that they do not yet understand. Summative Assessment Revisited Research indicates that many of the problems with assessment arise from its summative nature (Black & Wiliam, 1998a; Harlen & Deakin Crick, 2003; ARG, 1999). A significant systemic obstacle frustrating the engagement of teachers with formative assessment strategies is the tension between the formative (assessment for learning) and summative (assessment of learning/assessment for qualifications) components of a program. This tension arises from the roles and responsibilities expected of teachers as both formative assessors, engaged in the process of student learning and as summative assessors required to behave as dispassionate judges of attainment (Knight & Yorke, 2003). Tests are usually viewed summatively by teachers, and their formative potential is largely overlooked (Morris et al, 1999) and 'while teachers are usually conscientious about marking student work they often fail to offer guidance on how work can be improved' (Hallam et al, 2004, ¶ 2.4). The Assessment Reform Group (ARG, 1999) at Cambridge University identifies various undesirable tendencies exhibited by classroom practitioners. These may be summarized as: a) a tendency for teachers to assess quantity of work and presentation rather than the quality of learning; b) a focus on marking and grading at the expense of providing advice for improvement, C urrent Issues in E ducation V ol. 13 N o. 3 10 which tends to lower the self esteem of pupils; c) a strong emphasis on comparing pupils with each other which demoralizes the less successful learners; d) teachers' feedback to pupils often attempts to serve managerial and social purposes rather than helping them to learn more effectively. The ARG go on to propose that improving learning through assessment depends on five key factors: i) a recognition of the profound influence instruction and specifically assessment has on the motivation and self esteem of pupils; ii) adjusting teaching to take account of the results of assessment; iv) the provision of effective feedback to pupils; iii) the active involvement of pupils in their own learning; v) the need for pupils to be able to assess themselves and understand how to improve. Particular emphasis was placed upon the sharing of learning goals with pupils; encouraging equality and mutuality between students; involving pupils in self-assessment and providing feedback which leads to pupils recognizing their next steps and how to take them. A consequence of summative assessment strategies is the creation of large numbers of disaffected students, particularly among lower achieving students (Harlen & Deakin Crick, 2003). This negative impact of current practice has been well documented. Both the ARG (1999, 2002) and the University of London's EPPI-Centre (2002) found that students show high levels of test anxiety when facing summative tests and much prefer other forms of assessment. High-stakes testing creates disaffection among students who do not achieve their full potential. As with all systemic phenomena, the outcomes are cyclical and reinforcing. Those students who exhibit disaffection often experience an ever-deepening spiral of self-defeating learning behavior and worsening motivational problems (Covington, 1984). The ARG (1999, 2002) found that the use of formative assessment techniques in the classroom mitigates the problems identified with summative testing and creates large positive increases in the level of student engagement. Consequently, choices need to be made between assessment techniques which support the process Project 1: Formative Assessment 11 of classroom learning by involving the student with the meta-process of learning and those that de- motivate students by frequently testing performance outcomes (Clark, 2008). Not only does the predominance of summative testing systems create disaffection on a wide-scale, it perpetuates its own existence - a phenomenon noted by Wiliam et al (2004) when they write: '... the introduction of high-stakes state-mandated testing, such as now exists in England and most states in the U.S.A, makes the effective implementation of formative assessment even more difficult. This is because ... attempts to maximise student and school scores appear to result in a lack of attention to the kinds of higher-order thinking involved in formative assessment' (p. 49). However, it should be noted that there is great importance in ensuring the constructive alignment between aims, learning and teaching processes and so summative assessment methods have been emphasized as methods to ensure effective learning (Biggs, 1996). The difference between summative assessment and formative assessment is that while constructive alignment can promote success on summatively assessed learning outcomes, formative assessment is specifically designed to support the learning process itself. AfL/AaL Curriculum: Four Key Architectural Principles If wide-scale educational innovation is to be successful, the strategic aims of the curriculum should be embodied in a comprehensive blueprint: the 'curriculum architecture'. It must also be understood that such a blueprint presents the structure of a curriculum and as such, it is an unresponsive or 'inorganic' entity. It is not designed to (and therefore does not) predict or limit the valid learning activities which arise in a classroom where formative assessment strategies are in use. While the AfL/AaL classroom system is built upon the blueprint, AfL/AaL is an organic process and a process of continuous adaptation which seeks to meet the learning needs of the C urrent Issues in E ducation V ol. 13 N o. 3 12 students. The AfL/AaL classroom is a place where students and teachers work together and 'feedback is used to modify the teaching and learning activities in which they are engaged' (Black & Wiliam, 1998b, p.2). The fundamental need for constant readjustment means that a prescription for teacher action, which seeks to specify the 'best' course of action given certain conditions is impossible even in principle (Wiliam, 2003). The central importance of the dynamic adaptation of instructional methods to the needs of individual students is therefore powerfully emphasized in the formative assessment classroom. Four interdependent principals govern the construction of an effective assessment infused curriculum; 1. coherence; 2. progression; 3. relevance; 4. engagement/motivation and personalization/choice. The following sub-sections will discuss these principles of curriculum architecture and how they orient teachers and students to the learning process. The Principle of Coherence The first curriculum design principle is coherence, or the alignment of program content with program aims to ensure that learning activities combine to form a coherent experience. To avert a misalignment, which can cause serious issues across time, there should be clear links between the different aspects of children's learning, including carefully considered opportunities for extended activities which further consolidate learning and draw different strands of learning together. Assessment should support the learning process, not drive it and assessments should be part of a holistic and balanced learning and teaching process. Learning behaviour is determined by the kind of assessments learners are confronted by and therefore assessments 'define what specific patterns of thinking we want students to demonstrate' (Sparks, 1999). If we want students to demonstrate higher order cognitive skills they need to be engaged in assessment activities which support that kind of learning. In order to support the learning process and realize learning goals, assessment of Project 1: Formative Assessment 13 content should support higher order learning goals by involving learners in the processes of their own learning. The curriculum architecture should underpin the process of orchestrating high quality learning encounters that provide students with the means for achieving desired learning outcomes (Knight, 2001). As such, curriculum coherence is in part conceptualized as a 'series of encounters between students' (Parker, 2003, p. 532) in collaboration with other stakeholders in the learning process. The principle of coherence was a vital aspect of the successful development of Project 1 in Scottish schools; it is the fundamental need for coherence the AAG/APMG refer to when they say, 'assessment is for learning connects assessment and learning/teaching,' (AAG/APMG, 2002-2008). Learner: Assessment is an important part of my learning. I understand how the assessment activities I do can help me learn. Teacher: In planning learning and teaching, I consider what information is needed to demonstrate that learning has taken place and how this evidence can be gathered from learning activities. In my planning I share assessment information with others involved in the learner’s learning across the curriculum. Together we ensure learning is planned on the basis of sound evidence about skills development. The Principle of Progression The second curriculum design principle is progression (AAG/APMG, 2002-2008). As part of learning and teaching, evidence needs to be collected which demonstrates and promotes the full breadth and depth of learning as experienced by learners and described by the outcomes. All children should have opportunities for a broad, suitably weighted range of experiences. The C urrent Issues in E ducation V ol. 13 N o. 3 14 curriculum should be organised so that they will learn and develop through a variety of contexts within both the classroom and other aspects of school life. There should be opportunities for children to develop their full capacity for different types of thinking and learning. As they progress, they should develop and apply increasing intellectual rigour, drawing different strands of learning together, and exploring and achieving more advanced levels of understanding. Learner: Assessment information provides me with a full picture of my learning. It shows me how much I have learned and how well and helps me to see pathways into the future of my learning both in and beyond school. Teacher: I design assessment in a way which enables the learner to demonstrate the breadth and depth of their learning. I use information on their progress and their learning goals to plan appropriate learning opportunities, building on what they have already achieved. The Principle of Relevance The third principle is relevance, and may be divided into two parts a) validity and b) consistency. Many students will learn whatever is necessary to get the grades they desire. Many summatively assessed programs are based on memorizing details. In those circumstances students will usually attempt to deploy a learning strategy that focuses on the retention of facts. The development of higher order cognition is a focal instructional strategy of Project 1 (Hallam et al, 2004). Formative assessment is fundamentally underpinned by dialogue (Black & Wiliam, 2006). It is therefore an assessment process that supports learning by stressing higher order thinking skills Project 1: Formative Assessment 15 through collaborative planning and organising of work; compromise; conflict resolution (Baines, Blatchford & Chowne, 2007); the sharing and augmentation of knowledge and feedback (Barron, 2000). a) Validity: A range of assessment approaches are used which are fit for purpose. This means that they should be valid and reliable – they should be well designed and capture the full range of learning and skills that are being developed; particularly high-order problem solving skills. Learner: I understand why I am being assessed and my assessment clearly reflects the learning I am trying to achieve. They provide me with valuable feedback about how much and how well I have learned so that I can plan next steps. Teacher: In planning learning activities, I consider the learning intentions and range of evidence I will gather to demonstrate this learning. I develop valid and reliable methods of assessment accordingly. Parent: I have accurate and succinct information about my child’s learning across a wide range of learning and skills. b) Consistency: Learners’ achievements are assessed fairly and consistently across the system, based on shared and commonly understood and consistently applied standards. Learner: I understand what good learning looks like and I know what I have to do to achieve my learning goals. Self and peer evaluation help me to develop my understanding of what I need to do. C urrent Issues in E ducation V ol. 13 N o. 3 16 Teacher: I know what quality in learning looks like. I share a common understanding of standards with my colleagues, and apply these with consistency across learners. The Principles of Engagement/Motivation and Personalisation/Choice The decision to develop formative assessment strategies in Scottish classrooms arose from the compelling research of Black and William (1998a, b) and subsequent reports on their work (ARG, 1999, 2002) which emphasized the potential of formative assessment strategies to create a large positive increase in the level of student engagement with the learning process. The principles of engagement and motivation are contiguous with those of personalisation and choice, and are highly significant features of an effective curriculum architecture. The arrangements for assessment should be responsive to individual needs and support particular aptitudes and talents in order to challenge and motivate all learners to develop to their fullest. The curriculum should recognize that learners progress in different ways and, based on their experiences go on to make different choices. Assessment should reflect these differences by giving each child increasing opportunities for exercising responsible personal choice as they move through their school career. Once they have achieved suitable levels of attainment across a wide range of areas of learning the choice should become as open as possible. There should be safeguards to ensure that choices are soundly based and lead to successful outcomes. The effectiveness of such safeguards is dependent on the ability of individual teachers to understand the principles of assessment for learning and apply them in the classroom in order to support the fullest development of learners, while minimising distortions in learning (AAG/APMG, 2002-2008). Project 1: Formative Assessment 17 The notion of personalization and choice is by no means new. For more than 30 years researchers have been aware that relating new knowledge to learners' existing understandings of the world is an effective way for learners to acquire deeper meaning from new information. Since personalization of the curriculum emerged as a valid pedagogical principle, it has been discussed as a means of motivating learners by incorporating their goals and choice of topics into a curriculum, particularly for addressing values (Howe & Howe, 1975), and as a model of behaviour modification for disruptive students (Mamchak & Mamchak, 1976). Some 35 years later the Project 1 classroom system seeks to fully value and recognize learners' achievements by acknowledging the individuality of personal experience and the accommodation of learners' needs in the instruction. Learner: I am involved in planning my learning and setting challenging goals for myself. Assessment information helps me to see how far I have come and what I need to do to achieve my goals and motivates me to set new goals. I am assessed in ways that allow me to demonstrate and recognise the full range of my achievements. Where appropriate, I am involved in decisions about when and how this assessment will take place. Teacher: The learning, teaching and assessment activities I plan create opportunities for dialogue to help learners set and achieve challenging goals based on high quality feedback on their progress. In planning teaching learning and assessment, I consider the needs of the wide needs of learners. I plan an appropriate range of activities to give learners the opportunity to demonstrate their learning and provide feedback accordingly. A Practitioner Evaluation of 'Project 1': A Literature Review C urrent Issues in E ducation V ol. 13 N o. 3 18 This section of the article features the findings of two separate research papers which evaluate the crucial development stages of Project 1 (2002-2004) in UK schools: 1) Hallam, Kiston, Peffers, Robertson and Stobart (2004) investigated Project 1only and; 2) Condie, Livingstone and Seagraves (2005) examined the impact of Project 1 alongside various other projects within the AifL programme. It may be of interest to note that teachers exhibited a consistently greater interest in responding to questions regarding Project 1 than any other project included in Condie’s research. At the behest of SEED, Hallam et al undertook to investigate effective classroom approaches to formative assessment (October 2004). Their investigation is founded upon the earlier (and continuing) research of Black and Wiliam (1998a, b). They draw various sources of data directly from school settings: teachers' agreements, action plans, program dairies, case study reports, relevant school self-evaluations and field visits. A primary function of the early field visits was to introduce participating staff to the use of a rating scale used to quantify the perceived effectiveness of Project 1 to a range of outcomes. All data was analyzed to consider the extent of change across the early period of the innovation in the following areas: student learning, motivation and behavior; student awareness of assessment; attainment; classroom practice; teacher beliefs and attitudes; teacher understanding of assessment; parental/carer involvement in their child's education. Student Learning, Motivation and Behaviour Hallam et al (2004) found strong positive consensus among teachers (n=72) regarding enhanced involvement, motivation and confidence in their students. Generally, there were fewer comments regarding behavior. Hallam et al (2004) speculate that 'this may have been in part, because the pupils participating in the project were on the whole already well behaved (¶ 4.7.4).' A Project 1: Formative Assessment 19 slightly later SEED commissioned study into AiFL and the effectiveness of formative assessment strategies was undertaken by Condie et al in December 2005. The reluctance of teachers to comment on changes in behavior was a remarkable feature here as well. Condie et al (2005) used questionnaires as a part of their method and issued the same basic template in two phases spaced 16 months apart. 44 teachers and 21 head-teachers participated in phase 1; 56 teachers and 26 head-teachers participated in phase 2 of the survey. In both phases, 21% of participating teachers declined to evaluate the statement: 'Pupils have shown improved classroom behavior'. A further interesting feature of Condie's research is that while 73.5% of respondents strongly agreed or agreed (SA/A) with the statement on discipline in phase one, only 46% did so in phase two (2005, ¶ 4.4). The marked decline of positive responses in phase two of Condie's questionnaire was explained by head-teachers' as an indication of the generally good discipline found across participating classes at the time, and so what may not be observed may not be reported. Condie's report did not seek to examine if the good discipline reported by head-teachers at the time of phase two was the outcome of earlier Project 1 interventions in the classroom. The responses to the Condie et al questionnaire concur with the favorable indications received by Hallam et al, regarding learning, confidence and motivation: Project 1 Questionnaire (Condie et al, 2005, ¶ 4.4) Phase 1 Phase 2 SA/A SA/A The developments enhance the learning of all pupils 75.00% 67.00% Pupils have increased confidence and show greater self-esteem 70.50% 72.00% Pupils are more motivated toward learning 70.00% n/a C urrent Issues in E ducation V ol. 13 N o. 3 20 Pupils have become more actively involved in the learning process 81.00% 82.00% Student Awareness of Assessment An important aspect of Project 1 was the effect on students' meta-cognitive skills. Participating students became more aware of their learning needs and what they had to do in order to make progress (Hallam et al, 2004). For example, Hallam reports a strong positive reaction from students after the use of a feedback sheet designed to develop their insights into their writing skills (2004, ¶ 4.7.5). Condie solicited the perceptions of teachers on this matter: Phase 1 Phase 2 SA/A SA/A Pupils are better equipped to assess their own learning 79.50% 78.00% Pupils are able to set realistic targets 66.00% 66.00% Pupils are developing skills in peer assessment 66.00% 77.00% Attainment Attainment was an area in which teachers were reluctant to offer comment. Hallam states that drawing conclusions about Project 1's impact on attainment would be premature because of the early development stage of the project. They do however report that some teachers were very positive about the impact of formative assessment strategies, while some in policy positions (head- teachers and local government officials) were more reserved in their praise (2004, ¶ 4.7.8). Condie posed the following statement on attainment to teachers: ‘I have evidence that pupil attainment has Project 1: Formative Assessment 21 improved through project activities.’ Only 32% of respondents in phase 1, and 28% in phase 2 strongly agreed or agreed with the statement. The low approval rating afforded to this statement is in part explained by a marked reluctance among participating teachers to commit any form of response on this issue: 44% of teachers in phase one and 42% in phase two chose to offer no response. It is therefore impossible to ascertain if they inclined towards agreement or disagreement on the issue of attainment. Classroom Practices 'A major outcome...was the change in classroom practices which increased the active engagement of pupils, who were encouraged to take ownership of their learning rather than being the passive recipients of the delivery of curriculum' (Hallam et al, 2004, ¶ 4.7.1). Hallam reported that relationships with the students 'blossomed' and it became possible to bring forward difficulties without negative impact (2004, ¶ 4.7.1). Significantly, Hallam found that the students were generally positive about the impact of the project. Similarly, Condie et al (2005) requested that teachers respond to the following statement: ‘Pupils themselves report positive views of project activities.’ Practitioner responses were favourable, with 62.5% (in phase 1) and 65% (in phase 2) of teachers strongly agreeing or agreeing with the statement. It can be seen that there is a consensus which runs throughout the responses to the Condie research questionnaire. Teachers consistently indicate that students are more motivated, more involved and better equipped to assess their own learning. In addition, students are positive about project activities, show greater confidence and are generally able to set realistic learning targets. Teacher Beliefs and Attitudes The AfL project tapped into the existing beliefs and attitudes of many participating teachers. Hallam et al (2004) observe that: C urrent Issues in E ducation V ol. 13 N o. 3 22 'The nature of the project itself with its focus on pupil learning and the enhancement of teaching resonates with teachers, taking them back to fundamental pedagogical principles...It's success in delivering improved pupil motivation and engagement provided teachers with immediate positive reinforcement for continuing development, implementation and evaluation of the strategies' (¶ 1.5.11). Indeed, 100% of respondents assigned a rating of VS/S to the project improving their motivation (Hallam et al, 2004, ¶ 1.5.18). Condie et al reported numerous positive descriptions: 're-energised, satisfied, confident, renewed enjoyment' (2005, ¶ 4.4). It is evident that the students are not the sole beneficiary of a movement from a teacher-centered pedagogy to one which places the student and their learning needs at the heart of teaching. Teachers themselves report it to be an essential opportunity to reflect on their existing attitudes on classroom practice (Hallam et al, 2004). Hallam et al present a particularly meaningful remark from a participating teacher: 'Even in terms of where I was in education 10 years ago, you can see that is a shift, and it’s almost as if now the focus is on the process rather than the product' (¶ 4.8.2). This is a very significant insight into the essential focus of formative assessment being one of process and not oriented to performance. Both Hallam and Condie gathered extensive evidence of the impact of Project 1 on the personal and professional development of teachers, many of which used the opportunity to reflect on their past understandings of teaching and learning and compare them to their current practices. Teacher Understandings of Assessment Hallam's report on Project 1 stresses the need for a careful clarification of the meaning of formative assessment, followed by the embedding of formative assessment strategies in practice. 'Teachers need to understand the underlying principles rather than have a superficial knowledge of strategies' (Hallam et al, 2004, ¶ 1.8). Condie et al discovered that teachers and head-teachers Project 1: Formative Assessment 23 participating in the initial development phase of Project 1 were troubled by a lack of understanding regarding the underlying theories and principles of formative assessment (2005, ¶ 4.4). Both studies elicited feedback which indicated that a much greater depth of understanding was required by participating staff prior to the commencement of work in schools. However, as the development phase of Project 1 continued, teachers reported the evolving understanding of how assessment and learning connect as a very powerful learning experience (Hallam et al, 2004, ¶ 4.8.1). By the latter stages of project development, 100% of participating teachers in Project 1 evaluated the project as VS/S at improving their understanding of assessments (2004, ¶ 1.5.18). In terms of the overall effect of the project upon teachers, the Hallam et al report reveals that the project had a significant impact and led to: • the understandings that formative assessment takes time and requires long-term commitment; • improved feedback to students; • a deeper understanding of assessment; • teachers applying formative assessment strategies beyond the prescribed boundaries of the project; • teachers becoming increasingly aware of how students learn and their learning needs. Parent/Carer Involvement One of the key aims of the wider multi-project AiFL innovation is to involve parents and carers more actively in the process of their children's learning. This overall aim is therefore equally significant when considering Project 1 and the task of developing formative assessment strategies in schools. The importance of parental/carer inclusion is confirmed by Townsend (1997),who C urrent Issues in E ducation V ol. 13 N o. 3 24 believes that effective schools are those which welcome parents by engaging them and involving them in the widest range of school activities; most crucially those concerning their child's development. Although a key aim of Project 1, the level of involvement by parents and carers by the end of the development phase was disappointing, particularly when compared to the other more successful aspects of the project discussed above. Only 23% of teachers indicated that parental/carer involvement had increased as a result of the project (Hallam et al, 2004, ¶ 1.5.18). Condie et al found that only 38% of head-teachers reported that involvement practices had changed as a result of the project. Indeed, some schools neglected to inform the parents about the project (2005, ¶ 4.4). Condie determined six key themes by analyzing teachers' comments about the project. From a total of 410 recorded thematic references there were only 22 mentions of better links to parents and carers (2005, ¶ 4.4). It is apparent that the space between the school system and parents/carers had not been sufficiently explored by the end of the project development stage. It is to be hoped that the level of 'public engagement' exhibited by schools increases in order to create communities of practice which work together in the spirit of trust and under the agreement of mutual accountability. It is this kind of relationship which brings into existence legitimate partnerships (Marx 2000). Project 1: Challenges Encountered in the Development Stage (2002-2004) Instigating change in schools is a highly complex process. Any complex system can be broken down into sub-parts and various authors have sought to examine the process of educational change management in terms of categories. For example, Johnston and Neidermeier (1987) see change in terms of 3 broad categories of variables: a) structural (leadership, finance, human resources); b) procedural (time, organization, technical assistance) and c) attitudinal, primarily relating to negative responses to top-own policy making and the existence of internal politics. Project 1: Formative Assessment 25 Knoop (1987) suggests four key areas of change: i) administrative (policies and objectives); ii) curriculum/program; iii) human resources (staff interactions, beliefs, expectations; and iv) teaching and learning, where Knoop highlighted group interactions between teachers and students and methods of teaching pertaining to classroom relations and communication. Fullan (1991) confirms that the actual process of educational change management is a highly complex process requiring the strong commitment of principals and external policy makers. It can be seen that the factors which ensured the success of the Project 1 (stated earlier) take these change variables into full account. The careful arrangement of consultative networks by SEED led to project success despite the reputation of teachers for being inherently stubborn when faced with innovation, in part due to their strong sense of acculturation and continuity. Hallam et al report very positive attitudes by schools to the difficulties faced during the developmental stage of Project 1. When asked about the difficulties encountered schools identified relatively few, preferring instead to see problem solving as developmental and positive (2004, ¶ 1.5.6) – an indication that the overwhelming majority of participating teachers and principals were committed to the project from the outset. It was in this spirit that various developmental challenges were reported by participating schools. Hallam et al (2004) found that some teachers, even those who grasped the underlying principles of formative assessment, were unclear on what they were supposed to do in their classrooms. One teacher explains that this barrier to change exists because, 'teachers are used to attending various in-service courses where they are told exactly what to do in the classroom' (Hallam et al 2004, ¶ 4.6.1). Consequently, participating teachers expressed concerns about the movement from concept to operationalization. Some felt that more direction was required on the differentiation of strategies taking into account the wide range of attainment in some classes (Hallam et al, 2004). These are valid concerns, not least when one considers issues of C urrent Issues in E ducation V ol. 13 N o. 3 26 choice and equity, which are fundamental themes of any credible curriculum architecture. All students, no matter what their ability, gender, ethnicity or economic status, need to be supported equally as they gain mastery of the basic curriculum and so it is essential that schools 'identify and remove physical and programmatic barriers that produce disproportionality in academic achievement' (OSPI website). This situation must exist in such a way that students do not perceive bias or discrimination in their treatment. If it does not, it is to betray sensitivity and trust and destroy the central pillar supporting the AfL/AaL classroom: cooperation (ARG 2002). In addition, some teachers found the integration of formative and summative assessments very challenging (George Street Research [GSR], 2007). Such initial uncertainty among some teachers is not indicative of shortcomings in their preparation by SEED. Such preparations simply did not take place and traditionally precise instruction regarding the specifics of implementation were carefully avoided. Wiliam, Lee, Harrison and Black (2004) explain the rationale behind this non-prescriptive approach, 'the central tenet of the research project was that if the promise of formative assessment was to be realised, traditional research designs, in which teachers are 'told' what to do by researchers would not be appropriate (p. 50).' As the project progressed, the lack of clarity exhibited by some teachers at the beginning evolved into an advantage as they appreciated that it had compelled them to focus on their own teaching materials and devise their own action plan (Hallam et al, 2004) in effect they were adjusting to the atmosphere outside of the 'black box' (Black & Wiliam, 1998b). However, in a 2007 review of the entire AiFL project research conducted by George Street Research found that awareness and understanding of AifL was not firmly embedded across teachers in Scotland and understanding was not clear by teachers in terms of the principles underpinning assessment strategies (GSR, 2007). As the government funded part Project 1: Formative Assessment 27 of the AiFL project drew to close in late 2007 the quest for fine-grained understandings of formative assessment principles among teachers continued. Other challenges arising from the development of formative assessment strategies included: • Some staff and students didn't like to be videoed or lacked the technical support to expedite this process; • Teachers spoke of the need for self awareness to avoid reverting to old habits; • Tensions between formative and summative assessment demands; • Concerns about funding for further development; • Competing agenda and external accountability demands. Many teachers identified their primary concern as insufficient time to engage in the process of strategic development (planning, preparing, reflecting, reporting) (Condie et al, 2005). Time was also a concern when it came to covering the curriculum, with many teachers expressing concern that the material was being covered at a slower pace. These initial concerns were to some extent mitigated by the emerging quality of work and learning as the project progressed (Hallam et al, 2004). Project 1: Factors Contributing to Success Both Hallam et al (2004) and Condie et al (2005) found the development stage of Project 1 to be a success from which lessons can be learned and applied to future projects which seek to develop innovative changes to classroom practice. It is clear that the development of formative assessment strategies in the classroom stimulated teachers personally and improved them professionally. Hallam et al attribute various factors to the success of the project: • Participating teachers had time to plan, prepare, reflect and evaluate. This was made possible C urrent Issues in E ducation V ol. 13 N o. 3 28 by the serious intent of SEED to instigate change at a policy level; • Once teachers became accustomed to the idea of exploring formative assessment strategies in their classrooms, the action-research elements of the project became very meaningful; • Head-teachers, senior management team members and teachers were committed to the project from the outset; • Teamwork between all stakeholders in the project; • Regular meetings and project forums to reflect upon project efficacy and to plan next steps; • Resources (supply teachers, I.T, instructional materials, time and funding) were provided which supported the project and the strategies; • The incorporation of the project into the school development program. Conclusion The development of formative assessment strategies in the Scottish region of the UK was a highly complex process that required the political commitment of government to be matched by that of participating teachers and principals. Even when this collaboration was secured and the processes of change management among stakeholders carefully planned, conceptual difficulties persisted beyond the 2002-2004 development phase of the AiFL project (GSR, 2007). The embedding of innovative pedagogical principles is beset by challenges and tensions. As Black & Wiliam (2005) observe, the effective integration of formative and summative assessment will require a different change- management strategy depending on national circumstances, and in some cases may be very challenging indeed. Innovations in education invite such challenges and are inevitable (Fullan, 1991). From the out-set, the constructive problem-solving attitude exhibited by participating schools in Scotland was Project 1: Formative Assessment 29 a large contributory factor to the successful development of formative assessment strategies (Hallam et al, 2004). A primary source of conceptual challenge associated with Project 1 was the acculturated preference of teachers for prescriptive and precise instructions on how to implement new classroom practices (William et al, 2004; Hallam et al, 2004). Despite the challenges discussed in this article, teachers responded very positively on an evaluative rating scale: 100% of respondents reported that the project was ‘very successful’ or ‘successful’ (VS/S) in increasing teacher understanding of the role of assessment and in improving teacher motivation. Further, 91% of respondents indicated that Project 1 was VS/S in changing classroom practices; 92% indicated the project was VS/S at changing assessment practices; 94% the extent of questioning; 95% in increasing the level of discussion and 97% of respondents assigned a VS/S rating to an improved teacher focus on the learning needs of students (Hallam et al, 2004). If formative assessment interventions are to be successful on a large and long-term scale certain conditions should exist: (i) dedicated political support at all levels of government; (ii) a clear and compelling expression of the conceptual framework which underpins formative assessment and continuing support for practitioners beyond the development phase (iii) close collaboration between all stakeholders (teachers, administrators, parents/carers and students), who work together to engage students with the process of their own learning; (iv) a climate of change management in which practitioners approach obstacles as constructive and necessary challenges; (v) the alignment of summative and formative assessment activities so that they work in concert to support and evaluate learning. Finally, at the most fundamental level formative assessment practices are characterized by positive learning interactions and dialogue. As such, an important strand of future research resides in developing fine-grained understandings about high-quality C urrent Issues in E ducation V ol. 13 N o. 3 30 classroom dialogue between teacher/student and between peers, and how they support productive learning in practical settings. References American Psychological Association. (2001). Publication Manual of the American Psychological Association (5th ed.). Washington, DC: Author Askew, S. (2000). Feedback for learning. London: Routledge/Falmer. AAG/APMG (Assessment Action Group /AiFL Programme Management Group). (2002 -2008). AifL--Assessment is for learning. Retrieved from http://www.ltscotland.org.uk/assess Assessment Reform Group. (1999). Assessment for learning: Beyond the black box. Cambridge University: Cambridge School of Education. Assessment Reform Group. (2002). Assessment for learning: 10 principles. Cambridge University: Cambridge School of Education. Project 1: Formative Assessment 31 Baines, E., Blatchford, P., & Chowne, A. (2007). Improving the effectiveness of collaborative group work in primary schools: Effects on science attainment. British Educational Research Journal, 33 (5), 663-680. Barron, B.J. (2000). Achieving coordination in collaborative problem solving groups. Journal of the Learning Sciences, 9(4), 403-436. Black, P. (2005). Formative assessment: Views through different lenses. Curriculum Journal, 16 (2), 133-135. Black, P. (2007). Full marks for feedback. In: Making the grade. Journal of the Institute of Educational Assessors. Spring 2007, 18-21. Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5(1), 7-74. Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. London: School of Education King's College. Black, P., & Wiliam, D. (2005). Lessons from around the world: how policies, politics and cultures constrain and afford assessment practices. The Curriculum Journal, 16(2), 249-261. Black, P., & Wiliam, D. (2006). Assessment for learning in the classroom. In J. Gardner (Ed.), Assessment and learning (pp. 9 -25). London: Sage. Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3), 347-364. Bloom, B. S. (1994). Taxonomy of educational objectives, the classification of educational goals, handbook I: Cognitive domain. In L. W. Anderson & L. A. Sosniak (Eds.), Bloom's taxonomy: A forty-year retrospective. Chicago: University of Chicago Press. Clark, I. (2008). Assessment is for Learning: Formative assessment and positive learning interactions. Florida Journal of Educational Administration and Policy (FJEAP), 2(1) pp. 1 – 16. Retrieved from http://www.coe.ufl.edu/leadership/fjeap/V2/FJEAP_Fall%202008_Clark- 3.pdf Condie, R., Livingstone. K., & Seagraves, L. (2005). Evaluation of the assessment is for learning programme: Final report and appendices. The Quality of Education Centre, University of Strathclyde. Retrieved from www.scotland.gov.uk/Publications/2005/12/0210792641/26428 Covington, M. (1984). The self-worth theory of motivation: Findings and implications. Elementary School Journal, 85(1), 5-20. Crooks, T. (1998). The impact of classroom evaluation practices on students. Review of Educational Research 58, 438-481. C urrent Issues in E ducation V ol. 13 N o. 3 32 EPPI-Centre. (2002). A systematic review of the impact of summative assessment and tests on students' motivation for learning. Social Sciences Research Unit. Institute of Education (IoE), University of London. Retrieved from http://eppi.ioe.ac.uk/cms/Default.aspx?tabid=108 Fullan, M. (1991). The new meaning of educational change. New York: Teachers College Press. Galton, M.J., Simon, B., & Croll, P. (1980). Inside the primary classroom. London: Routledge & Kegan Paul. Galton, M.J., Hargreaves, L., Comber, C., Wall, D. & Pell, A. (1999). Inside the primary classroom: 20 years on. London: Routledge. George Street Research (GSR). (2007). Evaluation of the status of assessment of learning in Scotland 2006-2007. Learning and Teaching Scotland (LTS). Retrieved from http://www.ltscotland.org.uk/publications/e/publication_tcm4509473.asp Goos, M., Galbraith, P., & Renshaw, P. (2002). Socially mediated metacognition: Creating collaborative zones of proximal development in small group problem solving. Educational Studies in Mathematics, 49 (192-223) Hallam, S., Kiston, A., Peffers, J., Robertson, P., & Stobart, G. (2004). Evaluation of Project 1 of the AiFL Development Programme: Support for professional practice in formative assessment. Institute of Education (IoE), University of London. Retrieved from www.scotland.gov.uk/Publications/2004/10/19947/42988 Harlen, W., & Deakin Crick, R. (2003). Testing and motivation for learning. Assessment in Education. 10(2), 169-207. Howe, L. W., & Howe, M. M. (1975). Personalizing education: values clarification and beyond. New York: Hart Pub. Co. Johnston, A., & Niedermeier, H. (1987). State sponsored education reform in the 1980s: Schools as policy junkyards. Educational Planning, 6 (3), 16-27. Knight, P.T. (2001). Complexity and curriculum: A process approach to curriculum-making. Teaching in Higher Education, 6(3), 369-381. Knight, P., & Yorke, M. (2003). Assessment, learning and employability. Society for Research into Higher Education. McGraw-Hill International. Knoop, R. (1987). Bringing about change. Education Canada, 27 (1), pp. 14-36. McCroskey, C., & Richmond, V. (1992). Communication, control, and concern. Lawrence Erlbaum Associates: Hillsdale, NJ. Project 1: Formative Assessment 33 Mamchak, P. S., & Mamchak, S. R. (1976). Personalized Behavioral Modification: Practical Techniques for Elementary Educators. West Nyack, NY: Parker Publishing Company, Inc. Marx, G. (2002). Ten Trends: Educating children for a profoundly different future. Center for Public Outreach. Educational Research Service. Morris, P. (1999). The project on feedback and assessment: Final report. Hong Kong: Hong Kong University. Neill, D. (1997). Transforming student assessment. Phi Delta Kappan, 78, 35-36. Organisation for Economic Co-operation and Development, (OECD). (2005). Formative Assessment: Improving learning in secondary classrooms. Centre for Educational Innovation and Research, OECD Pub. Parker, I. (2003). Reconceptualising the curriculum: From commodification to transformation. Teaching in Higher Education 8(4), 529-543. Scottish Executive Education Dept (SEED) (June 2005), Circular No.2 June 2005: Assessment and Reporting 3-18. Qualifications, Assessments and Curriculum Division. Retrieved from http://www.scotland.gov.uk/Publications/2005/06/2393450/34518 Scottish Executive Education Dept (SEED) (September 2005), Information Sheet on AiFL. Retrieved from http://www.scotland.gov.uk/Publications/2005/09/20105413/54156 Sparks, D. (1999). Journal of Staff Development, 20(2). National Staff Development Council. Retrieved from http://www.nsdc.org/library/publications/jsd/stiggins202.cfm Superintendant of Public Instruction (OSPI), Joint Policy: Recommendations for implementation, Retrieved from http://www.k12.wa.us/equity/jointpolicy.aspx Townsend, T. (1997). What makes schools effective? A comparison between school communities in Australia and the USA, School Effectiveness and School Improvement. 8(3), 311-326. Wiliam, D. (2003). The impact of educational research on mathematics education. In A. Bishop, M. A. Clements, C. Keitel, J. Kilpatrick, & F. K. S. Leung (Eds.). Second international handbook of mathematics education. Dordrecht, Netherlands: Kluwer Academic Publishers. Wiliam, D., Lee, C., Harrison, C., and Black, P. (2004). Teachers developing assessment for learning: Impact on student learning. Assessment in Education, 11(1), 49-65. C urrent Issues in E ducation V ol. 14 N o. 1 34 Current Issues in Education http://cie.asu.edu Volume 13, Number 3 ISSN 1099-839X Authors hold the copyright to articles published in Current Issues in Education. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. Permission is hereby granted to copy any article, provided CIE is credited and copies are not sold. Editorial Team Executive Editors Lori Ellingford Andrew Darian Section Editors Krista Adams Hillary Andrelchik Meg Burke Douglas Deiss Miriam Emran Tracy Geiger Sarah Heaslip Melinda Hollis Afzal Hossain Jeffery Johnson Seong Hee Kim Younsu Kim Alaya Kuntz Angeles Maldonado Carol Masser John Michael William Mitchell Elizabeth Reyes Lindsay Richerson Rory Schmitt Tapati Sen Jennifer Shea Kara Sujansky Melissa Tarango Andrew Tesoro Jill Wendt Jennifer Wojtulewicz Copy Editor Lucinda Watson Faculty Advisers Gustavo E. Fischman Jeanne M. Powers Debby Zambo