Emphasis in the original 1 Knowledge and judgement for assessing student teaching: a cross-institutional analysis of teaching practicum assessment instruments Lee Rusznyak and Carol Bertram Abstract Teaching practicum (TP) assessment instruments provide insight into the nature of the knowledge that the university expects university-appointed tutors and school-based supervising teachers to have in order to make fair judgements about a student’s teaching competence. This paper presents a comparative analysis of the TP assessment instruments used during 2012 by five South African universities offering initial teacher education. It describes the grounds upon which the comparative analysis was done, and offers a qualitative analysis of the knowledge base that the assessors of student teaching are assumed to have. We find that the structure and criteria of some TP assessment instruments tend to construct the assessment of student teaching as straight-forward exercise in verifying that certain technical requirements are met. In contrast, we show how others use structure and criteria potentially to enable a more professionally based judgement of the competence of student teaching. Introduction South African universities offering teacher education have the responsibility of verifying that qualifying teachers are sufficiently competent to assume responsibility for teaching a class of learners. It is on the basis of completing their initial teacher education qualification that teachers may register with the South African Council of Educators (SACE) and obtain a professional licence to teach. To this end, legislation requires that student teaching is ‘formally supervised and assessed’ during school-based teaching practicum sessions (Department of Higher Education and Training [DHET], 2011, p.23). Ideally,1 the university tutors and supervising teachers who observe student teaching are expected to understand the logic of the choices a student teacher makes 32 Journal of Education, No. 60, 2015 during the planning and delivery stages: how she has chosen to represent knowledge in particular ways, engage with learners and respond accordingly, as the lesson unfolds in sometimes unpredictable ways. In order to do this, the assessors also draw on their subject and pedagogic knowledge to verify the competence of a student’s teaching prior to her graduation. Teaching practicum (TP) assessment instruments purport to provide school-based mentor teachers and university-appointed staff (henceforth called supervising teachers and university tutors respectively, or assessors jointly) with a set of assessment criteria which reflect the knowledge and skills expected of student teachers during their practicum sessions. These criteria generally intersect with a rating scale which is used by the assessors to signify the level of competence a student’s teaching as attained. These instruments intend to support assessors reach a clear, transparent and standardised decision regarding the level of competence a student’s teaching has attained so that the university has confidence in the result decision (Rusznyak, 2011). Although there have been several studies of the nature of the practicum across South African institutions offering initial teacher education (e.g. Chief- Directorate: Teaching and Learning Development, 2010; Parker and Deacon, 2005; Reddy, Menkveld and Bitzer, 2008), these have focused primarily on the structure of the practicum and logistics of student placement, with little attention to what is assessed and how assessment decisions are made. A national review of South African initial teacher education programmes identified a pervasive “lack of a sectoral consensus” regarding what counts as quality in relation to the teaching practicum, manifesting in a “lack of common understanding of . . . [teaching practicum] assessment rubrics”. It is not only in South Africa that the assessment of student teaching is regarded as “contentious” and “complex” (Reddy, Menkveld and Bitzer, 2008, p.146). In the United States, Raths and Lyman (2003) argue that many incompetent students graduate, because “it is difficult to make a high-stakes judgement about an individual student armed only with vague decision rules” (p.208). They thus suggest that it is important to define what constitutes incompetence in student teaching. Another study in New Zealand found that some practicum assessors disregarded formally articulated criteria to make decisions about the teaching performances of student teachers in relation to “what they personally believed to be the important elements of a performance against standards they personally deemed appropriate” (Hawe, 2002, p.103). Hawe (2002) insists that there is a significant difference between making an assessment judgement of the competence of student teaching based on whimsical, personal preference and a professionally-based judgement that is informed by evidence Ruszynak and Bertram: Knowledge and judgement . . . 33 and has a rational basis. As gatekeepers to the profession, teaching practicum assessors need to have clearly articulated and principled reasons for why they deem student teaching to be as competent or not. It is not surprising then that the assessment of student teaching has been described as “one of the major challenges facing practicum supervisors and teacher educators in general” (Reddy et al., 2008, p.155). In this paper we analyse the TP assessment instruments used by five South African universities in 2012. Our study rests on an assumption that the criteria and rating scales of these assessment instruments provides insight into knowledge that assessors are expected to use when they make judgements about students’ teaching competence. We use two concepts as the lenses through which to analyse the five TP assessment instruments. The first concept is that of teacher knowledge, which we analyse as either general (that is, general pedagogic knowledge that is applicable across all phases, content subjects and school contexts) or specialised (that is, pedagogic content knowledge that is specialised to particular subjects, phases and learners). An engagement with knowledge for teaching is important because reasoned and intelligent action rests on a knowledge base (Muller, 2012; Winch, 2014). The second concept that we will use is that of judgement. We analyse the extent to which it is sufficient for assessors to verify that students are able to implement a set of ‘tips for teachers’ in a technical way or whether the TP assessment instruments demand that the assessment of student teaching considers the extent to which they use their teacher knowledge to make contextually appropriate judgements. The first section of the paper reviews the literature on the nature of professional knowledge for teachers and different ways in which the links between knowledge and judgement are understood in initial teacher education programmes. We then engage with the challenges of assessing student teaching within the South African context. We describe the methods we used to analyse the teaching practice assessment instruments and present the findings using the concepts of knowledge and judgement to structure the presentation of the analysis of the data. 34 Journal of Education, No. 60, 2015 Knowledge base for teachers Debate about the nature of the knowledge that teachers need was brought to the fore by the work of Lee Shulman who formulated his highly influential categories of the knowledge bases for teaching in 1986 (Shulman, 1986; Shulman, 1987). Since Shulman’s seminal work, research in the field of teacher knowledge has developed steadily (Ball, Thames and Phelps, 2008; Banks, Leach and Moon, 2005; Gess-Newsome and Carlson, 2013; Verloop, Van Driel and Meijer, 2001). Grossman (1990) reduced Shulman’s original seven categories of teacher knowledge to four domains, namely contextual knowledge, content knowledge, general pedagogic knowledge (GPK) and pedagogic content knowledge (PCK). In Grossman’s model, knowledge of context refers to the teacher’s knowledge of the milieu in which she teaches, including the curriculum to be covered, the school policies and environment, and the learners’ contexts. The second domain, Content Knowledge, comprises both the propositional knowledge and the procedural knowledge that the teacher has of the subject she is teaching. The domain of General Pedagogic Knowledge (GPK) refers to a teacher’s knowledge of a range of lesson planning, classroom organisation and assessment strategies, as well as her ability to use these strategies effectively. GPK is the general classroom knowledge that is shared by teachers irrespective of their subject or phase specialisations. The fourth domain, Pedagogic Content Knowledge (PCK), is understood as the teacher’s knowledge of how to teach specific content by using conceptually sound explanations, analogies, models or activities that are both accessible to learners and address their common misunderstandings about the topic. In the South Africa teacher education sector, there is much contestation about the relative importance that these four knowledge domains should play in preparing competent teachers. We identify three main orientations, namely programmes that foreground the importance of general pedagogic knowledge; those that foreground specialised content and pedagogic knowledge, and those that emphasise the importance of contextual knowledge for teaching. We consider each of these in turn. First, programmes that focus more on the development of teachers’ general pedagogic knowledge support the idea that pedagogical knowledge and skills are applicable across all contexts, phases and subject domains place strong emphasis in developing students’ general pedagogic knowledge (GPK) (Reeves and Robinson, 2014). In university- based coursework, prospective teachers are presented with a range of Ruszynak and Bertram: Knowledge and judgement . . . 35 pedagogical theories, principles and approaches that they select from according to their school context and subject. Textbooks, like Getting Practical (Gultig and Stielau, 2009), embody this approach, presenting strategies for whole class teaching (such as explanations, demonstration, questioning and discussion) and strategies for small group teaching (such as problem-based learning, projects, role-play and simulations). A general approach may also require students to adopt a privileged form of pedagogy or theory of learning, such as a constructivist and learner-centred approaches (Reeves and Robinson, 2014, p.246) without interrogating whether or not this approach is appropriate for the concept and subject being taught. Second, those who advocate for a specialised knowledge approach to teacher learning criticise generally focused initial teacher education programmes because they “pay insufficient attention to what is to be taught, to construe teaching and learning as generic activities, with scant reference to the content of what is being taught or learned” (Morrow, 2007, p.82). The key assumption is that the practice of teaching draws on conceptual knowledge that is specialised to the subject/phase that is being taught. Thus teachers of mathematics, teachers of early literacy, teachers of history or of science need to draw on different reservoirs of pedagogical knowledge that is specific to their discipline. Theorists in this tradition (e.g. Craib, 1992; Shalem, 2014; Winch, 2014) also emphasise the importance of a theoretical educational knowledge that enables a practitioner to develop non-intuitive, organising insights in interpreting and responding professionally to practice-based contexts. According to them, it is educational propositional knowledge (and not only accumulated experience) that enables rational professional judgement over a range of diverse contexts (Shalem and Slonimsky, 2013). In this way, teaching is conceptualised as a complex, principled practice requiring specialised disciplinary-based knowledge that enables professional judgement. In turn, the emphasis on specialised knowledge in South African teacher education programmes is criticised for being too decontextualised, urban- centric, and not sufficiently preparing student teachers to teach in underprivileged or rural contexts (Balfour, Mitchell and Moletsane, 2008; Gardiner, 2008). Proponents of the third orientation argue that knowledge of contexts and individual personal development of prospective teachers in those contexts should be the driving force of initial teacher programmes. This position insists that teaching is so contextually embedded that it cannot be understood outside of its context (Carr, 2006), and therefore decontextualised 36 Journal of Education, No. 60, 2015 initial teacher education programmes do not generally prepare teachers to link general concepts within the curricula to locally specific issues and concerns. From this perspective, student teachers should be encouraged to construct personal theories and/or philosophies from their contextually-specific practical teaching experiences, usually through conscious self-reflection and experience of community engagement. Because these approaches to initial teacher education place emphasis on different knowledge domains, the assumed basis for professional judgement is also constructed differently. In the case of the first approach, judgement is made in relation to principles arising from practice and/or one’s experiences from practice. In terms of the second, judgement is made in relation to specialised theoretical constructs, and in the third, judgement is individual and contextually embedded. These three approaches have enormous implications for how assessors might be expected to use their own knowledge and judgement when assessing the competence of student teaching. Assessing the competence of student teaching within the South African context There is contestation in South Africa teacher education sector about the relative importance of different knowledge domains and the relationship (Reeves and Robinson, 2014). It cannot be assumed that all supervising teachers and/or university tutors have a shared set of internalised criteria of what constitutes effective teaching. Many of the present university tutors and supervising teachers undertook their initial teacher education during the Apartheid era, when the provision of initial teacher education was fragmented over 19 different government departments. It is not surprising then that the quality of initial teacher education programmes varied. At worst, teacher training programmes offered students little more than technical skills for coping with classroom life and a rudimentary level of content knowledge (Welch, 2002). Even in the more rigorous teacher training programmes, a dominant theoretical discourse was the inherently authoritarian Fundamental Pedagogics which actively discouraged critical reflection, analysis and the development of innovative teaching strategies (Enslin, 1990). Many teachers were “[actively discouraged] from engaging in any form of dialogue about why they were doing what they were doing, what the alternatives might be in their teaching and how their interactions with learners and colleagues might Ruszynak and Bertram: Knowledge and judgement . . . 37 be different” (Robinson, 2000, p.214). Classroom practice in the majority of South African classrooms continues to be dominated by teacher talk; a low level of learner participation; rote learning; a lack of meaningful questioning; a lack of lesson structure; an absence of engaging learning activities; little meaningful interaction between learners; and few tasks requiring reading/writing (Hoadley, 2013; Taylor and Vinjevold, 1999). Given this historical context, it is not sufficient for school-based and university-based assessors to draw only on their personal experience of teaching to make judgements about what constitutes effective teaching. Context of this study In an attempt to consolidate the sector, initial teacher education became relocated into the higher education system in 2002. It was hoped that a single policy would help unite the sector and increase the content and pedagogical knowledge of qualifying South African teachers. The first post-apartheid teacher education policy, the Norms and Standards for Educators (NSE) (Department of Education, 2000) claimed to be premised on an integrated, holistic understanding of teaching, but the implementation of this policy was generally atomistic and technical (DHET, 2011). The focus was more on the outcomes (roles) that teachers needed to demonstrate, than on an acquisition of a coherent and systematic body of professional knowledge needed by teachers. Many teacher educators who moved from the college sector into the higher education sector located their legitimacy to contribute to the professional preparation of new teachers in their practical expertise and previous classroom-based experiences (Robinson and MacMillan, 2006). We argue that relying exclusively on one’s practical experiences as grounds for understanding what constitutes competence in teaching in the context of a post-apartheid South Africa is not unproblematic. A revised policy, the Minimum Requirements for Teacher Education Qualifications (henceforth, MRTEQ) (2011) urges all teacher education programmes explicitly to “address critical challenges facing education in South Africa today – especially the poor content and conceptual knowledge found amongst teachers, and the legacies of apartheid” (p.9). It rejects a “purely skills-based approach [that relies on]. . . evidence of demonstrable outcomes as measures of success, without paying attention as to how knowledge must underpin these skills for them to impact effectively on 38 Journal of Education, No. 60, 2015 The broader study is managed by JET Education Services, and conducted with the2 support of the Department of Basic Education, Department of Higher Education and Training, and the Education Deans’ Forum. learning” (DHET, 2011, p.7). According to MRTEQ, for example, beginner teachers are expected to possess “sound subject knowledge”; “know how to teach their subject(s) and how to select, determine the sequence and pace content in accordance with both subject and learner needs”, and they must “know who their learners are and how they learn” (DHET, 2011, p.53) (authors’ emphasis). This emphasis on ensuring that student teachers develop theoretical, practical and professional knowledge bases for teaching stands in stark contrast to the skills-based approach which characterised much of teacher training during and immediately after the Apartheid era. In light of curricular changes to initial teacher education programmes in response to these policy shifts, it is an opportune time for teacher educators to re-examine the assumptions about teaching embedded within their TP assessment rubrics and the implication of these for student teachers and for assessors. This paper hopes to make a contribution in informing this important sectoral discussion. Methodology This paper emanates from a qualitative documentary analysis of the TP assessment instruments used during 2012 (before curriculum changes required by MRTEQ) by five South African universities that offer initial teacher education programmes. This paper is a small part of a comparative study into the initial teacher education programmes offered by five higher education institutions.2 Institutions offering teacher education differ according to geography (urban/rural), language of instruction (Afrikaans/ English), merger history (some experienced mergers of former teacher training colleges or technikons with universities) as well as resourcing. The five universities were purposively selected in order to characterise the complex range of histories, mergers and contexts of HEIs in South Africa, but with the understanding that Ruszynak and Bertram: Knowledge and judgement . . . 39 the data cannot be generalised beyond the institution itself. In this paper, the participating universities will be referred to as follows: Table 1: Descriptions of participating universities University A Formerly advantaged English-speaking university, located in urban context. Initial teacher education is offered in full time contact mode. University B University offering teacher education on multiple campus sites. Teacher education programme under review is located in a formerly advantaged Afrikaans speaking campus. Initial teacher education is offered through partial distance learning and full time contact modes. University C University offering teacher education to students through distance learning. Initial teacher education is offered through distance learning. University D Formerly disadvantaged university located in a rural context. Initial teacher education is offered in full time contact mode. University E Technical university offering teacher education, on multiple campus sites. Initial teacher education is offered in full time contact mode. In order to analyse what TP assessment instruments from different institutions expect of supervising teachers/university tutors, we developed a set of 22 questions to deductively guide our analysis and comparison of the five TP assessment instruments (see Appendix). We analysed both the structure of the TP assessment instruments and the criteria for effective teaching they present. Guided by these 22 questions, each TP assessment instrument was described and analysed in detail with frequent inter-rater checks to increase reliability of the analysis. The draft analysis was sent to each participating university for comment to ensure the accuracy of our analysis. We identified similarities and differences between the structure and criteria of the five TP assessment instruments. We then applied the conceptual lens of professional knowledge and professional judgement to consider expectations of how supervising teachers and university tutors draw on their knowledge of teaching to make judgements about the competence of the student teaching they assess. In presenting our findings, we examine the implications of how the design and criteria of TP assessment rubrics establish conditions for university tutors and/or supervising teachers to make judgements in relation to a set of given criteria when they assess the competence of student teaching. 40 Journal of Education, No. 60, 2015 Limitations and possibilities of this study The purpose of this paper is not to yield generalisations about TP assessment instruments in South Africa. Rather, we seek to illustrate design features that establish the potential for more and less professionally based assessment decisions about the competence of student teaching. The scope of this study is limited to consider what assumptions TP assessment instruments convey about what knowledge is required for assessing student teaching and grounds upon which assessment decisions are made. The analysis is based solely on what is stated in the ‘official’ TP assessment instrumentation used by university tutors and/or supervising teachers in assessing student teaching. We did not access concepts in coursework used to prepare student teachers for the practicum sessions, and the access of university tutors to that coursework. We are mindful, too of the logistical constraints experienced by institutions of higher learning in terms of allocating university staff to observe and assess every student teacher. For all these reasons, our findings are necessarily partial and our interpretation is provisional. Findings In this section we present the findings according to particular themes that emerged as we compared the five sets of documents. We draw examples from the various TP assessment instruments to illustrate our analysis. Who holds the knowledge required to assess the competence of student teaching? From the instruments analysed, we find three structural ways in which the input from university tutors and supervising teachers differs: first, where the assessment is done exclusively by school mentors or by university assessors; second, where the school and university staff complete different sections of an assessment; and third, where a joint assessment is done collaboratively between university and school staff. In contexts where student numbers are very high, or their distance from campus is too far, individual practicum supervision of each student teacher by a university staff member is unfeasible (e.g. Universities C and D). Supervising teachers, their Heads of Department and/or principals are Ruszynak and Bertram: Knowledge and judgement . . . 41 In the case of University A, provision is made for separate assessments to be submitted in3 cases where a consensus assessment between the supervising teacher and university tutor cannot be reached. required to complete all assessment relating to student teaching. When this structural feature of these TP assessment instruments is logistically (not pedagogically) determined and as such we cannot draw inferences about knowledge for assessing students from this structural arrangement. However, this arrangement might also be a pedagogic choice of programmes that prioritise contextual knowledge for teaching. Where supervising teachers and university tutors are asked to complete different sections of an assessment (e.g. University B), supervising teachers are not expected to share a common language of practice with university tutors. University B requires that supervising teachers assess students’ extra mural involvement, inter-personal relationships, and general professionalism over an extended period of time. University tutors, on the other hand, observe lessons and assess the extent to which a student teacher draws on her university coursework in her planning and execution of lessons. Supervising teachers are not assumed to have access to the preferred methodologies student teachers learn in their university-based coursework. When joint assessments of student teaching competence are required by the institution (e.g. Universities A and E), university tutors and supervising teachers are expected to be in contact with one another during the practicum. There is an expectation that they should seek consensus, and bring their particular perspectives together to enhance the overall assessment (such as the extended time that the supervising teacher spends with a student and the more global view of student teaching that the university tutor brings to bear on the assessment). The consensus model of joint school-based /university3 assessment assumes that effective teaching (as embodied in the stipulated criteria) should ideally be evident to both university lecturers and teacher mentor. Furthermore, such an arrangement suggests that they either share common understanding of the grounds upon which competence is recognised, or if not, that such principles are made explicit in the TP assessment instrument. 42 Journal of Education, No. 60, 2015 What knowledge bases for teaching do supervising teachers/ university tutors need to assess through the practicum? In this section, we consider what TP assessment instruments convey about the professional knowledge and judgement that university tutors and supervising teachers are expected to recognise, support and assess when ascertaining a student’s teaching competence. Drawing on Grossman’s (1990) four domains of teacher knowledge, we consider in particular categories relating to students understanding of the subject matter knowledge they teach; general pedagogic knowledge; PCK and contextual knowledge. We find significant differences in the way that the importance of content knowledge and pedagogic knowledge are prioritised in the instruments. This has important implications for the knowledge that assessors are assumed to have. All TP assessment instruments carry criteria that required student teachers to devise and teach lessons that move a class of learners through stages of an intentionally structured learning process. Without exception, all TP assessment instruments analysed include criteria that relate to students’ understanding of the subject/content knowledge; teaching and learning strategies used; learning and teaching support materials; assessment; language and communication; consideration of learner diversity; professionalism and relationship with learners. Although there is much commonality of the kinds of criteria listed, what is demanded of student teachers in relation to those criteria is highly variable between institutions. Although no participating institution used specialised TP assessment instruments for particular content subjects, some institutions (Universities A, C and E) provide different instruments for pre-primary and/or Foundation Phase students, and only one provides a different assessment instrument for Intermediate and Senior phases (University E). Student’s understanding of content knowledge Some instruments required that students possess ‘sound’, ‘sufficient and accurate’ content knowledge (Universities B and C), whereas others specify what student teachers are expected to do with that knowledge. For example, comprehensive and well-organised conceptual knowledge enables students to ‘foreground main ideas’ in their explanations (University A), use appropriate examples from the ‘real world’ (University D), and inform their pedagogical choices (University A). Ruszynak and Bertram: Knowledge and judgement . . . 43 A key issue emerging from our analysis is the differing importance that content knowledge plays in the perceived effectiveness of student teaching across institutions. For Universities A and E, students’ understanding of content knowledge is considered a non-negotiable, where misunderstanding of content knowledge constitutes justifiable grounds for a ‘no credit’ result. At least one of the assessors would need to have either a subject specialisation or a working knowledge of the demands of different subjects and their pedagogic implications. In comparison, a sound grasp of content knowledge is one of many criteria that contribute a portion to a students’ overall mark in Universities B, C and D. The weighting of students’ understanding of the content they teach ranges from 5–8% of the total mark allocated. It is conceivable then that in such assessments, students could misunderstand thecontent they teach but still obtain a credit for their teaching practicum on the strength of other dimensions of their teaching. General pedagogic knowledge and pedagogic content knowledge In relation to subject and phase specialisations, we distinguish between criteria that are expressed as general components of teaching over all subjects and phases, and those that require specialist subject and/or phase specific knowledge. We present a summary of how the same criterion can be phrased differently to present teaching as a practice underpinned by general pedagogic knowledge (GPK), or a specialist one where pedagogic content knowledge (PCK) demands a simultaneous consideration of the demands of the content knowledge and the learning needs of children. 44 Journal of Education, No. 60, 2015 Table 2: Examples that show how criteria for effective teaching are constructed in terms of a GPK and PCK knowledge base. Component of teaching Teaching informed by General Pedagogic Knowledge Teaching informed by Pedagogical Content Knowledge (italics shows focus on knowledge specialised to subject or phase) Language and communication Appropriate language level to explain, instruct and question (A, D, E) Uses and develops learners’ ability to use subject discourse (A) Teaching/Learning strategies Uses a variety of teaching strategies effectively (B) Thoughtfully selects and effectively uses teaching and learning strategies appropriate to both content and learners (A); Lesson introductions [should] have an appropriate contextualisation and include relevant activities (B) Learning/Teaching support materials (LTSM) Effective use of LTSM to enhance lesson presentation (A; B) Relevant LTSM are used to develop learners’ understanding of key concepts (D) Assessment Student monitors and probes learner understanding (A; B) Act on the assessment data to remediate conceptual misunderstandings (D; E) Classroom Management Creates positive/safe environment in which children can learn (A; E) No examples in the data Inclusivity Be responsive to diverse learner needs (A; B; E) No examples in the data From Table 2, it can be seen that a non-specialist assessor would be more easily able to work broadly with general criteria, but they may very well miss the nuances of deliberate teaching decisions that a student teacher may or may not have taken informed by their developing PCK. The more specialist phase or content knowledge is emphasised in the TP assessment rubric as a core part of the knowledge that informs teaching competence, the more crucial it becomes for at least one of the teacher mentor and/or university lecturers who are assessing student teachers to be phase/subject specialists. This is particularly for high stakes decisions (e.g. the final assessment of student teaching competence prior to qualifying). Ruszynak and Bertram: Knowledge and judgement . . . 45 Situational/contextual knowledge In all the TP instruments, criteria that made reference to contextual knowledge considered the extent to which the student teacher works productively within a particular school environment; her ability to recognise aspects of learner diversity within that context, and be responsive to the diverse learning needs of children in her class/es. The TP assessment instruments analysed reflect different understandings of the term ‘diversity’. University D’s TP assessment instrument associates diversity with ethnicity and gender. Similarly, University C requires that student teachers have an ‘accommodating attitude’ towards learners who are different to themselves. University A and E made explicit reference to the pedagogical implications of diversity within a class – such as making pedagogical choices that considered potential differences in learners’ prior knowledge, vocabulary, reading competence, attention spans and so on. Notwithstanding the arguments by those who advocate for a contextually driven teacher education, the five institutions’ TP assessment documentation made very little provision for the way in which student teachers respond to the limitations, challenge and opportunities within the context of their school placement. This observation may be because university based lecturers (and the designers of the TP assessment instruments) may have a less nuanced understanding of the contextual limitations and possibilities that school-based mentors would presumably have. Our analysis identifies this as a significant oversight in the criteria for assessing competent teaching, especially for students who undertake their practicum sessions in particularly challenging contexts. To what extent are assessors required to evaluate the appropriateness of the professional judgements students make in their teaching? Certain criteria express an expectation that the students should employ a preferred approach, teaching strategy, or type of resource. These normative expectations are exemplified in the second column of Table 3. The importance of students’ developing professional judgement is minimised in such criteria and expectations are expressed as technical application of particular principles/strategies. The assumption here is that the assessor would 46 Journal of Education, No. 60, 2015 have knowledge of these preferred techniques, and be able to judge their correct application. In contrast, other criteria require consideration of the ways in which content knowledge and pedagogy intersect to enable students to make of conceptually-informed pedagogically appropriate choices. Recognising the role of professional judgement phrasing of criteria indicate that certain choices are better than others, as can be seen from the examples in the third column of Table 3. While words such as ‘appropriate’, ‘relevant’ and ‘selects’ imply that a deliberate pedagogical choice has been made, an observer would only have access to the grounds for such a choice if the lesson preparation contains an articulated rationale for the lesson design in ways that show their deep conceptual understanding of the content they are teaching, as well as their understanding of the learners’ levels of understanding and social contexts (Rusznyak and Walton, 2011). The assumption underpinning these criteria is that the assessor/s would have sufficient content and pedagogic knowledge to recognise where in/appropriate pedagogic choices have been made by the student teacher. Table 3: Examples that show how criteria for effective teaching are constructed in terms of a technical application of a preferred technique or an application of a reasoned judgement. Component of teaching Technical application Reasoned judgement Teaching/Learning Strategies Apply principles of cooperative learning (E) Linking with prior knowledge (C) Thoughtfully selects and effectively uses teaching and learning strategies appropriate to both content and learners (A) Learning/Teaching support materials (LTSM) Use media correctly (E); Lessons have quality handouts (C) Relevant LTSM are selected and used to develop learners’ understanding of key concepts (D) Assessment Use of formative and summative assessment strategies; marking of learning tasks (C) Student is able to act on assessment data to remediate conceptual misunderstandings (D; E) Ruszynak and Bertram: Knowledge and judgement . . . 47 To what extent do assessors need to rely on an internalised sense of competence in making assessment decisions? In this section, we consider the extent to which the rating scales and level descriptors given in the TP assessment rubric are made explicit to assessors. In four of the TP assessment documentation (Universities B, C, D and E), student teaching is assessed on a four or five point rating scale. Levels of competence range from Excellent/Outstanding/Highly Developed on one side of a rating scale, to Not achieved/Unsatisfactory/Poor at the other. Without detailed descriptors of what constitutes excellence, satisfactory or inadequacy against each criterion, the assessors of student teaching (either supervising teachers or university staff) are assumed to have an internalised sense of what constitutes competence at the specified levels of competence. While the use of a checklist of criteria along with a simple rating scale may seem like a user- friendly way to structure the assessment of student teaching, it relies on all supervising teachers and a large number of university tutors being able to interpret each criterion at different levels of competence. Given the concerns articulated about the fragmented history of initial teacher education in South Africa, and the dismal quality of many initial teacher training programmes, the use of a simple rating scale may not adequately provide many university tutors and supervising teachers with the necessary support to make informed assessment judgements. Thus these assessors can only rely on their own experience of what ‘good teaching’ is to make a judgement. Furthermore, given the extreme diversity within the South African schooling system, simple rating scales do not allow consideration of the different (and often profoundly uneven) school contexts in which students teachers are encouraged to undertake a practicum session. The standardisation of student assessments using simple rating scales over different contexts is thus compromised. The excerpt below shows how the supervisors for University C will need to rely on their own personal experience to assess whether the criteria are achieved or not, since the instrument does not provide explicit descriptors. 48 Journal of Education, No. 60, 2015 Rating scale: 1 – Not achieved; 2 – Partly achieved; 3 – Average; 4 – Above average; 5 – Excellent Category Criterion Mark Student’s mark % Comments Planning and writing out of lesson Situation analysis (relevant and effective) Lesson outcomes (clear and achievable) Lesson phases (all phases accounted for) 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 /15 Lesson presentation (learners must be brought into contact with new outcomes and content) Beginning of lesson Linking with prior knowledge Authentic examples 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 /15 Middle phase Revealing new content (effectiveness) Teaching and learning activities (appropriateness) Formative assessment 5 4 3 2 1 5 4 3 2 1 5 4 3 2 1 /15 Etc. Figure 1: Excerpt from University C instrument: Assessment of student performance by supervisor In contrast to the others, University A provides highly explicit descriptors to define what constitutes each level of four levels of competence against every criterion, ranging from what a student’s teaching would look if he/she was ‘Not yet coping’ to ‘Thoughtful and insightful teaching competence’in terms of each criterion listed. It relies less on assessors using their tacit understanding to interpret what constitutes competence in student teaching at each level and against each criterion. For example, four levels of students’ understanding of content knowledge is described as follows: Ruszynak and Bertram: Knowledge and judgement . . . 49 Also termed work-place learning; teaching experience.4 Not yet coping Emerging teaching competence Developing skilled teaching competence Thoughtful insightful teaching competence Knowledge and understanding of content Inaccurate content or misunderstands concepts frequently Knowledge often limited to what learners need to know Research evident, demonstrates sound understanding of topics beyond what learners need to know Comprehensive, well organised knowledge of topics; foregrounds main ideas; networked examples Formulation of purpose Limited consideration or understanding of lesson’s purpose Purpose of the lesson is unclear or vaguely formulated Clear purpose in terms of key questions; skills; attitudes and values Purpose is subject specific, reflecting the knowledge, skills and dispositions of the subject discipline Conceptualisa- tion of lessions Incoherent lesson steps not aligned with purpose Lesson steps often disjointed without links between steps Lesson steps coherent but not always thoughtfully scaffolded Thoughtfully conceptualised and scaffolded lesson steps Etc. Figure 2: Extract from University A instrument: Formative assessment for teaching experience In this example, each criterion is unpacked explicitly, and in so doing, provides a language of description for assessing teaching practice in ways that a straightforward five-point rating scale cannot. The extent to which judgement of competent teaching requires assessors to undertake a holistic assessment of student teaching In all cases, teaching practice is a compulsory university course for which4 student teachers must register and thereby requires a course mark for promotion. While University A required a numerical mark only for the final practicum session before qualifying, other universities require that assessors 50 Journal of Education, No. 60, 2015 provide a numerical mark, either for every lesson observed or for each teaching practicum session. This section considers the extent to which assessors are prompted to consider components of teaching as separate competences or as an integrated whole in order to arrive at an assessment decision/course mark. Three TP assessment instruments analysed present a list of criteria which are each awarded a mark or rating, whose arithmetical sum then reflects the overall mark awarded to the student (Universities B, C and D). This structure transmits a notion that the practice of teaching is reducible to a list of discrete observable competences. The integrated nature of teaching as a coherent practice is largely hidden as a consideration during the assessment. The excerpt in Figure 3 below shows how an assessor allocates a mark from 0–5 for each criterion and then adds up the marks to achieve a mark out of 100. Ruszynak and Bertram: Knowledge and judgement . . . 51 5 4 3 2 1 0 A Lesson introduction 1 Ability to introduce topic, learning outcomes and lesson outcomes to be achieved at the end of the lesson intelligently 2 Effectiveness in motivating and arousing interest in learners 3 Efficient use of learners’ previous knowledge to achieve assessment standards 4 Linking lesson contents with reality for meaningful learning B Lesson development B1 Communication skills B1.2 Ability to use voice, gestures and eye contact and movement to enhance teaching effectiveness B1.2 Ability to use main language of instruction to explain, describe and discuss key concepts B1.3 Ability to mediate learning in a manner that shows awareness of the cognitive development of learners B1.4 Create and maintain learning environments that are interesting, challenging , orderly, safe, purposeful and supportive. B2 Classroom management B2.1 Ability to facilitate occasions where learners are taught in groups, pairs and individuals B2.2 Ability to use a variety of discipline strategies well matched to the situation and the learner B2.3 Ability to create a positive learning environment for student learning and involvement B2.4 Abiltity to demonstrate openness to student challenges about information and ideas B2.5 Ability to manage time (for him/herself and learners) and maintain lesson momentum Etc. HD = Highly developed/very satisfactory 75–100% C = Competent/satisfactory 60–75% D = Developing/Almost satisfactory 50–59% E = Experiencing difficulties/Unsatisfactory 01-49% Figure 3: Excerpt from University D instrument: BEd/PGCE summative classroom observation and evaluation form 52 Journal of Education, No. 60, 2015 By way of contrast, our analysis of the data shows three ways in which two TP assessment instruments (Universities A and E) encourage a more holistic assessment of student teaching. Firstly, we notice a network of relationships between different criteria. For example, in the TP assessment instrument of University E the ‘quality of learning’ is considered in terms of ‘learner participation’, ‘learner understanding of lesson’s concepts’, an ‘emphasis on content knowledge’, and the ‘achievement of outcomes’. In this example, a criterion is expressed in terms of a relationship with others, so that the internal connections between criteria are made explicit. However, the potential of assessment rubrics to transmit a view of teaching as a networked, integrated practice can be undermined by the linear structure of the rubric that requires that a mark is awarded against each criterion. When a student’s mark is calculated by adding together the discrete marks awarded, the view of teaching as a coherent whole is compromised. Although in several cases, ‘coherence of teaching’ is listed as one out of many criteria, the structure of the TP assessment rubric suggests that it is not. A second way in which holistic assessment is undertaken is through assessors awarding students a global impression mark, after rating their teaching performance on a checklist. This method might work if all assessors had strongly internalised criteria of what constitutes competence in student teaching. Although there is no direct relationship between the checklist profile and the mark awarded, the grounds for the awarded mark are highly subjective, and this method would be difficult to standardise across large numbers of university tutors and supervising teachers. A third way in which holistic assessment of student teaching is enabled is through a TP assessment instrument designed as a two dimensional grid that plots a student teacher’s knowledge, understanding and thinking against the effectiveness of her classroom performance (University A). This structure conveys to students and assessors alike that effective teaching involves both a cognitive and a performance dimension, and that if one component is weak, the effectiveness of a student’s teaching as a whole is compromised (Rusznyak, 2012). Ruszynak and Bertram: Knowledge and judgement . . . 53 Implications of findings Recent research in the field of teacher education supports a more holistic understanding of teaching as an integrated knowledge-based professional practice (Darling-Hammond and Bransford, 2005; Hoban, 2005). The MRTEQ policy (DHET, 2011) follows this trend, explicitly rejecting an exclusively skills-based approach in initial teacher education, and in its recognition of specialised pedagogic and content knowledge that informs effective teaching. There are several implications for the assessment of student teaching, which we now discuss. First, if university-based coursework presents teaching as an integrated, complex practice but student teaching is assessed as a collection of discrete skills or general competencies, then internal coherence within the teacher education programme is unduly compromised. The use of TE assessment instruments that do not adequately support student teacher professional development in planning conceptually strong and well-executed lessons not only misses an opportunity to support students’ professional development, it may also perpetuate the prevalence of technicist guidance provided to them during their practicum sessions. Second, if theoretical developments and policy shifts both support the idea that teaching requires a specialist knowledge base and the development of professional judgement, then the use of general criteria that do not recognise the importance of a specialist knowledge base is problematic. However, expressing criteria in terms of a specialised knowledge base requires that at least one of the supervising teacher and university tutor has access to that specialist knowledge base, and understands its use to make appropriate pedagogic choices. While recognising the logistical constraints of placing large numbers of student teachers under the mentorship of specialist supervising teachers, and the enormous demands placed on specialist university tutors, this may be one area where institutions and government allocate additional funding resources to ensure that specialist mentoring and assessment of student teaching is not compromised by logistical and budgetary constraints. Third, supervising teachers in schools may potentially work with student teachers from different institutions at different times over a year. The high variability over expectations contained in TP assessment instruments may 54 Journal of Education, No. 60, 2015 undermine the potential for the teacher education sector as a whole to introduce a coherent language of practice into school communities. The lack of sector consensus presents a challenge to the teacher education sector to interrogate itself and understand more clearly the tensions that lead to variability and contestations that characterise conceptions of effective teaching within the sector. Conclusion Internationally recognition of the importance of content knowledge for making appropriate pedagogical choices and the role of PCK in informing subject-based pedagogy has prompted policies and teacher education programmes to shift from a technological orientation to an academic one, and from conceptualising teaching as a generic practice to a specialised one. However, it does not make for coherent initial teacher education programmes if these shifts are made in university-based coursework but are not carried into the practicum and how competence in student teaching is assessed. As significant players in influencing the discourse of how the nature of teaching is portrayed to the wider profession, it is an urgent imperative for teacher educator robustly and critically to analyse the conceptions of teaching embedded and transmitted by the TP assessment instruments. We distinguish between design features in the TP assessment instruments analysed that encourage a more technical approach to verifying the competence of student teaching, and those that promote more of a professional judgement. Our analysis suggests that a technical approach to assessing teaching is fostered when assessors simply need to verify that students have complied with a list of specified requirements as they teach. Such criteria are expressed as a normative application of particular preferred strategies or expectations. Because a collection of techniques need not add up to a coherent practice, this approach tends to determine the competence of a student’s teaching by the sum of criteria each one verified in an atomistic manner. In contrast, a more academic approach to the assessment of student teaching is encouraged where criteria are explicit in how students are required to use their content and/or pedagogic knowledge to make appropriate choices and decisions in their planning and teaching, and provide a rationale for these choices. A professional approach also demands that teaching constitutes a coherent practice, and would require a holistic assessment of teaching where Ruszynak and Bertram: Knowledge and judgement . . . 55 the judgement is based on principled grounds, rather than a global impression mark based on an assessor’s personal sense of a student’s teaching competence. References Balfour, R.J., Mitchell, C. and Moletsane, R. 2008. Troubling contexts: toward a generative theory of rurality as education research. Journal of Rural and Community Development, 3(3): pp.95–107. Ball, D.L., Thames, M.H. and Phelps, G. 2008. Content knowledge for teaching: what makes it special? Journal of Teacher Education, 59: pp.389–407. Banks, F., Leach, J. and Moon, B. 2005. Extract from ‘New understandings of teachers’ pedagogic knowledge’. The Curriculum Journal, 16(3): pp.331–340. Carr, W. 2006. Education without theory. British Journal of Educational Studies, 54(2): pp.136–159. Craib, I. 1992. Modern social theory: from Parsons to Habermas. New York: Harvester Wheatsheaf. Darling-Hammond, L. and Bransford, J. (Eds). 2005. Preparing teachers for a changing world. What teachers should learn and be able to do. San Francisco: Jossey Bass. Department of Education, 2000. Norms and standards for educators. Pretoria. Department of Higher Education and Training, 2011. National Qualifications Framework Act 67 of 2008: Policy on the Minimum Requirements for Teacher Education Qualifications. Pretoria: Government Gazette. Enslin, P. 1990. Science and doctrine: theoretical discourse in South African teacher education In Nkomo, M. (Ed.), Pedagogy of domination: towards a democratic education in South Africa. New Jersey: African World Press, pp.77–92. 56 Journal of Education, No. 60, 2015 Gardiner, M. 2008. Education in rural areas. In Pampallis, J. (Ed.), Issues in education policy (Vol. 4). Braamfontein: Centre for Education Policy Development. Gess-Newsome, J. and Carlson, J. 2013. The PCK summit consensus model and definition of pedagogical content knowledge “Reports from the pedagogical content knowledge (PCK) summit: ESERA Conference, Colorado Springs, USA, October 2012 Grossman, P. 1990. The making of a teacher. Teacher knowledge and teacher education. New York and London: Teachers College Press. Gultig, J. and Stielau, J. (Eds). 2009. Getting practical. About classroom- based teaching for the National Curriculum Statement (2nd ed.). Cape Town: Oxford University Press and South African Institute for Distance Education. Hawe, E. 2002. Assessment in a pre-service teacher education programme: the rhetoric and the practice of standards-based assessment. Asia-Pacific Journal of Teacher Education, 30(1): pp,93–106. Hoadley, U. 2013. What do we know about teaching and learning in South African primary schools? Education as Change, 16(2): pp.187–202. Hoban, G.F. 2005. Developing a multi-linked conceptual framework for teacher education design. In Hoban, G.F. (Ed.), The missing links in teacher education design. Springer, pp. 1–17. Morrow, W. 2007. Learning to teach in South Africa. Pretoria: HSRC Press. Muller, J. 2012. The body of knowledge/le corps du savoir. Paper presented at the Seventh Basil Bernstein Symposium, Aix-en-Provence, France, 27–30 June 2012 Raths, J. and Lyman, F. 2003. Summative evaluation of student teachers: an enduring problem. Journal of Teacher Education, 54(3): pp.206–216. Reddy, C., Menkveld, H. and Bitzer, E. 2008. The practicum in pre-service teacher education: a survey of institutional practices. Southern African Review of Education, 14(1): pp.143–163. Ruszynak and Bertram: Knowledge and judgement . . . 57 Reeves, C. and Robinson, M. 2014. Assumptions underpinning the conceptualisation of professional learning in teacher education. South African Journal of Higher Education, 28(1): pp.236–253. Robinson, M. 2000. Mentoring and teacher development: a case study. Unpublished doctoral thesis, University of Western Cape, Cape Town. Robinson, M. and MacMillan, W. 2006. Who teaches the teachers? Identity, discourse and policy in teacher education. Teaching and Teacher Education, 22(3): pp.327–336. Rusznyak, L. 2011. Student teaching assessment instruments: possibilities and pitfalls for promoting professional development. Journal of Education, 51: pp.115–136. Rusznyak, L. 2012. Summative assessment of student teaching: a proposed approach to quantifying practice. Journal of Education, 56: pp.91–115. Rusznyak, L. and Walton, E. 2011. Lesson planning guidelines for student teachers: a scaffold for the development of pedagogical content knowledge. Education as Change, 15(2): pp.271–285. Shalem, Y. 2014. What binds professional judgement? The case of teaching. In Young, M. and Muller, J. (Eds), Knowledge, expertise and the professions. London: Taylor and Francis, pp.93–105. Shalem, Y. and Slonimsky, L. 2013. Practical knowledge of teaching practice- what counts? Journal of Education, 58, pp.67–86. Shulman, L. 1986. Those who understand: knowledge growth in teaching. Educational Researcher, 15(2): pp.4–14. Shulman, L. 1987. Knowledge and teaching: foundations of the new reform. Harvard Educational Review, 57(1): p.1. Taylor, N. and Vinjevold, P. (Eds). 1999. Getting learning right. Report of the President's Education Initiative Research Project. Johannesburg: Joint Education Trust. 58 Journal of Education, No. 60, 2015 Verloop, N., Van Driel, J.H. and Meijer, P. 2001. Teacher knowledge and the knowledge base of teaching. International Journal of Educational Research, 35: pp.441–461. Welch, T. 2002. Teacher education in South Africa before, during and after apartheid: an overview. In Adler, J. and Reed, Y. (Eds), Challenges of teacher development. An investigation of take-up in South Africa. Pretoria: Van Schaik, pp.17–35. Winch, C. 2014. Know-how and knowledge in the professional curriculum. In Young, M. and Muller, J. (Eds), Knowledge, expertise and the professions. London and New York: Routledge, pp.47–60. Ruszynak and Bertram: Knowledge and judgement . . . 59 Appendix: Questions guiding the analysis of TP assessment instruments Structure of TP assessment instrument/s 1. Who completes the overall TP assessment instrument at the end of a TP session? 2. Is there one overall assessment instrument used for all students? If no… ! Are there differences in the assessment instrument used for junior students/senior students? If so, what are these differences? ! Are there differences in the way students in different phases are assessed? If so, what are these differences? ! Are there differences in the way students specialising in different subjects are assessed? If so, what are these differences? 3. What levels, descriptors or rating scales of teaching competence are used by the assessment instrument? How explicitly or implicitly are these competence levels defined? What explanation does the documentation provide about what is meant by each level of competence? 4. Is assessment of the students’ documentation (e.g. preparation file/ lesson planning/journal etc.) included in the assessment instrument, or is this done separately? Is the supporting documentation assessed by the same person who observed the students’ teaching? 5. Are criteria listed or clustered into groups? If so, how are the criteria clustered? 6. What is the logic informing the sequencing of the criteria? Formative and summative assessment of student teaching 7. What formative feedback about their developing practice could students obtain from the assessment document/s? (Ie. what they are doing well, what they are struggling with and what they need to do to improve their TP)? 60 Journal of Education, No. 60, 2015 8. When are marks for TP awarded? How are the marks determined? How are criteria weighted? 9. Are the grounds for a distinctive performance in TP implicitly or explicitly conveyed by the TP assessment Rubric? If so, please elaborate. 10. Are the grounds for a fail in TP implicitly or explicitly conveyed by the TP assessment rubric/s? If so, please elaborate. 11. How user-friendly is the TP assessment document for teacher (outside of the university staff) find difficult to use the form? What might a supervising teacher find easy to complete/understand? What might a supervising teacher find difficult to complete/understand? 12. To what extent does the documentation encourage articulation between the lesson observations reports and the formative/summative assessment of students’ TP? Domains of Teacher Knowledge 13. What does the assessment instrument convey to students about the importance of their content knowledge? 14. What does the assessment instrument convey to students about general pedagogical knowledge? ! Classroom management/organisation ! Inclusion, diverse learner needs ! Teaching strategies ! Teaching resources ! Language competence ! Relationship with learners ! Other? 15. What does the assessment instrument convey to students about working with curricular documentation? Ruszynak and Bertram: Knowledge and judgement . . . 61 16. How does the instrument promote an assessment of students’ pedagogical content knowledge? 17. What is privileged about how students are expected to plan their lessons? 18. What does the assessment instrument convey to students about what is important in assessing learners’ understanding? Professional reasoning and judgement 19. How does the assessment instrument prompt teachers/tutors to consider the students’ ability to make justifiable pedagogical choices? 20. How does the assessment instrument prompt teachers/tutors to consider the students’ ability to reflect in and on action? 21. What does the assessment instrument/s convey to students about what constitutes ‘teacher professionalism’? 22. To what extent does the TP assessment instrument/s consider the way in which students’ lessons are internally coherent? Lee Rusznyak School of Education University of the Witwatersrand lee.rusznyak@wits.ac.za Carol Bertram School of Education University of Kwazulu-Natal bertramc@ukzn.ac.za mailto:lee.rusznyak@wits.ac.za mailto:bertramc@ukzn.ac.za 62 Journal of Education, No. 60, 2015