Draft1_13March_JA Jill Adler is also Chair of M athematics Education at King’s College London. 1 Sitti M aesuri Patahuddin is a lecturer in the Department of M athematics at the State 2 University of Surabaya (Unesa) Indonesia. The paper has been developed during a post doctoral fellowship at the University of the W itwatersrand. Recontexualising items that measure mathematical knowledge for teaching into scenario based interviews: an investigation Jill Adler and Sitti Maesuri Patahuddin 1 2 Abstract This paper interrogates the recontextualisation of available assessment items developed for research purposes that measure mathematical knowledge for teaching, into scenarios for use in qualitative studies related to mathematics teachers’ subject matter knowledge. It draws from interviews with teacher participants in the Wits Maths Connect-Secondary project and their responses to two selected items from the Learning Mathematics for Teaching (LMT) project. The analysis shows that carefully constructed multiple choice items in the domain of (mathematics) subject matter knowledge have much potential in provoking teachers’ talk and their mathematical reasoning in relation to practice-based scenarios; and exploring with teachers a range of connected knowledge related to the teaching of a particular concept or topic. We argue that productive use of such items further requires that researcher make explicit the mathematical ideas they expect to explore and assess in the developed items. Introduction In their comprehensive survey of assessing teachers’ mathematical knowledge for teaching (MKT), Hill, Sleep, Lewis and Ball (2007) make a useful distinction between “the quality of mathematics instruction” (which has embedded in it, value judgments on instructional approaches); and “the quality of mathematics in instruction” (p.150), where focus is “specifically on the actual mathematics deployed in the course of a lesson”. Mathematics in use is thus professional knowledge, deployed for the purposes of teaching, and not for its own sake. What then is entailed in accessing and assessing such knowledge? Hill et al. (2007) describe the qualitative research, based on observation of teaching (knowledge in action), and/or task-based interviews 18 Journal of Education, No. 56, 201218 where tasks are developed from knowledge of practice, and their more recent quantitative research in the Learning Mathematics for Teaching (LMT) project which has focused on developing, validating and then using measures of professional knowledge. Despite convergence in appreciation for the multi- faceted and complex nature of professional knowledge, these two lines of research remain disjoint. Hill et al. (2007) suggest that “. . . qualitative researchers have much to learn from large scale test developers” and vice versa, and that “. . . more cross over needs to occur” (p.152). The work we report here moves into the terrain of crossing over. In the Wits Maths Connect-Secondary (WMC-S) project we are concerned with teachers’ mathematical knowledge in use, and its growth through participation in professional development and over time. To this end, we have selected LMT items, and recontextualised them into a semi-structured interview setting. In this paper we interrogate our use of two such items in depth to explore the potential of such recontextualisation for producing what we see as ‘fit for purpose’ readings of project teachers’ professional knowledge. We begin with a selective review of the literature base on assessment of mathematical knowledge for teaching, followed by a description of the research project, and our research and development approach to professional knowledge. We then describe the interviews and the selected items and present our analysis of the teacher interview data. Our analysis will show that carefully constructed multiple choice items in the domain of subject matter knowledge (see below) have much potential in (1) provoking teachers’ talk, and their mathematical reasoning in relation to practice-based scenarios; and (2) exploring with teachers a range of connected knowledge related to the teaching of a particular concept or topic. In addition, from a methodological point of view, we will argue that productive use of such items in semi- structured interviews requires researchers to make explicit their assumptions as to what knowledge(s) are privileged in their assessments. Facilitating Hill et al.’s call for greater cross over and accumulation in our research, rests particularly on this latter point. Assessing teachers’ professional knowledge – what has been done? Research related to mathematics teachers’ professional knowledge (MKT), what it is, its relationship to practice and learning gains, how it grows, and Adler and Patahuddin: Recontextualising item s that m easure. . . 1919 more recently, how it can be validly and reliably measured, has mushroomed. A comprehensive review of research on assessing MKT in the US, focused on “what knowledge matters and what evidence counts”, traces the development of methods for describing and measuring professionally situated mathematical knowledge in the US (Hill et al., 2007). Briefly, in the 1980s and 1990s methods were geared towards uncovering mathematical knowledge for teaching through observations of teaching practice (e.g. Leinhardt and Smith, 1985), and/or exploring and describing teacher knowledge in task based interviews (e.g. Ma, 1999; Borko, Eisenhart, Brown, Underhill, Jones and Agard, 1992). Through studies of expert mathematics teachers, experienced teachers across cultural contexts, and of the complexity of learning to teach respectively, this work has contributed significantly to elaborating the specificity of professional knowledge in and for mathematics teaching. Hill et al. locate the recent measures work, and their LMT project, in the context of this qualitative research. They argue that, notwithstanding its advances, a major weakness is that it is necessarily small scale. They build from this work to enable large scale, reliable and valid ways of assessing professionally situated knowledge. The results of the LMT research have been widely published and include reflection on how, building from Shulman’s (1986) initial work, the development of measures simultaneously produced an elaboration of the construct MKT and its component parts. As they developed measures, they were able to distinguish and describe Subject Matter Knowledge (SMK) and Pedagogic Content Knowledge (PCK), and categories of knowledge within each of these domains as illustrated in Figure 1. Common Content Knowledge (CCK – mathematics that might be used across a range of practices) was delineated from Specialised Content Knowledge (SCK – mathematics used specifically in carrying out tasks of teaching) (Ball, Thames and Phelps, 2008). Simply, recognising an incorrect answer to a calculation (CCK) is not synonymous with being able to reason across a range of responses to a calculation as to their mathematical validity and worth, as task teachers continuously do (SCK). Within PCK, where knowledge of mathematics is intertwined with knowledge of teaching and learning, they distinguish Knowledge of Content and Students (KCS – e.g. knowledge about typical errors learners make, or misconceptions they might hold), from Knowledge of Content and Teaching (KCT – e.g. knowledge of particular tasks that could be used to introduce a topic). All their items are presented in multiple choice format, and whether SMK or PCK, are set in a teaching context. In addition to describing their MKT constructs and exemplifying measures of these, they have elaborated the work done to produce their measures (Hill, Schilling and 20 Journal of Education, No. 56, 201220 W e include the figure of Ball et al’s components of M KT to assist the reading of our paper,3 as we refer to these through their abbreviations here and later in the paper; it is not as an object of attention in itself. Ball, 2004), and reported on positive correlations they found in their study of the relationship between measures of teachers’ MKT, the quality of their mathematics teaching and their learners’ performance (Hill, Rowan and Ball, 2005; Hill, 2008). Figure 1: Domains of mathematical knowledge for teaching (MKT)3 (Ball et al., 2008) In their concern for construct validation, the LMT project has subjected its work to extensive critique. A whole issue of Measurement (Vol. 5, No.2–3, 2007) is turned to this purpose, and makes visible just how complex, and costly, such assessment practices are. Invited commentary highlights the limitations of quantitative measurement of professional knowledge. Schoenfeld (2007, same issue) argues that any assessment must be explicit and clear in what is being assessed, and that it is not clear what exactly individual LMT items and their distractors do, nor how they accumulate. Difficulties entailed in measures work are critiqued within the LMT project itself, particularly PCK items aimed at KCS (Hill et al., 2007; Hill, 2008). The strength of the construct of PCK, in their terms, depends on how well it can be distinguished from knowledge of the mathematical content itself. LMT validity tests, including clinical interviews on these items, failed to separate Adler and Patahuddin: Recontextualising item s that m easure. . . 2121 Nardi, Biza and Zachariades (2012)’s study of teachers’ argumentation in task-based4 interviews contributes to this debate. It provides a careful operationalising of forms of argument, and teachers’ practical, yet complex reasoning. KCS from related measures of content knowledge. Scores on KCS items correlated highly with CCK scores. As Alonzo (2007, same issue) comments, this result has the danger of suggesting that all that matters is content knowledge, back-grounding the important work and progress that has been made across the field, in elaborating professional knowledge and particularly SCK. Hill et al. (2007) and Hill (2008) describe additional insights from their cognitive interviews on PCK-KCS items that showed that teachers also used mathematical reasoning, and test-taking skills, to decide on the correct answer. Teachers were asked to ‘think aloud’ as they talked about each item, their selection of the correct answer from four possible answers in the multiple choice format, and their justification for their selection. In their analysis of teachers’ talk, it was difficult to separate out teachers’ KCS from their mathematical reasoning about their choices, and so their SMK in use. Hill et al. (2007) conclude that ‘this domain [PCK] remains underconceptualised and understudied” (p.395), despite wide agreement in the field that this kind of knowledge matters. Their reflection on their detailed PCK work presents considerable challenges for the field of mathematics education: the notion of PCK is widely invoked in mathematics teacher education research and practice, often without clear and operationalised definitions.4 The nature of the boundary between SMK and PCK has been critiqued by others researching in mathematics education. Huillet (2009), for example, argues from the perspective of the Anthropological Theory of Didactics that there is no ‘common content knowledge’; all knowledge is tied to activity. Hence, clear distinctions between SMK and PCK are problematic. Much of the critique on a hard boundary, including Huillet, has emerged from qualitative studies with stronger situative perspectives, and focused at the secondary level (e.g. Zazkis and Leikin, 2010; Nardi, Biza and Zachariades, 2012). WMC-S also has a strong situative perspective, but nevertheless took up the challenges of ‘crossing-over’ and using LMT measures. Interestingly, despite our selections of LMT items that assess SCK, we too will raise questions about the boundary between SMK and PCK in teachers’ mathematical reasoning. 22 Journal of Education, No. 56, 201222 COATIV refers to the project on Professional Competence of Teachers, Cognitively 5 Activating Instruction, and the Development of Student’ Mathematical Literacy. Construct delineation and validation is a strong feature of quantitative research, and central to the work of Krauss, Baumert and Blum (2008) in their large scale study of secondary mathematics teachers’ professional knowledge and its relationship to learner performance. Based in Germany, their measure development and use in the COACTIV project, like Hill et al., worked from5 the assumption that professional knowledge is situated, specialised, and thus requires assessments that are not synonymous with tests at particular levels of institutionalised mathematics (be this school or university). Indeed, for Krauss Baumert and Blum, secondary teachers’ SMK (what they call Content Knowledge – or CK) sits in a space between school mathematics and tertiary mathematics (p.876), and is clearly bounded from their interpretation of PCK. They report on two hypotheses related to professional knowledge and growing knowledge. They conducted CK and PCK tests on different groups selected with respect to professional knowledge (i.e. mathematical knowledge in and for teaching): two groups of experienced secondary mathematics teachers with different mathematics pre-service training; other teachers (of biology and chemistry); mathematics majors in university; and students in their final 13th year of mathematics study in school. Their results confirmed their professional knowledge hypothesis – experienced teachers irrespective of their teacher education route showed high PCK scores. At the same time, however, mathematics major students performed unexpectedly well on PCK items. Krauss, Baumert and Blum (2008, p.885) explore this interesting outcome in their study – how it was that mathematics major students, who had no teaching training or experience, were relatively strong on their PCK items. We zoom in here to bring into focus the diverse ways in which professional knowledge constructs have been operationalised in the field. For example, some of the examples of Krauss et al.’s PCK items are more aligned with Ball, Thames and Phelps construct of SCK, than with their elaboration of PCK. Although both research groups include knowledge of students, and knowledge of tasks as PCK, their interpretation of these into measures differs. Krauss, Baumert and Blum, for example, exemplify a PCK task item that asks: “How does the surface area of a square change when the side length is tripled? Show your reasoning. Please note down as many different ways of solving this problem as possible”. The sample response given includes both an algebraic and geometric representation (p.889). In Ball, Thames and Phelps’ terms, this response does not require specific or Adler and Patahuddin: Recontextualising item s that m easure. . . 2323 local knowledge of students, nor of curricula, or particular teaching tasks, and hence, in their terms would be SCK, and distinct from PCK. In other words, knowledge of multiple representations shifts between PCK and SMK across these two studies. We do not go further here into other studies that have developed measures of SMK and PCK e.g. TEDS as reported in Tatto and Senk (2011), and MT21 reported in Schmidt, Blömeke and Tatto (2011). While they are in themselves of interest, they do not add to the key issue that emerges from a review of measures research in MKT: this construct and its components are differently operationalised in different studies, a point made by Hill et al., (2007) and noted as a shortcoming in this research. A major reason for this incoherence refers us back to Schoenfeld’s comment that it is necessary that the responses expected on assessment items are made explicit. Descriptions of the principles guiding construct delineation on their own are insufficient. Specifically, what needs to be made clear are the kinds of knowledge and reasoning, be it mathematical or pedagogical, that are being provoked, at least at the level of intention, in and across items, and thus assessed. Readers of quantitative research are not privy to the conceptualising processes behind items exemplified. Even though COACTIV uses open ended and not multiple choice items, as a quantitative study we do not see the mathematical analysis behind the items, nor the detail of the analysis and coding of teachers’ responses. For this would be to reveal the items, and thus muddy any ongoing research related to the measures. We have done our own mathematical analysis of the items we use, which we will argue is a critical step in their recontextualisation. Whether this matches the underlying analysis in their construction is not relevant. However, as most items are confidential, and as will become visible later in the paper, there are constraints on our reporting, and so on developmental cross-over. The WMC-S research and its use of LMT items The two items we describe, analyse and then discuss below were used in a semi-structured interview with 30 teachers participating in the WMC–S project. Both were identified as SMK in LMT and in algebra (a content focus in the project), one on linear equations and one on quadratics. Before we proceed with the detail of the methodology we used, we provide a brief introduction to the project, and our orientation to teachers’ professional knowledge and how this can be read within an interview setting. 24 Journal of Education, No. 56, 201224 W e are indebted to the generosity of LM T, and the time and resources committed to sharing 6 the LM T knowledge and experience with W M C-S. Their workshop at W its, December 2009, enabled the work reported here. WMC-S is a 5-year research-informed and data driven development and research project working with the mathematics teachers in ten schools in one district in Johannesburg, South Africa. Mathematics in use in teaching (i.e. in instruction) is a central focus of WMC-S. Its vision and intervention model has been shaped by previous research on teacher development (Adler and Reed, 2002) and the follow-on work of the QUANTUM project (Adler and Davis, 2006; Adler and Davis, 2011), with its focus on mathematical knowledge in and for teaching. WMC-S professional development (PD) work thus aims to enhance teachers’ mathematical knowledge for teaching. Our practice is guided by deliberate teaching focused on key mathematical objects of learning (Marton, Runesson and Tsui, 2004), and thus with a bias towards SMK as elaborated by Ball, Thames and Phelps (2008) (i.e. CCK and SMK), or towards CK and PCK in Krauss, Baumert and Blum’s terms; and we are researching this process together with the maths teachers in participating schools. Our structuring of the scenarios in the interviews is a function of our orientation to the centrality of evaluation in pedagogic communication (Bernstein, 2000). For Bernstein, pedagogic communication condenses in evaluation (p.36). While communication about practice in an interview is not synonymous with a pedagogic encounter between a transmitter and acquirer, it nevertheless provokes discussion of school mathematical knowledge, and with this legitimating criteria as to what counts as appropriate knowledge in the interview context. As previously argued, teachers call in a range of knowledge resources in their teaching through which we can read how criteria come to work, and so what knowledge is made available to learn. Similarly, what teachers recruit to legitimate what counts as appropriate knowledge in scenario based discussion in an interview setting can be read as their knowledge in use. In this context, the LMT items appeared useful precisely because of their multiple choice format, and the way in which we used these. We asked teachers to consider the scenario and then discuss with us, each of the four multiple choice options offered. Based on our engagement with LMT items in a training workshop the previous year, it seemed productive and efficient to6 use available items in our interviews. As interview items, they were necessarily recontextualised. In addition to situational change, we did not ask teachers to choose an answer and then, thinking aloud, justify their choice. Adler and Patahuddin: Recontextualising item s that m easure. . . 2525 Nor did we present a scenario, and ask for open comment. We gave the scenarios together with the multiple choice responses to teachers two weeks before the interview, and asked teachers to prepare for the interview by reading the scenario and considering each of its four options. In the interview, we would then ask them to share their thinking about each option within each scenario with us. Our purposes here were to set up a context within which teachers would be required to reason, and talk about connected mathematical ideas. We would then be able to develop a dynamic reading of mathematical knowledge for teaching in use in the interviews within and across all the teachers. This recontextualised use marks out its difference from the clinical interviews done for validation purposes in LMT on the one hand, and task based interviews in similar qualitative teacher interviews on the other. LMT has released a set of items for public use. However, the two SMK items that we discuss here were drawn from the full set of items to which we had access for use in the project, but which remain confidential. As we hope will become evident, the items we chose have been very productive in eliciting teachers’ talk and so information from which we could read their knowledge in use. However, as with all other reports on LMT item work, we are required to mask the detail of the items here. This inevitably constrains our reporting. As each item was set in a teaching context, our descriptions below include a description of this context or scenario, followed by the multiple choices. We present Scenario 1 in detail: its mathematical analysis, coded teacher data and discussion of three teachers’ responses. Space limitations necessitate a brief treatment of Scenario 2. Scenario 1 Mathematical analysis: Scenario 1 presents an interaction between Grade 10 learners on solving an equation in the form ax = bx, a, b natural numbers, and b divisible by a . The2 discussion is focused on why, if you can divide both sides by a, can you not also divide through by x to get x = b/a (also a natural number) as the solution? After the presentation of this situation, the LTM item asks respondents to select the most appropriate response from four learner responses that followed in four bulleted points, each with some reasoning for or against this solution method offered. These related to (1) x being a variable, i.e. divisibility 26 Journal of Education, No. 56, 201226 by x is not allowed because is varies (2) x being a real number, (3) finding the square root i.e. the solution requires finding square roots, and (4) composite reasoning involving dividing by 0, and the quadratic form of the equation. As already noted, we changed the requirement for selecting the most appropriate response, asking instead that teachers consider and then discuss with us, their interpretations of each of the four student responses. We were interested in using this item in our interviews to see how teachers engaged with each of the four responses offered, and so with the connected and inter- related concepts embedded in this scenario; specifically, what we would regards as subject matter knowledge related to solving quadratic equations. Mathematical analysis of the item as a whole involved unpacking the general form of a quadratic equation, its structure, roots, together with attention to those features reflected in the multiple choice responses of statements made about the solution strategy. Through this analysis we identified and then coded seven such connected concepts or mathematical ideas embedded in the scenario: (1) the notion of the variable x in the equation (Mv), and awareness that division by a variable could include x = 0 on the one hand and thus a constraint on x would be required, and that while x was a variable, its value was an unknown (2) the real number system (MR) with respect to possible solutions and awareness that the fact of x being a real number was not an argument for its divisibility, and again the constraint of x = 0; (3) the quadratic equation and its two solutions (MQ), and thus that obtaining one solution indicated that the solution was incomplete; (4) non-divisibility of 0 (MÔ); (5) understanding of square root i.e. its meaning and conditions (MSq); (6) demonstration of a method for solving a quadratic equation (MDm); and (7) the understanding of the logic of a composite statement related to division by zero, and thus the ‘loss’ of one solution (MLg). Coding the interview data After a first careful and systematic analysis of all teachers’ talk on this item in the interview, and using the coding above, we found this initial analysis masked the quality of responses, and produced a reading of absences within and across teachers. We thus refined our coding by adding labels positive, zero, and negative to each code. For example, MQ+ means the response clearly indicates recognition of a quadratic equation having two solutions. We used MQ to indicate that there was insufficient data to claim whether or not0 there was such recognition, and MQ- when the teachers’ talk contained incorrect mathematics. Simply, positive (+) indicates correct; zero (0) we Adler and Patahuddin: Recontextualising item s that m easure. . . 2727 The table presents coding for 24 teachers. At the time we did the coding (2011), five teachers7 interviewed in 2010 had moved schools and 1 teacher did not give any answer on this scenario. W e excluded them from our analysis, as they will not be tracked. cannot claim anything with respect to teachers’ SMK with respect to a particular concept; and negative (-) indicates incorrect or a misunderstanding. We tested and then applied this coding across the full set of interview text related to this scenario. Results of our interview analysis: Scenario 1 In the Appendix, we present a summary of our analysis against each of the seven key mathematical ideas discussed above for all the teachers. This result7 of analysis shows clearly that whilst the scenario potentially provokes a wide range of mathematical discussion, such discussion was not present across all teachers. Teacher T10, for example, confidently discussed the quadratic form of the equation, and demonstrated a method to ensure two solutions were obtained. But, he did not engage with x as real variable, hence the 0 coding for both ‘real’ and ‘variable’. This absence of discussion cannot be taken as an absence of knowing. We move on to present three cases in detail, Teacher 19, Teacher 10, and Teacher 3, each of whom had different qualifications and years of experience, but all are qualified secondary mathematics teachers. While their histories and contexts of teaching are significant in the programme, these are not relevant to our purposes here. We have selected these three teachers because together they illuminate the variation of teachers’ responses on identified key mathematical ideas on the one hand, and how these emerged within the social setting of the interview on the other. The responses of the selected three teachers are summarised in Table 1 below, and the coding in the table is used in the analysis of the interview extracts that follow. 28 Journal of Education, No. 56, 201228 Table 1: Coding of three teachers Adler and Patahuddin: Recontextualising item s that m easure. . . 2929 In analysing the interview, we saw potential in the above coding for enabling us to track teachers’ professional knowledge related to this item. We were also interested in the dynamics of the teachers’ mathematical reasoning, and the knowledge resources called in and so how the item played out in the interview setting. Specifically, we were interested in whether teachers’ responses drew on their SMK on the one hand, and how responses were shaped by the way the interviewer presented and then probed in relation to the scenario. In the detailed analysis below we present the full extract from each of the three teachers’ interviews, followed by a brief analysis that connects to the relevant coding categories and highlights key features of the interviews. Teacher 19 had, as requested, looked at the scenarios before the interview, and thus was prepared for the discussion about them. 30 Journal of Education, No. 56, 201230 Table 2: Teacher 19 interview and its coding Interview 19 Inter- view er Teacher’s know ledge I : So there’s this little episode in class and then here are some statements. W hat do you think about these statements? B : Y ou know with the first statement I think the learner does not understand the, doesn’t know the difference between linear and the quadratic so that is why here he divide everything, he divided everything by a which is fine, right, and then now he got this. T his is a quadratic, but then the learner cannot see that it’s a quadratic. H e divided by x, right, whereas we are looking for x. So he divided by x and then it means he lose what - he lose one value of x. So he lost that and then he left with 1 as if it was a linear,. . . I : O K B : So it means the learner does not understand the difference between the quadratic and the linear. I : O K . So what do you think about these statements? B : Y ou see these statements I read and then I couldn’t understand. O K , [Read Bullet 1] . So here if since x is a variable it can differ. I don’t know, I mean yes it can differ because I mean x is a letter, right? Obviously because it’s a quadratic you are going to have two different answers. I : Right B : So here I don’t understand also… So [R e-read Bullet 1] , so of course you can’t do different. I don’t understand that. Actually I didn’t understand that. I : I think what they mean here, and that would be interesting to hear what you think, is that what this child is saying is that the reason why you can’t cancel x here and x there is because here x might be 3 and here x might be 2. B : O h, O K I : I mean is that a right kind of reasoning? B : N o, no, no. I don’t know it’s not, it’s not. ... B : Ja, but like he says, what did the learner want to say here? H e tried to say you can cancel x here on both sides but then, but because the x can vary, you cannot do that. T hat’s what you’re saying. B ut I don’t know. Again I will just insist on saying that it’s because you know what, they just have to know the difference between quadratic and the linear equation. I : O K , alright, excellent. I : So what about this one? [Read Bullet 2] You agree with that? B : B ut then you don’t know yet. Y ou just know that it’s just… You don’t know yet that it’s a real number or it’s going to be a fraction. ... B : Ja, so it cannot be a reason. I : O K . [Read Bullet 3] ? ... B : Like most of the time when they were, O K , O K, where did you, when you have something like that you have to get rid of two. IG P IG P IC – IPV interprets for teacher IPR IPSq PCK – K CS M Q – recognises the quadratic with 2 solutions (implicit) M V +; M Q – States she does not understand option 1, reads it, recognises x as variable, then states quadratic has 2 solutions (explicit). M V 0 – recognises flawed reasoning, but asserted not justified M Q + – dominant justification M R0 – Recognition that the solution could be non real again asserted not justified. (where important idea here is that x could be 0ªR) M Sq+ – Recognise finding the square root Adler and Patahuddin: Recontextualising item s that m easure. . . 3131 I : Right B : So if... Still the problem is that if it’s not that .... because once you do that you can see x is there and x is also there. So if you find the square root of ... here you will find the value of x, but then here as well you will find the x, so what are you actually looking for? I : Right. It’s not helping. B : So it’s not helping. I : O K . And the last one? B : T he last one, [Read Bullet 4] . Y es I can. . . Y ou know what, when I look at this, because these people they don’t understand the difference between B + I : a linear and a quadratic I : so none of these really help I-revoices IPÔ , R, Lg is not leading to solution PCK -K CS; M Q +; Lg- Q uadratic recognition dominates her reasoning; she did not engage with the composite argument (Lg) in B ullet 4. Coding: Interview er utterances IGP – G eneral probe – interview asks what the teacher thinks about each/all statements; IPV, IPSq, IPR, IPÔ . . . specific probes, following the seven key concepts and codes identified above. IC – Clarification related to the scenario What is clear from this interview extract is that the interviewer deliberately led the teacher to discuss each of the four multiple choice statements. For this teacher, we coded three of the key ideas as positive (the notion of variable, recognition of quadratic equation, and the notion of square root); and one negative i.e. her difficulties in understanding the composite statement in Bullet 4. We thus see from Table 2 that even though the interviewer deliberately asked for response to each of the four statements, three aspects (divisibility by zero, Real number system, methods of solving) were coded as zero. This does not mean that this teacher did not know these, but she did not talk about them. The teacher repeats what appears to be her dominant response to this scenario – the importance of recognising that it is a quadratic equation. The interview data here illuminates the recontextualising issues as the item is adapted from asking ‘which one is correct’ to ‘what is your view of each of these statements’. The change opens a space for the teacher to talk about their teaching practice. Teacher 19, as she goes through the four options, talks about her learners and what they do, and what she would emphasise in her teaching. As a result, the item which is identified in LMT as measuring SMK seems here to provoke reasoning and discussion about students and instruction, and so PCK. Indeed, it is her PCK, one could argue, that shapes the dominant response given, and her lack of attention to Bullet 4, which in the multiple choice setting is the correct answer. It involves composite reasoning involving the possibility of 32 Journal of Education, No. 56, 201232 dividing by 0, and ‘losing’ a solution. It is possible to read her PCK focus as a side-stepping or avoiding the mathematics being probed. It is also possible, given her reasoning in the interview, that in a test setting, and having to choose between the four options, Teacher 19 would choose option 4, having eliminated the others. Interestingly, a number of our teachers found the fourth option difficult to interpret and discuss in the interview setting. Thus, from a measurement perspective, the validity of this item comes into question. What begins to emerge from the detailed data and analysis here is that what is produced as teachers’ knowledge in use in relation to this recontextualised LMT item is indeed dynamic and an interaction between the item itself, what the teacher recruits into the discussion, and what and how the interviewer probes the teacher’s responses. Table 3: Teacher 10 interview and its coding Interview T10 (also prepared for interview ) Inter- view er Teacher’s know ledge I : [reading the problem] W hat are your views on those statements? J : (laughs) I : O k, let’s go with the first one. W hat do you think of the first bullet? J : [Read Bullet 1] W ell, I think my approach to this one would be most of these things are not right. If you look at the question there, it’s times a, it’s a times a. Once it is this one here it is important for learners to know this is a quadratic thing. H ow many answers? Ask your learners. “T wo answers.” G ood, you must get two answers; your x has got to have two, um, answers, real numbers there, whatever it is. So these learners who are saying: “W e divide by x what what what”... Yes I understand but that’s not the procedure. T he procedure is it’s a quadratic, ok. B ecause quadratic therefore transposes the x, the bx to the other side, equate it to zero, quadratic equations must be eq ual to zero. Do you have seen that quadratic equation? “N o, we don’t have seen.” T hereafter, ok, fine, what do you do next? “Look for a common factor.” I : B ut what do you think about that learner who says: [Read Bullet 1] . W hat do you think of that? J : T hat learner yes is partial because the main idea is for this learner to get x, alright? I : M mm J : So he is saying: “N o, why should I worry? If I eliminate one x but I still going to remain with x then I’m still, I’m still fine, because what is required is an x”. Do you see that? I : M mm J : So the argument of that learner there you don’t need to take it lightly, you see? B ut unless you go back to the mistakes of the learners, what are their errors? I : M mm J : T hat one must emphasise it’s quadratic, it must be two answers, right? IPG IPV IPV IPR -Partial M Q + Quadratic recognition M D m – M ethods of solving PC K – KCS M Q + Adler and Patahuddin: Recontextualising item s that m easure. . . 3333 I : O k, and if [Read a part of Bullet 2] ? J : If (reads part of bullet 2) I : M mm J : Right? You remain with x on the left. I : M mm J : And no, and no ri… and no x on the right. So you’ve got only one x. I : O k J : Y ou get one answer. I : O k J : N o, two answers. So that must be very clear to the learners. You eliminate one x there, fine; you remain with one x, isn’t it? Fine. B ecause this learner has a very good reason to say: “Hey, I can eliminate the 2, right? D ivide both sides by 2. W hy not dividing by x both sides?” So the one learner who says: “N o, no, I’m not dividing by 2x both sides.” D o you understand? I : M mm J : T he procedure of that learner is not wrong. It’s also correct. B ut it’s quadratic, it must have two answers. T hat one we must make it very clear, because those are some of the things that we need to understand. I : So what do you think about the fourth bullet then? [Read Bullet 4] J : T hat’s again. T ake, um, taking our learners back to the real numbers and non-real numbers, we have dealt with that one there. I : M mm J : It is not permissible to divide by zero. Not permissible at all. T hat’s a mathematical, a mathematical suicide. D on’t ever do it, don’t divide by zero. The situation is undefined. Do it on your calculator. Divide by zero; error. Divide by zero; error. W hy? N ot allowed. You cannot do it. I : M mm J : Y ou cannot divide by zero. IPÔ , R, Lg, M Q + PCK – reasoning about learners, B U T possibility of x = 0 not considered. M Ô + – divisibility by 0 We coded T10 as having three of the seven key ideas correct, three as zero, and one as negative. At the same time, while T10 and T19 reasoned in similar ways in their interviews, the interviewer probes were not consistent, and so different ideas were provoked for discussion. Table 3 shows that there was no explicit probing of the third option related to the notion of a square root and so this is absent in T10’s interview. Also the second bullet or option was only partially probed. As a result, this teacher did not talk about the idea of finding the square root of each side, and therefore we coded zero for understanding of square root. In this context, while the teacher did talk about variable values for x, divisibility by zero and the quadratic equation, his reasoning was shaped not only by the interview probes or non-probes but also the opportunities opened up for him to talk about his experiences of learners and his preferred teaching methods in dealing with this particular problem. As a result, many of the mathematical ideas embedded in this item are coded as zero to indicate that we 34 Journal of Education, No. 56, 201234 do not have data on these. As with Teacher 19, the fourth bullet proved to be difficult for this teacher. However, this option opened up the opportunity for him to talk about the notion of divisibility by zero. It is also interesting to note here that while both T10 and T19 emphasise the importance of recognition of the quadratic equation, they do so in different ways. T19 emphasised learners (mis)understanding; T10 emphasised how to teach. And again, what emerges is blurring of PCK and SMK in the teachers’ reasoning. What is thus produced as knowledge in use is clearly co-constituted by the item itself, the interviewer probes, and what the teacher recruits. Teacher 3 too had prepared, and her responses evidence weaker SMK. Table 4: Teacher 3 interview and its coding Interview T3 Interview er Teacher’s know ledge I : [Read the scenario] W hat is your view of each of these statements? T he statements where the teachers asked the rest of the learners to say their views about what these other learners were debating about? M aybe if you look at the first one, [Read Bullet 1. T : T his one, no. over here we are looking at x as the common factor, as the real… x as the common factor, meaning that as a common factor, as a common variable. So it is possible for them to divide throughout. So the first statement it’s out. If I have 2x… if I have this I can divide throughout by x. And then they say [Read Bullet 2] . You can say yes it’s a real number but then we don’t know the value of that real number so yes it’s a common factor, we can divide throughout the x. And then [Read Bullet 3] . B ut then if you take the square root of [], or you say by then, the one thing that if you’re working with square root, most of them, some of them it confuses them… some not most, some. It confuses them. So if they find… especially when they have to find the value of x, they can only. . .they love working with square root, squares and square roots, or cube and cube roots, but then when you have to find the value of x and you get the comma something something, it even looks wrong to them. ... T hen they say [Read Bullet 4] . M eaning that the value of x that you are dividing with might be zero. [Re-read Bullet 4] . It’s possible. U nlike when you take the other side, when you transpose the [], then look for the common factor, yes, you’d find two solutions, that x is equal to [], equals to [], and x is equals to []. B ut then you divide both sides by 2x, you only get one solution, [] … I : So at the end of it all, what is your response to this learner number two who says, you cannot divide both sides by x? T : N o, you can. You can divide both sides by x. IG P IPV – IC clarification M V 0 M R0 PC K – KCS M sq0 M ethods of solving – M D m Q uadratic recognition – M Q Real Number-M R- Adler and Patahuddin: Recontextualising item s that m easure. . . 3535 Table 4 shows that there was far less probing in this interview – a function of the teacher responding to all four bulleted statements without interruption. At the end, the interviewer asked “at the end of it all, what is your response to this learner number two who says, you cannot divide both sides by x?”, and so goes back to the second option but not bullets 3 or 4. This may be the most critical question for the interviewer, who possibly noticed the contradiction in the teacher’s responses: T3 realised that dividing by 2x lead to only one solution but previously said that “we can divide throughout the x”, an argument she commits to. So here only one of the 7 mathematical ideas – a demonstration of the procedure to solve the quadratic equation – is coded as correct. Interestingly this demonstration is offered as the teacher responds to the fourth option. Here too we don’t have access to the teachers’ thinking on the correctness of the last statement. We do have access to her view that division by a variable ‘on both sides’ is possible. Compared to T10 and T19, T3 engages less with the elements embedded in the item, and the interview shows that only one aspect was coded positive even though it was not accurate. T3 has a clear idea about the procedure of solving this type of equation, but the connected set of ideas related to a quadratic equation is not evident. Elaborating our analysis briefly through Scenario 2 Scenario 2 is in the form of a teacher asking her learners to construct a story that would be appropriately modelled by a linear equation of the form y = ax + b, x= 1, 2, 3, ..., where a and b were given. Four possible stories are provided, and in the interview we asked teachers to consider each of these and which would be appropriate for the given equation. The first two of the stories generate the wrong sequence and the next two options generate the right sequence. The stories are obscured for confidentiality and presented in a general from. ! Option one uses b as a starting point in the story and plays with the phrase ‘make twice as many’ to produce b, 2b, 2(2b), .... 2^n (b). ! Option two also generates a different sequence, here using a as a starting point, to which b is added. This played with the words ‘each time’, to produce the sequence: a+b, 2(a+b), 3(a+b), 4(a+b) .... n(a+b). 36 Journal of Education, No. 56, 201236 ! Option three is the appropriate story playing with the words ‘two more’, and generating the sequence b+2, (b+2)+2, ((b+2)+2)+2 ..... b + 2n, n = 1, 2, 3, .... which is well-matched with the linear equation given. ! The story in option 4 generates a similar numerical sequence but requires a different model. The starting point of the story is the number ‘c’, where c = a+b. So c�a and c�b. It used the words ‘two more each day’, generating a sequence: c, c+2, (c+2) +2, ((c+2)+2)+2 ..... , and thus the same pattern as option 3. However, the model for this is y = c + (x-1)2, which is equivalent with y = ax + b. Scenario 2 is also identified in LMT as measuring SMK. We have provided as much detail above as possible to reveal the power of this item in the construction of the distractors, and particularly in how language is used in the four stories. For example, while all four ask a similar question at the end of the story, there is a play on ‘two more’, ‘twice as many’, and ‘two more each day’ across the options, thus probing the use of the mathematics register. Most of the teachers interviewed found this question difficult. This observation is based on our analysis across the interviews where there was less talk about the item and its options. In addition, some of the teachers stated their difficulty e.g. “these are hard”. Zooming in here on T3, T10 and T19, an additional observation is that, in contrast to Scenario 1, none of these three teachers recruited pedagogic knowledge (i.e. knowledge of teaching or of learners) to discuss the item. They were preoccupied in understanding the story problems, and their related sequences, and then discussing their views of each of the stories in relation to the given equation. T10 and T19 talked through how they worked out the sequence generated by each story. This then led to them being able to identify the incorrect stories. Distinguishing between those that generated the same sequence was more difficult. T10, in fact, was one of the teachers who stated that this problem was difficult for him, and that he does not focus on “word problems” in his teaching. The greater difficulty with this item was evident in T3’s interview where we have no data about her thinking of the four story options. While her engagement in Scenario 1 was more limited than T10 and T19, she nevertheless had much to say. Here, in Scenario 2, however, it seems the teacher struggled and the interviewer did not probe further. Adler and Patahuddin: Recontextualising item s that m easure. . . 3737 W e can posit here (though this is beyond the scope of this paper) that the kind of analysis we8 have done enables us to develop a two-dimensional continuum across which we could provisionally place teachers’ responses in this interview, and that can be contrasted with their responses on these items at a later point, and after participation in the project. W e are still in the process of developing this continuum and clear descriptions of points related to PCK and SM K as these emerged in the interview setting. The placing of teachers would, of course, need to include awareness of where we did not have data in these initial interviews, and disclaimers that these are not ‘measures’ of teachers’ knowledge, but qualitative indicators of knowledge in use in an interview setting. This assessment will combine with other teacher data in the project. What is clear here is that the differential familiarity across Scenarios 1 and 2 provide different access to teachers SMK in the interview setting. This raises questions then about how more difficult items are dealt with in the interview setting. Discussion We remind readers that our interest in this paper is not the teachers’ knowledge per se. Our interest is in the potential of items, developed and validated for purposes of measurement and correlation with learner performance and teaching quality, for illuminating teachers’ mathematical knowledge for teaching in an interview setting. We discussed how the use of the multiple choice format suited our project and orientation to teachers’ knowledge in use. We are seeking a dynamic descriptive assessment of teacher professional knowledge; but more importantly, we have interrogated particular items we have used to critically reflect on what such recontextualised use does. What reading is produced about teachers’ knowledge in use? How does this ‘fit’ with our purposes and with what constraints? 8 The first point we wish to make is that, despite the difficulties of not being able to present the items in full, we have shown that in the scenario, together with the multiple choice possibilities that follow, there is a rich set of mathematical ideas or concepts connected to the item. Our mathematical analysis of the LMT items shows their mathematical potential, precisely in the combination of a mathematics problem and a varied set of possible solutions, not all of which are correct. The item provides the possibility of exploring a connected sets of ideas with teachers, and so an important element of their SMK. Working productively with quadratic equations, for example, rests on 38 Journal of Education, No. 56, 201238 key ideas like: divisibility by a real variable and so too by zero; square roots; as well as a procedure that can be used to solve these equations. Secondly, and linked to the first point, is that structuring the interview so that each of the four multiple choice options is discussed not only opens the space for teachers to talk, but it provokes directed talk and related reasoning. Looking across the responses of the teachers, both in the Appendix and Table 1, and the detailed data for Scenario 1, we see that the LMT items, as we used them, provide the possibility for teachers to talk about the range of mathematical ideas embedded in the item. This possibility was not evenly realised, in many cases because the interviewing was uneven. Not all interviewers deliberately asked teachers to focus on each of the four options in each of the scenarios. What we have learned from our analysis is that optimising the potential of these items in an interview setting requires a semi- structured interview that is explicit in what the interviewer needs to ask and then probe. It is possible, with greater structure and consistency in the interview, to create the conditions that all interviewees engage with and have to justify their thinking in relation to a connected set of ideas. Furthermore, our analysis of Scenario 2 suggests that it might be necessary to further structure the interview when the item is more difficult and less familiar. A probe in the interview on Scenario 2, that steers teachers to generate the sequence for each problem before they relate this to the model, and consider its appropriacy, could provide for further SMK related talk. We thus contend that there is much to be gained from using carefully constructed multiple choice items in an interview setting. What we point to above, is the importance of being explicit about what is being assessed in the scenario or item, and thus concur with Schoenfeld (2007) that the lack of specification of expected responses is a limitation to the measures work. In this work, we need to be clear, at least at the level of intention, at what it is we are attempting to access and assess. In the context of this paper, and Hill et al.’s call, development crossover requires such explication. Thirdly, our use of the items has shown the complexity in marking out a hard boundary between PCK and SMK, and SCK (specialised content knowledge) in particular. LMT stands by its distinctive categories within MKT, while acknowledging that much work is needed for clearly delineating KCS (knowledge of content and students) in particular. What our interviews have shown is that in the interview setting, the situating of the subject matter Adler and Patahuddin: Recontextualising item s that m easure. . . 3939 knowledge in teaching contexts, particularly if this was a familiar context, lead teachers to recruit pedagogical considerations into their rationales, and these shifted between KCS (thus knowing what to focus on because of errors learners will make e.g. they don’t recognise the quadratic form), and/or KCT (how to teach so that learners discern the quadratic form and the related procedure). And this is not simply a function of the interviewer steering the talk in one or other direction. As noted above, for T19 and T10, their confidence with their pedagogical knowledge (T19 of her students thinking; and T10 with what must be emphasised in teaching), was highly visible in the interview. Only further probing could elicit whether their PCK emphasis in the interview was a mathematical avoidance strategy. The introduction of the item in the interview with T19, and its probing by the interviewer focused directly on interpretation of the options, without reference to the teachers’ classroom. It would require skilled interviewing to focus attention on the mathematics in the item. And again, Schoenfeld’s comment has salience. LMT is not explicit in exactly what they are assessing and how the distractors provide for such assessment. Our analysis of Scenario 1, and then the teachers’ responses to this suggests that it would be possible, perhaps even likely in a test setting that Bullet 4 can be selected by elimination, and not through the composite logical reasoning entailed in the statement. It is precisely these complexities of measurement that motived our consideration in this paper, and of course, in our practice, of whether our use of the items is indeed ‘fit’ for our purposes. Our reading of SMK in use with respect to connections across a range of mathematical ideas and related reasoning across the interviews, convince us of the difficulty of being able to clearly mark out and measure each of SMK and PCK, and particularly aspects of SCK as defined by Ball, Thames and Phelps. That COACTIV and LMT use different interpretations and operationalisations of SMK and PCK in particular, require that as a field, we critically interrogate generalised conclusions about the inter-relation of these domains of professional knowledge. Finally, our discursive orientation to knowledge cautions our descriptions by underscoring that what is produced as teachers’ mathematical knowledge through our use of the items in an interview setting is an interaction between the item itself, the knowledge resources the teacher recruits into the interview, and the way in which the interaction in the interview unfolds. Assessments, wherever they are, and however they are used, are a discursive product, and not a transparent view into a teacher’s ‘mind’. 40 Journal of Education, No. 56, 201240 Acknowledgement This work is based upon the research of the Wits Mathematics Connect Project at the University of the Witwatersrand, supported by the FirstRand Foundation Mathematics Education Chairs Initiative of the First Rand Foundation the Department of Science and Technology and the National Research Foundation (NRF). Any opinion, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the FRF, DST, or NRF. We are grateful to the teachers who engage with us so productively in the project, and generously give of their time to our research interests. We also thank the reviewers of this paper for insightful and generative feedback. References Adler, J. and Davis, Z. 2011. Modelling teaching in mathematics teacher education and the constitution of mathematics for teaching. In Rowland, T. and Ruthven, K. (Eds), Mathematical knowledge in teaching. New York: Springer, pp.139–160. Adler, J. and Reed, Y. 2002. Challenges of teacher development: an investigation of take-up in South Africa. Pretoria, South Africa: Van Schaik. Adler, J. and Davis, Z. 2006. Opening another black box: researching mathematics for teaching in mathematics teacher education. Journal for Research in Mathematics Education, 37(4): pp.270–296. Alonzo, A.C. 2007. Commentary: Challenges of simultaneously defining and measuring knowledge for teaching. Measurement, 5(2/3): pp.131–137. Ball, D.L., Thames, M.H. and Phelps, G. 2008. Content knowledge for teaching: what makes it special? Journal of Teacher Education, 59: pp.389–407. Bernstein, B. 2000. Pedagogy, symbolic control and identity: theory, research, and critique (Revised edn.). Lanham: Rowman and Littlefield Publishers, Inc. Borko, H., Eisenhart, M., Brown, C.A., Underhill, R.G., Jones, D. and Agard, P.C. 1992. Learning to teach hard mathematics: do novice teachers and their instructors give up too easily? Journal for Research in Mathematics Education, 23(3): pp.194–222. Adler and Patahuddin: Recontextualising item s that m easure. . . 4141 Hill, H.C. 2008. Unpacking pedagogical content knowledge: conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4): pp.372–400. Hill, H.C., Blunk, M.L., Charalambos, Y.C., Lewis, J.M., Phelps, G.C., Sleep, L. and Ball, D. 2008. Mathematical knowledge for teaching and the mathematical quality of instruction: an exploratory study. Cognition and Instruction, 26: pp.430–511. Hill, H.C., Rowan, B. and Ball, D.L. 2005. Effects of teachers’ mathematical knowledge for teaching on student achievement American Educational Research Journal, 42(2): pp.371–406. Hill, H.C., Schilling, S.G. and Ball, D.L. 2004. Developing measures of teachers’ mathematics knowledge for teaching. Elementary School Journal, 105(1): pp.11–30. Hill, H.C., Sleep, L., Lewis, J.M. and Ball, D.L. 2007. Assessing teachers’ mathematical knowledge: what knowledge matters and what evidence counts? In Lester, J.R.K. (Ed.), Second handbook of research on mathematics teaching and learning. Charlotte, NC: Information Age, pp.111–155. Huillet, D. 2009. Mathematics for teaching: an anthropological approach and its use in teacher training. For the Learning of Mathematics, 29(3): pp.4–10. Krauss, S., Baumert, J. and Blum, W. 2008. Secondary mathematics teachers’ pedagogical content knowledge and content knowledge: validation of the COACTIV constructs. ZDM, 40(5): pp.873–892. Leinhardt, G. and Smith, D.A. 1985. Expertise in mathematics instruction: subject matter knowledge. Journal of Educational Psychology, 77(3): pp.247– 271. Ma, L. 1999. Knowing and teaching elementary mathematics: teachers’ understanding of fundamental mathematics in China and the United States. New Jersey: Lawrence Erlbaum Associates. Marton, F., Runesson, U. and Tsui, A.B.M. 2004. Classroom discourse and the space of learning. New Jersey: Lawrence Erlbaum Associates. 42 Journal of Education, No. 56, 201242 Nardi, E., Biza, I. and Zachariades, T. 2012. ‘Warrant’ revisited: integrating mathematics teachers’ pedagogical and epistemological considerations into Toulmin’s model for argumentation. Educational Studies in Mathematics, 79(2): pp.157–173. Schmidt, W., Blömeke, S. and Tatto, M.T. 2011. Teacher education matters: a study of middle school mathematics teacher preparation in six countries. New York, NY: Teachers College Press. Schoenfeld, A.H. 2007. Commentary: The complexities of assessing teacher knowledge. Measurement, 5(2/3), pp.198–204. Shulman, L.S. 1986. Those who understand: knowledge growth in teaching. Educational Researcher, 15(2): pp.4–14. Tatto, M.T. and Senk, S. 2011. The mathematics education of future primary and secondary teachers: methods and findings from the teacher education and development study in mathematics. Journal of Teacher Education, 62(2): pp.121–137. Zazkis, R. and Leikin, R. 2010. Advanced mathematical knowledge in teaching practice: perceptions of secondary mathematics teachers. Mathematical Thinking and Learning, 12(4): pp.263–281. Jill Adler and Sitti Maesuri Patahuddin Marang Centre for Mathematics and Science Education University of the Witwatersrand Jill.Adler@wits.ac.za Sitti.Patahuddin@wits.ac.za mailto:Jill.Adler@wits.ac.za mailto:Sitti.Patahuddin@wits.ac.za Adler and Patahuddin: Recontextualising item s that m easure. . . 4343 A summary of the interview analysis of 24 teachers Appendix: Table 3: Teacher 10 interview and its coding Table 4: Teacher 3 interview and its coding