Microsoft Word - 30-40 Taxonomy-matters-Berger.docx 30 Pythagoras, 71, 30-40 (July 2010) Taxonomy Matters: Cognitive Levels and Types of  Mathematical Activities in Mathematics Examinations    Margot Berger, Lynn Bowie & Lovemore Nyaumwe  Marang Centre for Mathematics and Science Education  University of the Witwatersrand  margot.berger@wits.ac.za; lynn.bowie@ wits.ac.za & nyauml@unisa.ac.za    We argue that there are intrinsic difficulties in using the Subject Assessment Guidelines  for Mathematics  (SAGM)  for evaluating  the standard of a South African matriculation  mathematics examination and  for determining how  it aligns with  the curriculum. Our  argument has two prongs. First, the SAGM taxonomy conflates cognitive level with type  of mathematical activity; we contend that such a conflation leads to problems in assessing  the complexity of the examination item. Second, the SAGM taxonomy provides no space  for  important mathematical activities such as  justification and conjecturing despite  the  promotion  of  these  activities  in  key  curriculum  documents  (Department  of  Education,  2003). This absence may obscure weak alignment of the examination and the curriculum.  These arguments are  illustrated  through examples  taken  from  the 2008 Department of  Education  (DOE)  and  Independent  Examination  Board  (IEB)  Grade  12  Mathematics  examination papers. We also examine other well‐known taxonomies in order to discern  those  aspects  of  a  taxonomy  which  may  be  useful  for  evaluation  and  alignment.  We  conclude with the construction of a matrix that could provide an appropriate taxonomy  for the South African matriculation Mathematics examination. It has activities on one axis  and levels of complexity on the other.   For any high-stake national school matriculation examination there is a need for a set of criteria which can be used to determine the difficulty level of examination items and how the examination aligns with the curriculum. This is necessary in order to assess the examination’s fairness as a tool for selection of learners for admission to further study or careers and as a measure of a nation’s mathematical competence. In South Africa, the national Department of Education publishes Subject Assessment Guidelines for Mathematics (SAGM) that outlines the criteria and expected weightings of Learning Outcomes stated in the National Curriculum Statement (DOE, 2003). DOE (2008a) also publishes a SAGM taxonomy designed for use in constructing and assessing school leavers’ final mathematics examination. This paper argues that there are intrinsic difficulties in applying the SAGM taxonomy. The argument is based on two major reasons. First, and in common with many other taxonomies, the SAGM taxonomy (DOE, 2008a) conflates cognitive level with type of mathematical activity. In the SAGM taxonomy it is assumed that cognitive level increases with the type of mathematical activity; that is, memorisation has the lowest cognitive level, then routine procedures, then complex procedures, then problem solving. This assumption is questioned in this paper. Second, the SAGM taxonomy provides no space for key mathematical activities such as justification, conjecturing, and communicating mathematical ideas despite the National Curriculum Statement’s (NCS) promotion of these activities as important components of the curriculum (DOE, 2003). Absence of these mathematical activities within the SAGM could lead to weak alignment of the examination and NCS. Margot Berger, Lynn Bowie & Lovemore Nyaumwe 31 The two arguments stated above are illustrated through examples taken from the 2008 DOE and IEB Grade 12 Mathematics examination papers (DOE, 2008b, 2008c; IEB, 2008a, 2008b). The arguments are also broadened and deepened by considering other taxonomies such as Porter (2002), Stein, Smith, Henningsen and Silver (2000, 2009) and National Assessment of Educational Progress (NAEP) (National Assessment Governing Body (NAGB), 2008). Towards the end of the paper we theorise a desirable matrix that could provide an appropriate taxonomy for assessment in the NCS. Such a taxonomy would have mathematical practices advocated by the NCS on one axis and different levels of complexity on the other axis. Background At the end of 2008, South African Grade 12 school learners wrote the DOE National Senior Certificate (NSC) examinations for the first time. These examinations were based on the new NCS. The DOE Mathematics examination was written by about 300 000 learners and provoked considerable public interest in terms of its standard, partly because it was a new form of examination. Up until the end of 2007, students taking Mathematics as a school-leaving subject studied Higher Grade Mathematics or Standard Grade Mathematics. Higher Grade Mathematics was of a higher academic standard than Standard Grade Mathematics and it was generally considered a necessary preparation for further study in engineering or the sciences at tertiary level. Under the new NCS system, Higher and Standard Grade Mathematics have been merged into one subject, Mathematics. The concern of civil society and the tertiary education sector was to what extent this lowered the standard of Higher Grade Mathematics and what mark could be considered as acceptable for entry into various university courses. Two examination boards, DOE and IEB, exist in South Africa. The majority of schools in South Africa are public institutions and mostly write DOE examinations; private schools write IEB examinations. The standard of the 2008 Mathematics examination, particularly the DOE examination, has been the subject of a number of comments (Concerned Mathematics Educators, 2009; Dibetle, 2009; Maree, 2009) and reviews (Ministerial Panel, 2009; Umalusi, 2009) and is not the focus of this paper. However these reviews rely heavily on the SAGM taxonomy (DOE, 2008a). The application of this taxonomy to the examination, together with our exploration of alternative taxonomies, leads us to address the issue: How does the use of different taxonomies enable us to assess both the standard of a mathematics examination and the spread of different types of mathematical activities in that examination? Different taxonomies Within mathematics, a number of different taxonomies have been developed for various purposes. These purposes may be related to the cognitive levels and/or types of mathematical activities and tasks in an assessment (DOE, 2008a, National Assessment Governing Body, 2008), classroom tasks (Stein et al., 2000) or alignment of curriculum with assessment (Porter, 2002)1. Although the purposes of some of these taxonomies are not specifically assessment, we believe that aspects of these different taxonomies suggest useful views of the relationship between taxonomies and assessment. We use them to highlight certain arguments about what features of taxonomy may be useful for setting and evaluating items in the NSC examinations. The SAGM taxonomy is based on the 1999 TIMSS mathematics survey (DOE, 2008a). Clearly we wish to examine this taxonomy since this is the official taxonomy which informs the setting of the Grade 12 DOE Mathematics paper. The SAGM taxonomy uses categories of knowledge, routine procedures, complex procedures and problem-solving, each with their own descriptors. It suggests the approximate proportion of the examination paper which should be allocated to each category. 1 In the Appendix we present a summary version of these taxonomies. Taxonomy matters: Cognitive levels and types of mathematical activities 32 The Porter (2002) taxonomy has five categories: memorisation, perform procedures, communicate understanding, solve non-routine problems and conjecture, generalise and prove. Porter’s taxonomy is widely used and a respected tool in the USA (Hess, 2006, Porter, 2002) and some of its categories refer to particularly important mathematical practices. For example, categories such as “conjecture, prove, and generalise” are relevant to the NCS’s vision which states: “competence in mathematical process skills such as investigating, generalising and proving is more important than the acquisition of content knowledge for its own sake” (DOE, 2003, p. 9). Another important category of Porter’s scheme not evident in the SAGM taxonomy but again contained in the NCS’s critical outcomes is the need for learners to be able to communicate mathematically (DOE, 2003, p. 2). Porter’s category “demonstrate understanding of mathematical ideas” refers to a learner’s ability to communicate or explain mathematical ideas and so addresses this critical outcome. Stein et al. (2000) designed a taxonomy consisting of four categories, namely, memorisation, procedures without connections, procedures with connections and doing mathematics. Although this taxonomy’s primary purpose is for analysing classroom rather than assessment tasks, we believe it illuminates important distinctions in types of mathematical tasks. In particular we found the distinction they made between the categories “procedures with connections” and “procedures without connections” particularly useful. “Procedures without connections” refer broadly to “tasks that require students to perform a memorised procedure in a routine manner” (Stein et al., 2009, p. 1). “Procedures with connections” refer broadly to “tasks that demand engagement with concepts and that stimulate students to make powerful connections to meaning or relevant mathematical ideas” (ibid.). SAGM (DOE, 2008a), Stein et al. (2000) and Porter (2002) all use some form of mathematical practice or activity as their taxonomical categories, for example, routine procedures, memorisation, nonroutine problems, procedures with connections, and conjecture, generalise and prove. Furthermore, they merge these practices with cognitive levels. As we later demonstrate, this conflation of the type of the thinking (as described by the practice, for example, doing procedures or proving and so on) with level of thinking (as described by the complexity of the item) makes it very difficult to use these taxonomies as tools for measuring the standard of an examination. We will show that in order to have an effective taxonomy which can be used to measure both type and level of thinking, these two attributes need to be separated. In contrast, NAEP (NAGB, 2008) focuses on complexity levels of items only (not mathematical practices). NAEP (NAGB, 2008, p. 18) describe these levels as the “specific level of thinking” which the item is designed to measure. Thus mathematical complexity level refers to an attribute of a task which measures the level of thinking required to complete the task. The NAEP taxonomy distinguishes between three levels of complexity: low, moderate and high. Low complexity items require students to recall or recognise concepts and/or to perform routine procedures. Moderate complexity refers to items for which the method of solution is not directly given; the learner needs to decide on how to approach the problem. More flexibility of thinking is required compared to the low complexity category. Items with high complexity require that students use reasoning, planning, analysis, judgment, and creative thought. In addition they may be expected to generalise or to justify mathematical statements or construct mathematical arguments. Placement of a task in an appropriate category using any of the above taxonomies can be tricky (Stein et al., 2000). Besides other considerations, the level of thinking required by a task depends on the familiarity of a learner with the mathematical content of the task (NAGB, 2008). For instance, learners must have encountered a mathematical procedure many times in order to achieve the “routine procedure” cognitive demand level. Those learners who are not familiar or lack experience with the tasks cannot successfully achieve this cognitive demand level. Realisation that demands of a task in part depends on the experience of learners necessitates that due consideration be given to learner age and grade level when deciding the cognitive demand level of a task. Furthermore, and as we demonstrate below, conflation of cognitive level with type of thinking required, is problematic. Margot Berger, Lynn Bowie & Lovemore Nyaumwe 33 Applying the taxonomies to the NSC examinations We explored the four different taxonomies by applying them to each item in 2008 DOE and IEB Mathematics Examinations papers and, when necessary, reaching consensus on a categorisation. From this taxonomical analysis, we chose exemplars (presented below) to illustrate our arguments. Do the taxonomies provide a useful way of evaluating the standard of the examinations? The Stein, SAGM and Porter taxonomies purport to reflect the cognitive demand of tasks. Stein et al (2009, p. 1) define cognitive demand as “the kind and level of thinking required of students in order to successfully engage with and solve a task”. SAGM and Porter assume a similar conflation of kind and level of activity. In Stein’s definition “level” would seem to imply that we can take cognitive demand as an indicator of difficulty, but the fact that it is coupled with “the kind of thinking” gives pause for thought. Indeed our experiences of applying the different taxonomies to analysing the paper suggest that the conflation of the notions of “levels” and “kinds” of thinking makes this chain of deduction less straightforward. We illustrate the difficulty of applying the taxonomies when level and kind of thinking are conflated, through an example which focuses on the Stein category “procedures with connections”. Question 5.3 (DOE, 2008b) states (the learners have been given that 8)1(2)( 2  xxf and have drawn this graph in question 5.1): 5.3 The graph of f is shifted 2 units to the LEFT. Write down the equation of the new graph. The fact that a learner has to make connections between different representations led us to classify this problem as a “procedure with connections” task in the Stein taxonomy (a high level cognitive demand category). It is, of course, possible to do this task by recourse to a learnt algorithm (i.e. if a graph shifts 2 units to the left, replace x with x + 2). However as we cannot determine how a learner will do the task we have to rely on the descriptors of the category, which suggest that tasks that involve links between different representations are classified as “procedures with connections”. The fact that shifting graphs and its effect on the algebraic representation for a function is a stated outcome in the curriculum and that this kind of problem would thus be very familiar to learners, led us to classify this question as a routine procedure in the SAGM taxonomy (a low level cognitive demand category). What we see here is a focus on two different aspects of the cognitive demand of the task. Noting the connections between different representations that needs to be made seems to prioritise the “kind of thinking” that is needed. Noting that the task is familiar seems to prioritise the “level of thinking” that is needed. In a related fashion, DOE (2008a) refers to the SAGM taxonomy of categories (memorisation, routine procedures, complex procedures and problem solving) as both taxonomical categories (p. 12) and cognitive levels (p. 13). Again we suggest that the conflation of kind and level of thinking in SAGM taxonomy makes it difficult to use the taxonomy to evaluate an examination and that it will probably also make it difficult to use the taxonomy to construct an examination. We illustrate our arguments using two sets of examples drawn from IEB and DOE Mathematics Paper 1 (IEB, 2008a, DOE, 2008b). Each example shows how different items placed in the same SAGM taxonomical category may have very different complexity levels. Question 7(b) (IEB, 2008a) and Question 1.3 (DOE, 2008b) were both classified as “solving problems” in the SAGM taxonomy and “non-routine problems” in the Porter taxonomy. 7 (b) Given 6 6 6 6 ...x      (1) Write down x2 in terms of x. (2) Hence determine the value of x. Taxonomy matters: Cognitive levels and types of mathematical activities 34 1.3 Given x = 999 999 999 999, determine the exact value of 2 4 2 x x   . Show ALL your calculations. We suggest that Question 7(b) is of high complexity because its notation is unfamiliar; its method of solution is not direct and the learner needs to treat 6 6 6 6 ...    as an object rather than a process. That is, to solve the problem, the learner needs to start by squaring both sides of the equation, 6 6 6 6 ...x      . This leads to the equation 2 6 6 6 6 6 ...x       which is equivalent to 2 6x x  . Solving 2 6 0x x   gives 3x  or 2x   . But 2x   since x is a positive square root. In contrast we suggest that Question 1.3 is of moderate complexity. The expression 2 4 2 x x   should be familiar to many students. Complexity lies in the need for the student to ‘see’ that factorisation and cancellation will transform the expression into a simple form, 2, 2x x  . To solve the problem, the learner just needs to substitute 999 999 999 999 into 2x  to get 1 000 000 000 001. These two examples illustrate how the Porter and SAGM taxonomies may obscure subtle differences in complexity between these two questions. In this example, Stein’s categories of “doing maths” and “procedures with connections” do distinguish between the different nature of these two questions. Question 7(b) is “doing maths” since it requires non-algorithmic thinking; in addition it requires that the learner understands the nature of certain mathematical concepts and processes (and the interplay between the process and the concept). Question 1.3 is “procedures with connections”: its execution uses procedures and in so doing highlights the relationships between factorisation, canceling and substitution. We use one last example to further illustrate how taxonomies which conflate complexity and type of mathematical activity may obscure differences between levels of thinking in a particular item. We consider Question 7(c) and Question 4(c)(2) (IEB, 2008a). According to SAGM taxonomy both questions are “complex procedures”. However we suggest that this classification obscures the different levels of thinking (complexity) required. Question 7 (c) reads: 7(c) Given: ( ) 3f x x , find a simplified expression for    11 1( ) .f x f f x x f x        In order to solve this problem, the learner needs to deal with each expression in turn. Computing 1 f x       requires an understanding of how to work with functional notation; this understanding may be purely procedural. That is, to obtain 1 f x       , the learner substitutes 1 x for x in the expression 3x. Evaluating   1 f x similarly requires procedural knowledge (substitute 3x for f(x)). Finding the inverse function  1f x similarly is procedural. Ultimately the learner needs to add these expressions together (a computational task). Given the focus on the use of procedures to simplify several expressions, we regard this task as having moderate complexity. Margot Berger, Lynn Bowie & Lovemore Nyaumwe 35 In contrast, Question 4(c)(2) below has high complexity: 4(c) Given: 3 2 2y x x x    (1) Find the equation of the tangent to the curve at the point where x = 1. (2) Find the x-coordinate of another point on the curve at which the tangent is parallel to the one found in (1). In order to solve Question 4(c)(2), the learner has to first solve Question 4(c)(1), a routine procedure of moderate complexity. This yields the equation of the tangent as y = 4x + 5. To complete Question 4(c)(2), the learner needs to first realise that the ‘new’ tangent has a slope of 4 (the slope of the ‘old’ tangent). This requires a conceptual understanding of what it means for one line to be parallel to another. The learner must then set the derivative of the original function equal to 4, i.e. 3x2 2x  1 = 4, a complex procedure requiring non standard thinking. The learner then needs to solve this equation to find x = 1 (already given in part 4(c)(1)) or 3 5x . Thus 3 5x . This example illustrates how the SAGM taxonomy is unable to distinguish between a “complex procedure” with high complexity and a “complex procedure” with moderate complexity. Similarly, using the Stein taxonomy both items would be classified as “procedures with connections”: Question 7(c) illustrates the subtle relationship between notations and the relevant mathematical objects; Question 4(c)(2) focuses on the conceptual connection between the notion of slope and derivative. With regard to Porter, this example illustrates the difficulty of using this taxonomy. Both Question 4(c)(2) and Question 7(c) require the use of non-routine procedures. So do we classify either of these as “perform procedure” or is it a non-routine problem? Arguments may be made for either classification. Application of the NAEP taxonomy (NAGB, 2008) on its own is more reliable as a yardstick for measuring the standard of the examination. However it does not distinguish between different types of mathematical practices. For example, ‘easy’ examination tasks, such as conjecturing a simple pattern, is put in the same category as a familiar and routine procedure, no matter the different nature of the activities. Do the taxonomies provide a useful way of distinguishing between different types of mathematical practices in an examination? Porter’s taxonomy is attractive in that, unlike Stein or SAGM, it has specific categories for key mathematical practices such as communicating, understanding, conjecturing, generalising and solving non-routine problems. However the conflation of type and level of thinking again confounds the categorisation and leads to questions with very different levels of complexity being placed in the same category. We illustrate through two ‘comparable’ items, namely Question 8(b)(3) (IEB, 2008b) and Question 1.3 (DOE, 2008c). See overleaf. Using Porter’s taxonomy, we classify both Question 1.3 and Question 8(b)(3) in the “conjecture, generalise, prove” category. Question 1.3 can be successfully completed by a straightforward application of the distance formula and the converse of Pythagoras’ theorem (low complexity in the NAEP taxonomy). However the route to the proof in Question 8(b)(3) is not straightforward and to construct an entirely coherent and complete proof requires considerable insight (high complexity in the NAEP taxonomy). Porter’s category “conjecture, generalise, prove” thus encompasses both low and high complexity items. Actually, Porter’s taxonomy is designed to measure the degree of alignment between curriculum, instruction and assessment, not to reflect a complexity rating. He uses a two dimensional matrix that combines topics and cognitive demand as a tool to describe the content of an intended curriculum, an assessment and of instructional practice separately. He then uses the matrices to calculate the degree of alignment between any two of these (e.g. between the intended curriculum and the assessment). Thus the Taxonomy m 36 8(b) (1) ( ( (2) (3) Question 1. 1.3 Prove quality of t curriculum disaggregate one could a and series” t matters: Cog (i) Find, corr x-axis. (ii) Find, cor the x-axis Study the ta the form y  Based on th following se The inclinat Prove the co 3 is: e that ADC the assessme and/or instru e the assessm argue that the topic area ne Equation Inclination Equation Inclination gnitive levels rect to two d rrect to two d s. ables below w ax and y he informati entence: tions of the li onjecture tha C = 90o. ent is determ uction matrix ment in a qu e NCS (DOE eeds to be set y = 3x y = 71,60 18, y = 5x y = 78,70 11, and types o decimal digits decimal digi which contai 1 x a  for a ion in the ta ines y ax at you formul mined by the x. This notion uite specific f E, 2003) sugg t at the cogni = 13 x ,40 = 15 x ,30 of mathematic s, the acute a its, the acute in the values  3, 4, 5a  . ables above, and 1 y x a  lated in (2). e closeness o n of alignmen fashion. For gests that a c itive demand Equation Inclination cal activities angle of incl e angle of in of the inclin , formulate x where a  of fit betwee nt means that example, us certain perce d level of “co y = 4x y = 76,00 14,0 ination of the nclination of nations of lin a conjecture 0 en the assess t Porter’s tax sing Porter’s entage of the njecture, gen 1 4 x 00 he line 2y  the line y  nes with equa e by comple sment matrix xonomy is in s notion of al e “patterns, s neralise, prov x to the 1 2 x to ations of eting the x and the ntended to lignment, sequences ve”. Margot Berger, Lynn Bowie & Lovemore Nyaumwe 37 Although the above broad brushstrokes analysis is not how Porter intended his taxonomy to be used, it illustrates why it is important to distinguish types of activity, such as “generalise, conjecture or prove” from level of complexity. The notion of using alignment as the criteria for assessing the fairness and spread of an examination is an appealing one. However the mechanisms by which to measure it and achieve it are anything but obvious. We could apply Porter’s taxonomy. However his taxonomy has been applied chiefly in contexts where some version of the American NCTM (National Council of Teachers of Mathematics) standards is being implemented and NCTM standards have different priorities compared to the NCS vision. Furthermore, as we demonstrated above, his taxonomy is not useful for answering questions about the standard of an examination. A new taxonomy In order to measure both the standard of an examination and the alignment of the examination with the curriculum, we need a tool which distinguishes between key mathematical practices as well as between complexity levels of items. The SAGM taxonomy does not fulfil requirements of such a tool. For example, tasks involving either reasoning or application of mathematics to real-world problems (activities prioritised in the NCS for Mathematics) would both be classified as “solving problems”. Similarly, SAGM has no specific space for key mathematical activities such as justification, conjecturing, and communicating mathematical ideas despite these activities being regarded as important components of the NSC curriculum. Building on the work of Graven (2002) and Parker (2006), Bowie (2009) has argued that four orientations towards mathematics are implicit in the NCS for Mathematics. Bowie makes these orientations explicit as follows: First, Mathematics is important for critical democratic citizenship. It empowers learners to critique mathematical applications in various social, political and economic contexts. Mathematics is part of broader society and is important for all learners. Second, Mathematics is relevant and practical. It has utilitarian value and can be applied to many aspects of everyday life and in other disciplines. Mathematics modelling is an important tool that can be used in the exploration of contexts. Third, Mathematics is created and organised using particular processes. These include investigating, conjecturing, generalising, justifying, proving, axiomatising and defining contexts in which concepts are constructed. Fourth, Mathematics is a body of knowledge and learners at school need to study a subset of it. This knowledge includes specific algorithms, facts, conventions, notations and forms of representation. Bowie (2009) argues that, although the orientation towards mathematics as a body of knowledge predominates, the orientations towards mathematics as relevant and practical and the orientation towards mathematical practices receive particular attention. We suggest that a taxonomy that highlights these aspects is important both because it reflects the implicit NCS and because it is in line with international taxonomies, such as Stein et al. (2009) and Porter (2002). Given the problems associated with using any of SAGM, Stein et al. or Porter, we suggest that, for purposes of setting an examination paper, a taxonomy which distinguishes between low, medium and high complexity on one scale, and categories which derive from the different orientations of mathematics explicated by Bowie, may be more useful. This can be constructed as shown in Table 1. An elaboration of the above taxonomy is beyond the scope and space constraints of this paper and will be the subject of a future paper. Its presence is to signal the possibility of distinguishing complexity (or difficulty level) from the type of mathematical activity in a taxonomy and of simultaneously using this taxonomy to align an examination with the intended curriculum. Taxonomy matters: Cognitive levels and types of mathematical activities 38 Table 1: A new taxonomy Categories Low complexity Moderate complexity High complexity Solving real-world problems; mathematical modelling Mathematical practices, such as conjecturing, generalising, justifying, proving. Mathematical knowledge (facts, procedures, forms of representation. References Bowie, L. (2009, May). Orientations to mathematics in the FET National Curriculum Statement. Paper presented at the Marang PhD Weekend. University of the Witwatersrand, Johannesburg. Concerned Mathematics Educators. (2009). Analysis of 2008 Grade 12 results: Letter to National Department of Education. Retrieved from http://www.mathsexcellence.co.za/letter.php Department of Education. (2003). National curriculum statement Grades 10-12 (General): Mathematics. Pretoria: Government Printer. Department of Education. (2008a). Subject assessment guidelines. Pretoria: Government Printer. Department of Education. (2008b). National Senior Certificate examination, Mathematics Paper I. Pretoria: Government Printer. Department of Education (2008c). National Senior Certificate examination, Mathematics Paper II. Pretoria: Government Printer. Dibetle, M. (2009, May 21). First-year fumble. Mail and Guardian Online. Retrieved from http://www.mg.co.za/article/2009-05-21-firstyear-fumble Graven, M. (2002). Mathematics teacher learning, communities of practice and the centrality of confidence. (Unpublished doctoral dissertation). University of the Witwatersrand, Johannesburg. Hess, K. K. (2006). Exploring cognitive demand in instruction and assessment. National Center for the Improvement of Educational Assessment, Dover NH. Retrieved from http://www.nciea.org/publications/ DOK_ApplyingWebb_KH08.pdf Independent Examination Board. (2008a). National Senior Certificate examination, Mathematics Paper I. Independent Examination Board, South Africa. Independent Examination Board. (2008b). National Senior Certificate examination, Mathematics Paper II. Independent Examination Board, South Africa. Maree, K. (2009). Editorial: Reflecting on best practice in diverse education milieus – modifying the education wheel. Perspectives in Education, 27(1), 1-5. Ministerial Panel. (2009). Report of the ministerial panel established to review the Grade 12 NSC mathematics examination papers of November 2008. Retrieved from http://www.mathsexcellence.co.za/papers/ Report%20of%20panel%20of%20Maths%20experts.pdf National Assessment Governing Board. (2008). Mathematics framework for the 2009 National Assessment of Educational Progress. Washington DC: US Department of Education. Available from http://www.nagb.org/publications/frameworks/math-framework09.pdf Porter, A. C. (2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3-14. doi: 10.3102/0013189X031007003 Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2000). Implementing standards-based mathematics instruction: A casebook for professional development. Reston, VA: National Council of Teachers of Mathematics. Stein, M. K., Smith, M. S., Henningsen, M. A., & Silver, E. A. (2009). Implementing standardsbased mathematics instruction, A casebook for professional development (2nd ed.). Reston, VA: National Council of Teachers of Mathematics. Umalusi. (2009). 2008 maintaining standards report. Part 1: Overview. Pretoria: Umalusi. Available from http://www.umalusi.org.za/ur/Research/UmalusiStandards09HighRes.pdf Margot Berger, Lynn Bowie & Lovemore Nyaumwe 39 Appendix: Summary of the different taxonomies SAGM Taxonomy Knowledge  Knowledge and use of formulae or algorithms. Routine procedure  Problems are not necessarily unfamiliar and can involve integration of different Learning Outcomes.  Well-known procedure.  Simple applications and calculation with many steps and which may require interpretation from given information.  Identifying and manipulating formulae. Complex procedures  Mainly unfamiliar and involve integration of different Learning Outcomes.  No direct route to solution but involve higher level calculation skills and/or reasoning.  May be abstract and require fairly complex procedures. Solving problems  Non-routine, unseen.  Interpreting and extrapolating from solutions obtained by solving problems based in unfamiliar contexts.  Using higher level cognitive skills and reasoning to solve non-routine problems  Breaking down problem into constituent parts.  Non-routine in real context. Porter Taxonomy Memorise facts/definitions/formulae  Recite basic math facts.  Recall math terms and definitions or formulae and computational procedures. Perform procedures  Use numbers to count, order, denote.  Computational procedures or algorithms.  Follow procedures/instructions.  Solve equations/formulae/ routine word problems.  Organise or display data.  Read or produce graphs or tables.  Execute geometric constructions. Demonstrate understanding of mathematical ideas  Communicate math ideas.  Use representations to model math ideas  Explain finding and results from data analysis strategies.  Develop relationships between concepts.  Show or explain relationships between models, diagrams or other representations. Solve non-routine problems/make connections  Apply and adapt a variety of appropriate strategies to solve non-routine problems.  Apply mathematics in contexts outside of maths.  Analyze data, recognise patterns.  Synthesise content and ideas from several sources. Conjecture/generalise/prove  Determine the truth of a math pattern or proposition.  Write formal or informal proofs.  Recognise, generate or create patterns.  Find a mathematical rule to generate a pattern or number sequence.  Make and investigate math conjectures. Taxonomy matters: Cognitive levels and types of mathematical activities 40  Identify faulty arguments.  Reason inductively or deductively. Stein Taxonomy Memorisation  Reproduction of previously learnt fact, formulae, rules or definitions.  Cannot be solved using a procedure.  Are not ambiguous  exact reproduction of previously seen material.  Have no connection to concepts or meanings that underlie the fact, rules, formulae or definitions. Procedures without connections  Algorithmic.  Little ambiguity about what needs to be done.  Have no connections to the concepts or meaning that underlie the procedure.  Focused on correct answer rather than understanding.  Require no explanations (or explanation solely on procedure). Procedures with connections  Focus students’ attention on use of procedures for purpose of developing deeper levels of understanding.  Suggest pathways to follow that are broad general procedures that have close connections to underlying conceptual ideas.  Make connections among multiple representations.  General procedures cannot be followed mindlessly. Students need to engage with the conceptual ideas that underlie the procedures. Doing mathematics  Require complex and non-algorithmic thinking  Require students to explore and understand nature of mathematical concepts, processes or relationships.  Demand self-monitoring or self-regulation of one’s own cognitive processes.  Require students to access and use relevant knowledge and experiences.  Require students to analyse the task and actively examine task constraints that may limit possible solution strategies and solutions.  Require considerable cognitive effort and may involve some level of anxiety due to unpredictable nature of solution processes. NAEP Taxonomy Low complexity  Students to recall or recognise concepts or procedures.  Items typically specify what the student is to do, which is often to carry out some procedure that can be performed mechanically.  The student does not need to use an original method or to demonstrate a line of reasoning. Moderate complexity  Items involve more flexibility of thinking and choice among alternatives than do those in the low- complexity category.  Student is expected to decide what to do and how to do it, bringing together concepts and processes from various domains. For example, student may need to represent a situation in more than one way, to draw a geometric figure that satisfies multiple conditions, or to solve a problem involving multiple unspecified operations.  Students might be asked to show explain their work but would not be expected to justify it mathematically. High complexity  Students are expected to use reasoning, planning, analysis, judgment, and creative thought.  Students may be expected to justify mathematical statements or construct a mathematical argument. Items might require students to generalise from specific examples.  Items at this level take more time than those at other levels due to the demands of the task, not due to the number of parts or steps. << /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Dot Gain 20%) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Error /CompatibilityLevel 1.4 /CompressObjects /Tags /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages true /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.0000 /ColorConversionStrategy /CMYK /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams false /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness true /PreserveHalftoneInfo false /PreserveOPIComments true /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Preserve /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages true /ColorImageMinResolution 300 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Bicubic /ColorImageResolution 300 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50000 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.15 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False /CreateJDFFile false /Description << /ARA /BGR /CHS /CHT /CZE /DAN /DEU /ESP /ETI /FRA /GRE /HEB /HRV (Za stvaranje Adobe PDF dokumenata najpogodnijih za visokokvalitetni ispis prije tiskanja koristite ove postavke. Stvoreni PDF dokumenti mogu se otvoriti Acrobat i Adobe Reader 5.0 i kasnijim verzijama.) /HUN /ITA /JPN /KOR /LTH /LVI /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken die zijn geoptimaliseerd voor prepress-afdrukken van hoge kwaliteit. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.) /NOR /POL /PTB /RUM /RUS /SKY /SLV /SUO /SVE /TUR /UKR /ENU (Use these settings to create Adobe PDF documents best suited for high-quality prepress printing. Created PDF documents can be opened with Acrobat and Adobe Reader 5.0 and later.) >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /ConvertColors /ConvertToCMYK /DestinationProfileName () /DestinationProfileSelector /DocumentCMYK /Downsample16BitImages true /FlattenerPreset << /PresetSelector /MediumResolution >> /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ] >> setdistillerparams << /HWResolution [2400 2400] /PageSize [612.000 792.000] >> setpagedevice