*Corresponding author – khkoh@ucalgary.ca Koh, K., & Chapman, O. (2019). Problem-based learning, assessment literacy, mathematics knowledge, and competencies in teacher education. Papers on Postsecondary Learning and Teaching: Proceedings of the University of Calgary Conference on Postsecondary Learning and Teaching, 3, 74-80. PROBLEM-BASED LEARNING, ASSESSMENT LITERACY, MATHEMATICS KNOWLEDGE, AND COMPETENCIES IN TEACHER EDUCATION Kim Koh* & Olive Chapman University of Calgary Problem-based learning could have a great impact in teacher education not only to support prospective teachers’ learning, but also to help them to design and implement learner-centered experiences to satisfy requirements of reform-based curriculum. In this paper, we discuss the nature and role of problem-based learning to support authentic learning opportunities in an undergraduate teacher education program. We address its use in an educational assessment course aimed at developing prospective teachers’ assessment literacy and competencies. We focus on two sections of the course for elementary school prospective teachers in which students were also engaged in activities involving assessment in teaching mathematics and share examples of the content of the course. A study of the impact of the course on the students’ mathematics knowledge is in progress. Keywords: Problem-based learning; prospective elementary school teachers; assessment literacy; assessment in mathematics Problem-Based Learning [PBL] could have a great impact in teacher education not only to support prospective teachers’ learning, but also to help them to design and implement learner- centered experiences to satisfy requirements of reform-based curriculum. In this paper, we discuss the characteristics and importance of PBL to support authentic learning opportunities at different levels of education and its use in undergraduate teacher education. In particular, we address its use in an undergraduate education assessment course at the University of Calgary aimed at developing prospective teachers’ assessment literacy and professional competencies. We focus on two sections of this course for prospective elementary school teachers. These sections included activities involving formative assessment in teaching mathematics. We offer examples of the PBL tasks used in the course. Our goal is to highlight the use of PBL to support prospective teachers’ development of assessment literacy based on theory and our experiences. CHARACTERISTICS OF PROBLEM-BASED LEARNING Barrows (1996, 2002), a leading authority on PBL pedagogy, identified the following key characteristics of a quality PBL experience: • Learning is student-centered. Students take responsibility for their own learning, determine what it is they need to learn and find the appropriate resources for the information from the world about them. Koh & Chapman (2019) 75 • Learning occurs in small student groups. Students work collaboratively to pool ideas and resources to address the problem. • Teachers are facilitators or guides. The teacher directs students to resources with which to solve the problem and provide support during their group work. • Problems form the organizing focus and stimulus for learning. The point of PBL is for students to discuss problems that will likely occur in a real professional setting. • Problems are a vehicle for the development of problem-solving skills. They are ill- structured, domain specific, and enable independent inquiry. They should be complex enough to challenge students to seek out new methods and techniques to solve them and apply skills and knowledge from different areas. • New information is acquired through self-directed learning. Students should develop new knowledge and expertise through their own experiences while solving problems. These characteristics continue to define PBL as one of the signature pedagogies (Shulman, 2005) being promoted in postsecondary teacher education. IMPORTANCE OF PROBLEM-BASED LEARNING PBL has been promoted as an effective pedagogical approach to promote undergraduate students’ authentic learning and development of disciplinary knowledge and professional competence (e.g., critical thinking, creativity and innovation, complex problem solving, self- directed learning, collaboration, and communication). The rapidly changing, global workplace has led to a high demand for graduates who possess not only excellent disciplinary knowledge, but also additional competencies that reflect their ability to think out of the box and to work independently as well as with others in solving complex problems. Hence, developing students’ competencies across the curriculum has become a priority in many higher education institutions (Anderson, 2017) and instructors are urged to infuse competence-based learning, such as PBL, into the courses they teach and to adopt innovative pedagogical approaches that provide university students with authentic learning experiences to develop and master competencies. This is also the case for teaching students in Kindergarten to Grade 12. PBL engages students in the learning process by requiring them to think critically, conduct research, and provide valid solutions to complex, real-world problems or authentic scenarios (Hmelo-Silver, 2004). In their systematic literature review to explore the effectiveness of PBL implemented with students in early elementary to grade 8 (ages 3–14) in mathematics and science classrooms, Merritt, Lee, Rillero and Kinach (2017) found that PBL was an effective method for improving these students’ academic achievement. Reform movements in education are also focusing more on K-12 students’ use of critical thinking in learner-centered, constructivist activities consistent with a PBL perspective. For example, Alberta Education (2017) promotes preparing students to deal with and contribute to a complex world by engaging them in meaningful learning experiences to develop 21st century competencies. Its standards for school curriculum include engaging students in learning experiences that broaden perspectives and enable students to create opportunities, challenge the status quo, take initiative to achieve their dreams, and take action to create a better world. Through the study of subjects that … develop competencies through learning outcomes, Koh & Chapman (2019) 76 students use their abilities to communicate respectfully, synthesize ideas, collaborate with others, think critically and solve complex problems. (p. 12) Such curriculum expectations place significant responsibility on teacher education programs to prepare teachers to teach in ways that may be different from how they were taught. Teacher preparation programs, then, have an essential role not only in helping prospective teachers learn how to design curriculum and assessment that align well with the 21st century standards, but also in developing their professional competencies. PBL can offer a way to support this outcome. PROBLEM-BASED LEARNING IN TEACHER EDUCATION In addressing PBL in teacher education, De Simone (2008) stated, “Current educational reform movements emphasize preparing teachers for pedagogical problem solving in the classroom” (p. 179). The American Association of Colleges for Teacher Education and the Partnership for 21st Century Skills (2010) also acknowledged that “New teacher candidates must be equipped with 21st century knowledge and skills and learn how to integrate them into their classroom practice for our nation to realize its goal of successfully meeting the challenges of this century” (p. 3). Thus, the inclusion of 21st century knowledge and skills formally into teacher preparation programs is of paramount importance. Teacher education programs must require students to practice innovative instructional strategies that elicit critical thinking and problem solving. In particular, if we intend our student teachers to be assessment literate – be competent in the design and use of innovative assessment methods to enable K-12 students to master disciplinary knowledge and to develop professional competencies, instructors need to model exemplary pedagogical and assessment practices to prospective teachers. Given the characteristics of PBL, the use of authentic assessment tasks in PBL could be essential to promote prospective teachers’ learning and development of professional competencies. Recent studies have been investigating PBL in teacher education. For example, Koh and Tan (2016) discussed promoting reflection in prospective teachers through PBL; Nariman and Chrispeels (2016) investigated elementary teachers perception of challenges and benefits in using PBL; Drew and Mardis (2008) investigated a PBL unit offered in a web-blended course for prospective teachers; and Rillero, Koerner, Jimenez-Silva, Merritt, and Farr (2017) studied the infusion of PBL into their teacher education programs to provide their student teachers with multiple opportunities to experience it as learners and to design and implement PBL experiences. Hemker, Prescher, and Narciss (2017) is one current study that looked at PBL and assessment. They implemented a PBL approach in four seminars on educational assessment. The results showed benefits of the PBL approach, for example, students welcomed the work with realistic, practical problems, but there was also room for improvement. However, there has been an under-representation in published research on what kinds of pedagogical approaches are well suited to support prospective teachers’ learning about assessment (DeLuca, Chavez, Bellara, & Cao, 2013). But DeLuca et al. also noted that there are two general forms of pedagogy in preservice assessment education: content-based or teacher-directed teaching and process-based or student-centered teaching that includes elements of PBL. Greenberg and Walsh’s (2012) found that most of the 180 teacher preparation programs they reviewed did not provide adequate assessment education and used a content-based teaching approach. Thus, more attention is needed to address the use of PBL in assessment education for prospective teachers. Our work in progress, discussed next, contributes to this in exploring how PBL can be Koh & Chapman (2019) 77 used to maximize authentic learning opportunities for undergraduate student teachers to develop their assessment literacy, mathematical knowledge, and professional competencies. PROBLEM-BASED LEARNING IN A PROSPECTIVE TEACHERS’ ASSESSMENT COURSE The required assessment course in the Bachelor of Education program at the University of Calgary was designed by the first author with a focus on PBL pedagogy. Beginning in 2017, the course is offered in the semester-2 of the 2-year, 4-semester program before students engage in any practicum teaching. The course outline describes the learning outcomes as follows: Students will be knowledgeable about: the definitions, purposes, functions, and principles of different forms of assessment; the design principles and features of authentic performance assessments; the principles and features of high quality rubrics; the alignment between high quality assessment tasks, rubrics, and assessment for learning; and the rationale for adopting sound grading and reporting practices (Koh, 2017, p. 1). In the two of the elementary school preservice teachers’ sections of the course taught by the authors (one each) in Winter 2018, in addition to these outcomes, we integrated mathematical activities as part of a project supported by a University of Calgary Teaching and Learning Grant. This project is exploring how the course and PBL supported prospective elementary school teachers, with little or no access to mathematics education courses in their program, in their learning and development of assessment literacy, mathematics knowledge for teaching, and related competencies. We share examples of the assessment and mathematics PBL problems or tasks to illustrate how the course engaged the students. Assessment Literacy The following five problems that form the basis for the course are organized around real-world issues in assessment to support students’ development of assessment literacy: (1) developing an assessment tool box: considering balance and purpose; (2) assessment for learning; (3) developing high quality assessment tasks; (4) developing high quality rubrics to enhance student learning; and (5) grading and reporting. Each was connected to a learning task based on an authentic scenario. For example, the following scenario was connected to problem (3): As a new teacher, you are asked by your principal to join the assessment task force in your school. In view of the provincial initiatives in curriculum redesign and Student Learning Assessments, the school is heading toward a whole-school implementation of performance assessment and assessment for learning. One of the common practices in your school is that teachers tend to search for ready-made assessment tasks and rubrics on the Internet and use them blindly in assessing student performance. Such a practice may lead to unintended negative consequences on learners’ experiences if the assessment tasks and rubrics are not suitable for the intended learning outcomes and the local school culture. Hence, members of the assessment task force are responsible to develop an online repository of exemplary performance assessments and associated rubrics in the school. Given that you are the only member who has mastered the most up-to-date Koh & Chapman (2019) 78 knowledge of classroom assessment in your preservice teacher education program, you have been asked by the chair of your committee to work with other committee members to provide a concrete example of an exemplary performance assessment and its associated rubric(s) in your [specialization] subject area. Working in groups of 4 or 5, students researched, reviewed, critiqued, redesigned or designed a performance assessment and its associated rubric(s). They were provided with required and suggested readings that included criteria for authentic intellectual quality (Koh, 2011a; Koh, 2011b). In addition to learning about performance assessment tasks and rubrics, the requirements of this PBL problem allowed students to engage in self-directed learning and other 21st century competencies (e.g., critical thinking, problem-solving, collaboration, and communication). Mathematics Knowledge for Teaching Mathematics knowledge for teaching, based on the work of Ball, Thames, and Phelps (2008), includes: (1) specialized content knowledge, for example, knowing alternative meanings/approaches/interpretations of a mathematics concept or procedure; (2) knowledge of content and teaching, for example, knowing instructional advantages of different representations; what mathematical representations to use with students and which of those representations are likely to be understood and misunderstood by students; (3) knowledge of content and students, for example, knowing the ways students understand the content; students’ mathematical thinking and alternative approaches. Mathematical activities integrated into the two sections of the course engaged the prospective teachers in each of these through exploring formative assessment tasks or cases. For some tasks, students addressed questions such as: What do I want my students to learn within this topic/lesson/unit? What can my students currently understand and do (perform)? What do I want my students to understand and be able to do based the Big Ideas and specific outcomes in the mathematics programs of study? For other tasks, they explored students’ thinking as in the following example related to knowledge of content and student. You notice several students doing the following during a lesson on adding whole numbers. (a) What do you do? (b) For each, explain what you think the students are doing? (c) Why do you think they are doing what they are doing? a. 47 b. 16 c. 56 d. 35 + 86 + 48 + 78 + 46 123 91 1214 171 Students worked in groups of 4 or 5 to investigate these students’ ways of thinking, their own understanding of this concept, and how they would deal with this situation from a formative Koh & Chapman (2019) 79 assessment perspective. Learning opportunities provided by this PBL task included learning an aspect of mathematics knowledge for teaching and engaging in competencies such as problem- solving, collaboration, communication, self-reflection, and noticing (students’ thinking). We are in the process of studying the course and at this point do not have final results of its impact on students’ learning. However, preliminary results point in the direction of the PBL activities supporting positive shifts in their assessment literacy, knowledge of formative assessment in teaching mathematics and learner-centered competencies. Our observations of students working on the problems or tasks in class suggest that PBL is useful in creating a supportive learning environment for preservice teachers to develop essential 21st century competencies and dispositions relevant to the teaching profession. REFERENCES Alberta Education. (2017). The Guiding Framework for the Design and Development of Kindergarten to Grade 12 Provincial Curriculum (Programs of Study). Retrieved from https://education.alberta.ca/ media/3575996/curriculum-development-guiding- framework.pdf. American Association of Colleges of Teacher Education and the Partnership for 21st Century Skills. (2010). 21st century knowledge and skills in educator preparation. Retrieved from http://www.p21.org/storage/documents/aacte_p21_whitepaper2010.pdf. Anderson, L. (2017). Competency-based education: Recent policy trends. Competency-based education, 3(1), 1-15. Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389–407. Barrows, H. S. (1996). Problem-based learning in medicine and beyond: a brief overview. New Directions for Teaching and Learning, 68, 13-21. Barrows, H. S. (2002). Is it truly possible to have such a thing as dPBL? Distance Education, 23(1), 119-122. DeLuca, C., Chavez, T., Bellara, A., & Cao, C. (2013). Pedagogies for preservice assessment education: Supporting teacher candidates’ assessment literacy development. The Teacher Educator, 48(2), 128-142. De Simone, C. (2008). Problem-based learning: A framework for prospective teachers’ pedagogical problem solving. Teacher Development, 12(3), 179-191. Drew, M. & Mardis, L. (2008). Building pre-service teachers' awareness of multiple perspectives through problem-based learning. International Journal of Learning, 15(5), 167-187. Greenberg, J., & Walsh, K. (2012). What teacher preparation programs teach about K–12 assessment: A review. New York City, NY: National Council on Teacher Quality. Hemker, L., Prescher, C., & Narciss, S. (2017). Design and Evaluation of a Problem-Based Learning Environment for Teacher Training. Interdisciplinary Journal of Problem-Based Learning, 11(2). Available at: https://doi.org/10.7771/1541-5015.1676. Hmelo-Silver, C. E. (2004). Problem-Based Learning: What and how do students learn? Educational Psychology Review, 16, 235-266. Koh, K. (2011a). Improving teachers’ assessment literacy. Singapore: Pearson. Koh, K. (2011b). Improving teachers' assessment literacy through professional development. Teaching Education, 22(3), 255-276. Koh, K. (2017). EDUC 456 Assessment Course Outline. Calgary, AB: University of Calgary. Koh & Chapman (2019) 80 Koh, K., & Tan, C. (2016). Promoting reflection in pre-service teachers through problem-based learning: An example from Canada. Reflective Practice, 17(3), 347-356. Merritt, J., Lee, M., Rillero, P., & Kinach, B. M. (2017). Problem-Based Learning in K–8 Mathematics and Science Education: A Literature Review. Interdisciplinary Journal of Problem-Based Learning, 11(2). Available at: https://doi.org/10.7771/1541-5015.1674. Nariman, N., & Chrispeels, J. (2016). PBL in the era of reform standards: Challenges and benefits perceived by teachers in one elementary school. Interdisciplinary Journal of Problem-Based Learning, 10(1). https://doi.org /10.7771/1541-5015.1521. Rillero, P., Koerner, M., Jimenez-Silva, M., Merritt, J., & Farr, W. J. (2017). Developing Teacher Competencies for Problem-Based Learning Pedagogy and for Supporting Learning in Language-Minority Students. Interdisciplinary Journal of Problem-Based Learning, 11(2). Available at: https://doi.org/10.7771/1541-5015.1675. Shulman, L. S. (2005). Signature pedagogies in the professions. Daedalus, 134(3), 52−59.