Microsoft Word - v9n2sullenberger-8019.doc Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009, pp. 1 - 9. Enhancing the success of SOTL research: A case study using modified problem-based learning in social work education Sabrina Williamson 1 and Valerie Chang 2 Abstract: This article describes a study which utilizes Modified Problem-Based Learning (MPBL) as a teaching method in undergraduate social work practice classes. The authors report both qualitative and quantitative findings of the research. Additionally, the authors reflect on the use of the MPBL method and on the lessons learned throughout this research on the scholarship of teaching and learning. Keywords: problem-based learning, transfer of learning, social work practice I. Introduction. A primary objective of postsecondary education is to prepare students for their professional careers. As such, when students declare a major and begin to pursue studies in a particular area, they begin a trajectory of movement from thinking like a student to thinking like a professional in their chosen discipline. In any given field, thinking like a professional involves discerning meaning from information that is presented to them, organizing knowledge around major principles, responding to changes in context, and accessing and retrieving knowledge smoothly (Thompson, Licklider, and Jungst, 2003). In schools of social work, as with other professional schools, faculty are responsible for developing and delivering curriculum that aids in the student’s transition from novice to professional (Koerin, Harrigan, and Reeves, 1990). In social work educators must identify and teach the step-by-step thinking process used by professional social workers and also be sure that students know how to appropriately apply knowledge and use professional skills as they work with clients. However, despite best efforts students often experience disconnects in their learning when they move from the classroom to professional practice (Lager and Robbins, 2004). Classroom knowledge often is not transferred to the real world experience. Consequently, students struggle in their internships as they either don’t use what they learned in class or apply very little of what they learned. In this article we will describe a study in which modified problem-based learning (MPBL) was used in two undergraduate social work practice courses to teach students the process of thinking like professional social workers. We will also discuss the “lessons learned” by the authors during the research process. During phases of data collection and analysis, the authors learned invaluable lessons about teaching and learning and about research in the field of scholarship of teaching and learning. We hope our reflections on this project will help other faculty as they develop their courses and consider engaging in SOTL projects. The authors wish to thank Indiana University’s Mack Center for Inquiry on Teaching and Learning for their 1 School of Social Work, Indiana University Bloomington, 1127 Atwater, Bloomington, IN 47401, sabawill@indiana.edu 2 School of Social Work, Indiana University Purdue University Indianapolis, 902 West New York Street, Indianapolis, IN, vchang@iupui.edu Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 2 support in this project, both in funding the authors as faculty fellows and in providing guidance and direction throughout the study. II. Literature review. Problem-based learning is a learner-centered approach in which learners “conduct research, integrate theory and practice and apply knowledge and skills to develop a viable solution to a defined problem” (Savery, 2006, p.12). In social work, the “problem” that students receive is a client scenario that is representative of a real-life situation that an individual, family, group or community might face. Students receive parts of the case a section at a time, replicating the process of working with a case in the field. Additionally, PBL is pedagogy based on the premise that acquiring process skills is as important as assimilating content (Margetson, 1991.) Using problem-based learning, students are presented with scenarios faced by social workers in the field and are challenged to find information, apply previously acquired knowledge and work with their colleagues (other students) in planning for assessment and developing interventions. As students work on the problem-based learning cases, faculty can help them learn the process of “thinking like a professional” (Middendorf and Pace, 2005). PBL is similar to case-based learning with the primary difference between the two being the manner in which information relevant to the case or problem is presented to the students. Typically, instructors utilizing case-based learning techniques present an entire case study to student groups rather than in the discrete parts utilized in problem-based learning. Course instructors help students analyze the problem, consider solutions, and determine the actions that a professional would take in the situation (Herreid, 1994). In addition to its use in social work education, problem-based learning has also been utilized extensively in other professional schools, including business (c.f. Saatci, 2008), medicine (c.f. Spencer, 2003), and education (c.f. Edwards and Hammer, 2005). In social work, Lam (2004) reports on a BSW program that utilizes PBL in four courses throughout the curriculum. This study measured students’ change in competence, values, and clinical skills before and after PBL instruction and compared these scores with pre-test and post-test scores of students in prior years who had completed the traditional (non-PBL) curriculum. Lam found that students who received PBL instruction were more efficient in searching out information and in taking responsibility for their learning. Other program faculty, however, rated the performance of PBL students as weaker than non-PBL students in written examinations concerning theories and practice skills. Similar to Lam’s finding, Chan and Ng (2004) reported that graduates from PBL training module were found to be “more self directed and creative than their predecessors” (p.316). Altshuler and Bosch (2003) used PBL in school social work policy and school social work practice classes to prepare students for work in various school settings. At the end of the PBL course work, they sought student feedback concerning perception of content and skills learned. Overall, the students thought that the PBL approach was conducive to learning. Gelman and Mirabito (2005) discuss using PBL techniques with MSW students specifically to teach crisis intervention. While not an empirical paper, the authors present the vignettes used to teach students and provide teaching points about how instructors can utilize this method in the classroom. While other authors have written about modifying problem-based learning techniques (c.f. Goodnough, 2005 and Baldwin, Bankston, Anderson, Echtenkamp, Haak, Smith and Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 3 Iatridis, 2002), there does not seem to be uniformity in what modifications are made. A method for using Modified Problem-Based Learning (MPBL) in social work education was developed by Chang and is described in Chang, Scott, Decker (2009). Like PBL, students using MPBL receive a section of a case at a time. This replicates what happens in most disciplines where the student has basic introductory information initially and gradually learns more. Also like PBL students work in groups in order to share perceptions and knowledge. Unlike PBL the students lead their own groups. With traditional PBL trained teaching assistants work with each student group. The teaching assistants are trained to ask the kind of questions that lead students to explore more deeply and to think like professionals. Since we do not have the resources for trained teaching assistants, we modified PBL first by teaching students to be leaders in their groups and second by following each section of a case with the kind of questions that professionals use to guide their thinking. The instructors provided consultation and guidance to the student groups as they used their life experience, knowledge from this course and previous courses, and new material found by doing research to fill information gaps to discuss and answer the case questions. III. Description of study. This study was conducted on two campuses of a Midwestern school of social work with undergraduate students. One campus is in a large urban area where students are predominantly commuters. The other campus is in a traditional college town setting where the majority of students live on or adjacent to campus. MPBL was simultaneously used in two undergraduate practice classes, one on each of the campuses. Each course has the same course objectives and uses the same textbooks. Students take this course in the fall semester of their junior year as their first introduction to practice skills prior to any practice experiences. The research team included the two social work faculty members who taught the practice course at the two research sites, a postdoctoral fellow who has taught social work courses, and a graduate student/research assistant. All of the data was collected by the faculty members. The entire team participated in the analysis of the data. Human subjects approval was obtained from the university Institutional Review Board prior to the start of data collection. A. Instructional methods. In each course, instructors spent the first four to five weeks of the semester reviewing and introducing theoretical foundations of social work practice. After completing this section of the course, the instructors introduced the concept of modified problem-based learning. Students were told that they would work in a small group. Each group would have a case that they would work with for the remainder of the fall semester. Instructors of the classes indicated that they would serve as consultants to the groups, but that the students were responsible for responding to the questions about the case. In the first week of working on the cases, each student group was given the first section of a case and questions related to that section. The fourteen questions on Part One of the case were designed to help students prepare to work with a client by identifying facts and knowledge needed and how they planned to work with and collaborate with their clients. Questions related to the first section of the case included: Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 4 What are the key facts in the case? What additional information will you need prior to meeting this client? What concerns do you have about working with this client? What are your preliminary impressions related to the case? (Chang, Scott, Decker, 2009, p. 79) Following each additional section of the case, students received additional questions to guide their thinking about the case. Questions asked about such things as changes in initial impressions, additional information needed, problems and goals identified, treatment plan developed, level of motivation, appropriate action, and eventually ending the work with the client. Student groups were given in class time to discuss their case, to share their perceptions and information gained. Each group received a different case and turned in a written report answering each question. After they turned in their work related to Part One, they received Part Two of the same case. Again, each group was expected to collaborate and to turn in a written document that addressed the questions related to the second part of the case. Each group received written feedback from the instructor concerning their responses to the questions. In one class toward the end of the semester, students were given time to share information about their cases and their responses to the questions about the cases. Each group engaged in a dialogue with the instructor and other students about their thinking related to their case. In the other class students reported on and discussed their case on the day they turned in each case report. In this class there were several discussions about the cases during the semester. The MPBL work was done in conjunction with more traditional course work of lecture, discussion, and role-plays. Parts One and Two of the cases were distributed in the fall semester in the first practice class. Students received the remaining five parts of their respective cases in their next practice course. B. Sample. The sample consists of thirty-five students. This represents 90% of the combined population of the two practice classes. Ninety-seven percent of the sample was female, 68% were traditional college aged (20 to 24) and 37% had worked in social service type jobs during the previous year. In terms of ethnicity, the sample consisted of 83% who identified as Caucasian, 9% who identified as Hispanic, 6% who identified as African-American, and 3% who identified as other. C. Data collection and analysis. During the first practice course, each student in the research project completed individual pre- tests and post- tests related to a case that had not been assigned to any of the groups. In the pre- test, students answered questions related to Parts One and Two of the case. This pre- test was conducted at the end of the first course module; i.e. after theoretical foundations of practice had been reviewed but before practice skills were introduced and before students began working in their MPBL groups. The post-test was identical to the pre-test and was administered at the end of the semester after students had been working in modified problem-based groups and after they had been exposed to new information and skills in the course. Pre-test and post-test data were analyzed both quantitatively and qualitatively. Quantitatively, a rubric was developed and used to score each student’s answers on the pre-test and post- test. The rubric, which consisted of answers to the case questions that are consistent with best practices in the social work profession, enabled researchers to evaluate the information Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 5 on each test. Using SPSS, T tests were then run to determine if there were significant differences between each student’s pre-test and post- test. Additionally, each student’s answers to the pre- and post-test were entered into a database and then analyzed in ATLAS-ti. Content analysis (Stemler, 2001) was used to determine qualitative differences in the students’ answers that might indicate improvement in the path of “thinking like a professional”. IV. Findings. Quantitatively, the findings were non-significant. Of course with such a small sample it would be very difficult to have significant findings. Qualitatively, there is indication that many students showed improvement from pre-test to post-test. Student comments on the experience of using modified problem-based learning illustrate many of these areas of improvement. Students appreciated the fact that the clients presented in the cases were real, not “made up”, and that the problems faced by the social worker were ones similar to what they would face in their future practice. Students’ reports included comments such as: “it was good to apply what we were learning to real people,” “it allowed me to get a real picture of social work,” “it was the next best thing to working in the field.” Students also benefited from working with their peers. They said, “it was useful to learn to collaborate in a group and research relevant material” and “Being able to discuss situations with other people was great and it helped me prepare to meet with others to professionally and respectfully discuss issues.” Students also recognized that the use of cases assisted them in thinking critically about client situations. Student comments to support this theme included, “this process taught me to look at a problem from all angles”, “using the cases helped me to think of possible ethical dilemmas and value conflicts” and “this taught me the importance of researching more than the obvious information.” Finally, students perceived that the use of cases helped them be more effective and confident in their field placements. They said that “using cases helped me to have the opportunity to experience case management before entering my practicum” and “it helped me become more efficient in my field placement” and “using cases helped me to realize the magnitude of my work, that everything isn’t easy and that there will always be bumps in the road”. In summary, doing MPBL enhanced students’ confidence in their readiness to work with clients, increased their ability to think like professionals, to use professional vocabulary, to understand the need for assessment throughout the life of the case, and to realize the on-going need for more knowledge related to client issues, background and culture, and necessary about resources. While this is gratifying to see, without a control classroom the question remains as to whether these students showed this type of improvement because of MPBL or if they would have improved in similar ways in a traditional practice course. V. Lessons learned about teaching and learning. Reflecting on this project, we learned ways to improve our teaching. First, it is important to do a pre-test at the beginning of the semester to assess students’ knowledge related to all of course objectives as well as a pre-test related to a case. The pre-test used in this project did not cover all the course objectives. A better pre-test would invite students to evaluate their knowledge and skills related to all the course objectives. If on the pre-test a student identifies that they have knowledge related to a course objective, a follow-up question should ask them to identify how the knowledge was acquired, e. g., by reading, in another course, or on the job. Reviewing the Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 6 pre-test allows the instructor to tailor the course to the needs of the students. After identifying the strengths that the students bring into the course, the instructor can develop ways to build on these strengths. For example, in the pre-test a number of students identified the importance of building a relationship and establishing rapport in the pre-test. If the instructor had reviewed the pre-test, s/he would have noticed this strength and been able to enhance and build on it. In the post-test some of these students focused on exploring what was wrong with the client and didn’t mention building a relationship with the client. The pre-test also gives information about the gaps in the students’ knowledge. Knowing the students’ areas of weakness from the beginning of the course allows instructors to tailor their teaching and assignments to help students’ master key course objectives. Using MPBL, instructors can design questions that invite students to explore, discuss and learn information and skills important to their professional growth. Another lesson is the importance of allowing time for instructor follow through. If the students had been given the post-test a few weeks before the end of the semester, rather than on the last day of class, instructors could have reviewed the post-test answers and helped the students correct any misunderstandings. For example, some students wrote responses that focused on an aspect of the case that was accurate but less important than other aspects of the case. After realizing this problem, the instructor could have helped the students think through the case again and learn to identify the most important aspects of the case. Since applying concepts to social work practice requires higher level learning, students need additional opportunities to practice. Therefore, follow through should continue in subsequent practice classes as well as in the field practicum discussion seminar. To enhance student learning, the field practicum seminar should emphasize the concepts learned in the previous course and require students to use the same MPBL case questions as they work with actual clients. The field seminar assignments should require students to demonstrate using the same thinking skills developed by using MBPL in the previous practice classes. A third important lesson is the importance of establishing measureable, achievable course objectives. For this course the objectives were established by a committee that was focusing on accreditation standards rather than what was possible. As educators interested in scholarly teaching we need to continue to take a stand on the establishment of measureable, achievable objectives; otherwise instructors often try to put more content into each class than students can retain and appropriately use in practice. In this course the pressure to cover course objectives that include too much led the instructors to use lecture, role-plays, and MPBL. The instructors were caught between their belief in the value of active collaborative learning and their requirement to cover a great deal of content. This leads to another important lesson. Adding a new approach or method of teaching can be positive, but in the case of MPBL it should have been the central focus and main teaching and learning method. Adding new approaches takes time. To enhance the value of using MPBL, more class time needs to be allowed for the student groups to discuss, process, work on the cases. Each student should have written out all the case answers before the group discussion of the case answers. Other course assignments should be structured to emphasize or use the work with the cases. Each student group should have regular opportunities to discuss their thinking about each case with the whole class. We know that classroom assessment of learning is very important (Angelo and Cross, 1993). MPBL is an excellent way to assess learning. Since during the semester, the student groups were able to appropriately answer the case questions, the instructors believed the students Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 7 had learned to appropriately apply content to cases. However, at the end of the semester, some individual students were not able to replicate best answers to the same questions related to a different case. The students reported that working with the cases helped them feel more confident as they approached working with clients in field placement. If the instructors had spent more time on MPBL, all the students might have not only felt more confident but individually been able to demonstrate greater ability to critically think through case material. The lesson for us is that multiple ways of assessing learning should be used. Specifically we should have had individual students answer the case questions. The instructor who could identify thinking problems should evaluate these individual answers. The group could work with all the individual answers as well as do additional research to develop more sophisticated, professional responses to each question. VI. Lessons learned about SOTL research. Besides lessons about teaching, we also learned some important lessons about research on teaching and learning. Before starting this research we studied the use of case-based and PBL with graduate students and were excited about the value of using MPBL with undergraduate students. The best course to introduce MBPL was a practice course that is offered only in the fall semester. Unfortunately, by the time we had developed our ideas the fall semester was starting. Not wanting to wait until the next fall, we jumped into the project too quickly. Both authors were teaching this practice course. Each course had the same objectives and same textbooks and we thought our plans for using MBPL were the same. Looking back we identified that our methods of using MBPL were similar but had several differences. One author spent more class time on MPBL than did the other author. One author had student groups report and discuss their cases more frequently during the semester. In retrospect we are aware that we should have taken the time to write a manual with step-by-step directions for using MBPL. Having a written manual would also allow the study to be replicated. Also, we could have had one class use MPBL and had the other class serve as the control group. This method would have strengthened our research design. If we had a control class, we would have been able to assess whether the changes that occurred could be attributed to using MPBL. We didn’t use a control class because both of us thought MPBL was a better way to teach this course. Our commitment to offering students what we considered the best teaching and learning method meant that, ethically, we could not utilize a control group. (Even if the use of a control group had been possible, however, sample size was too small to demonstrate significant findings.) Although most students showed good improvement from the pre-test to the post-test some students did not improve and some had less satisfactory answers on the post-test. There could be many reasons for this decline. We believe a significant reason was that the post-test was given on the last day of class when the students were anxious to get finished. Some of their answers were very brief and seemed hurried. In the future we will not wait until the final class to do the post-test. Finally, with more time we could have followed the students into their field practicum setting to assess whether what they learned using MPBL enhanced their ability to think like professional social workers in actual practice. Although this would strengthen the research, it would be very complicated to control because each student is in a different setting with a different field instructor. Even with a control group there would be many uncontrollable variables. Given all of these complications, we still believe that meeting with the students at the Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 8 end of their practicum semester to discuss their perceptions of the value of MPBL to their education would have been valuable. VII. Conclusion. As indicated in preceding paragraphs, each author believed that utilizing MPBL in practice classes was the best way to teach social work students how to think like professional social workers. Each certainly expected the method to show success. When the pre- and post-tests were analyzed and the evidence of success was inconclusive, all members of the research team were somewhat disappointed because the level of student improvement was less than what we had hoped. However, the lessons learned about the MPBL method and about conducting SOTL research have been invaluable. Just as there is a process of learning to “think like a professional,” there is, perhaps, a process of learning to “think like a SOTL researcher” as well. It is our hope that other SOTL researchers will continue this research using MPBL with students in other disciplines. As researchers in other disciplines have found (Lundeberg, 1999; Wolfer and Scales, 2006), using cases can help students transfer learning from the classroom to the world of work. In MPBL the instructor models for students the questions that professionals use as they approach challenging case situations. Working with these questions seems to be a logical way to help students in any discipline learn how to think about challenging case situations. References Altshuler, S.J., and Bosch, L.A. (2003). Problem-based learning in social work education. Journal of Teaching in Social Work, 23(1/2), 201-215. Angelo, T.A. and Cross, K.P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass. Baldwin, W., Bankston, P., Anderson, W.M., Echtenkamp, S., Haak, R., Smith, P, Iatridis, P.G. (2002). Can students in a modified PBL curriculum exceed the national mean on USMLE Part 1? Medical Education, 36 (8), 791-795. Chan, C.L.W. and Ng, S.M. (2004). The social work practitioner-researcher-educator: Encouraging innovations and empowerment in the 21 st century. International Social Work, 47(3), 312-320. Chang, V.N. ,Scott, S.T., and Decker, C.L. (2009) Developing Helping Skills: A Step by Step Approach. Belmont, CA: Brooks/Cole. Edwards, S. and Hammer, M. (2005). Laura’s story: Using problem-based learning in early childhood and primary teacher education. Teaching and Teacher Education, 22 (4), 465-477. Gelman, C.R. and Mirabito, D.M. (2005). Practicing what we teach: Using case studies from 9/11 to teach crisis intervention from a generalist perspective. Journal of Social Work Education, 41(3), 479-494 Williamson, S. and Chang, V. Journal of the Scholarship of Teaching and Learning, Vol. 9, No. 2, June 2009. 9 Goodnough, K. (2005). Issues in modified problem-based learning: A self-study in pre-service science-teacher education. Canadian Journal of Science, Mathematics, and Technology Education, 5 (3), p289-306. Herreid, C.F. (1994). Case studies in science education. Journal of College Science Teaching (23) 221–229. Koerin, B. B., Harrigan, M. P., and Reeves, J. W. (1990). Facilitating the transition from student to social worker: Challenges of the younger student. Journal of Social Work Education. 26(2), 199-208. Lager, P.B. and Robbins, V.C. (2004). Field education: Exploring the future, expanding the vision. Journal of Social Work Education, 40 (1), 3-12. Lam, D. (2004). Problem-based learning: An integration of theory and field. Journal of Social Work Education, 40(3), 371-391. Lundeberg, M. A. (1999). Discovering teaching and learning through cases. In M. A. Lundeberg, B. B. Levin, and H. L. Harrington (Eds.), Who learns what from cases and how? The research base for teaching with cases (pp. 3 – 23). Mahwah, NJ: Lawrence Erlbaum. Margetson, D. (1991). Why is problem-based learning a challenge? In D. Boud and G. Geletti (Eds.), The challenge of problem based learning. New York: St. Martin's Press, pp. 42-50. Middendorf, J., and Pace, D. (2005). Decoding the disciplines: A model for helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning, 98, 1-12. Saatci, E. (2008). Problem-based learning in an intercultural business communication course. Journal of Business and Technical Communication, 22 (2), 237-260. Savery, J.R. (2006). Overview of problem-based learning: Definitions and distinctions. The Interdisciplinary Journal of Problem-based Learning, 1(1), 9-20. Spencer, J. (2003). ABC of learning and teaching in medicine: Learning and teaching in the clinical environment. BMJ, 326, 591-594. Retrieved November 29, 2008 from http://www.bmj.com/cgi/reprint/326/7389/591 Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research and Evaluation, 7(17). Retrieved January 9, 2007 from http://PAREonline.net/getvn.asp?v=7andn=17 Thompson, J., Licklider, B., and Jungst, S. (2003). Learner-centered teaching: Postsecondary strategies that promote ‘Thinking Like A Professional’. Theory Into Practice, 42(2), 133-141. Wolfer, T.A and Scales, T.L. (Eds.) (2006). Decision cases for generalist practice: Thinking like a social worker. Belmont, CA, Brooks/Cole.