Microsoft Word - HoyertFinal.doc Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006, pp. 1 - 13. A Brief Intervention to Aid Struggling Students: A Case of Too Much Motivation? Mark Sudlow Hoyert and Cynthia D. O’Dell1 Abstract: This study examines two interventions for altering achievement goals in an effort to enhance academic success in struggling Introductory Psychology students. The procedures involved an in-class lecture and an interactive computerized tutorial. Both procedures were successful in altering motivation. This led to changes in study activities and increased examination and course grades. In fact, the rate of failure was cut in half in the students who completed the interventions. However, the procedures were not as successful at attracting students who were at risk for academic failure. Only 12% of the target students completed the first intervention while 55% of the target students completed the computerized tutorial. Participation in the computerized intervention was enough to markedly reduce the number of failing grades earned by the class as a whole. Additions and alterations to the interventions are proposed to extend their efficacy. Keywords: achievement goals, goal orientation, academic success, mastery, performance I. Introduction Many teachers firmly believe that motivation is important to the learning process. Most teachers believe that a minimal amount of motivation is necessary and that higher amounts of motivation are better. Teachers often attribute problems in the learning process to a lack of motivation. In this paper, we explore some of the links between motivation and academic success. Our work is informed by a theory, alternatively known as Achievement Motivation Theory or Goal Orientation Theory, which posits that multiple varieties of motivation exist and that moderate levels of some varieties are preferable to high levels of other varieties. In fact, some of the problems in the learning process may result from high levels of particular kinds of motivation. Goal orientation theory suggests that when students engage in a class, they strive to reach one or more goals (Ames & Archer, 1988; Dweck & Leggett, 1988; Harackiewicz, Barron, & Elliot, 1998; Hidi & Harackiewicz, 2000). Two goals are of primary importance: mastery goals and performance goals. Students who adopt mastery goals are interested in learning the material in the class and strive to master that material (Ames, 1992; Blumenfeld, 1992; Dweck & Leggett, 1988). Students who pursue performance goals are interested in demonstrating their competence, especially relative to other students (Dweck & Leggett, 1988; Urdan, 1997). Mastery and performance goals are multidimensional motivational constructs. That is, these goals provide a framework through which a variety of behavioral, cognitive and affective responses are energized and directed (Ames, 1992; Dweck & Leggett, 1988; Blumenfeld, 1992). For instance, research under a variety of laboratory and classroom settings has found that students who pursue mastery goals display a wide variety of largely adaptive behaviors and 1 Respectively, Professor of Psychology, Indiana University Northwest, mhoyert@iun.edu, and Associate Professor of Women’s Studies and Psychology, Indiana University Northwest, codell@iun.edu. Address correspondence to Mark Hoyert, Psychology Department, Indiana University Northwest, Gary, IN 46408, 219-980-6687. Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 2 attitudes not seen in all students. Mastery oriented students seek to improve their competence through acquiring new skills and knowledge and by surmounting novel and difficult problems. Mastery goals have been found to be associated with increased interest, the enjoyment of challenge and challenging tasks, and the belief that competence is obtained incrementally through effort. Students who pursue mastery goals commonly use effective learning strategies such as elaboration and organization, have developed multiple strategies, are interested in developing new skills, become involved in the learning process, display greater persistence, and are likely to respond to challenges through the use of greater effort and the exploration of alternative learning strategies (Albaili, 1998; Ames, 1992; Ames & Archer, 1988; Blumenfeld, 1992; Elliot & Harackiewicz, 1994; Graham & Golan, 1991, Harackiewicz, Barron, Carter, Lehto, & Elliot, 1997; Harackiewicz, Barron, Tauer, Carter, & Elliot, 2000; Harackiewicz & Elliot, 1998; Pintrich, Zusko, Schiefele, & Pekrum, 2001). The pursuit of performance goals is more complex, but sometimes has been associated with a less adaptive set of academic outcomes such as self-aggrandizing, task aversion, the pursuit of effort minimizing strategies, a reluctance to seek help, impaired problem solving, greater feelings of self-consciousness, self-handicapping, and helplessness. Students who adopt a performance goal orientation seek to elicit favorable judgments of their competence and avoid negative evaluations. These individuals tend to prefer and to seek out easier tasks where success and validation can be obtained and view competence as static and unaffected by effort. The maladaptive behaviors are more likely to appear when validation is not available (Albaili, 1998; Dweck, 1999; Dweck & Leggett, 1988; Graham & Golan, 1991; Harackiewicz, Barron, & Elliot, 1998; Harackiewicz, Barron, Tauer, Carter, & Elliot, 2000, Harackiewicz & Elliot, 1993; Kong & Hau, 1996; Midgley, 1993; Pintrich & Schunk, 1996; Pintrich, Zusko, Schiefele, & Pekrum, 2001; Ryan, Gheen, & Midgley, 2000; Somuncuoglu & Yildirim, 1999; Urdan, Kneisel, & Mason, 1999; Urdan & Maehr, 1995; Urdan, Midgely, & Anderman, 1998). The adoption of mastery goals should be associated with academic accomplishment. In the college classroom, this would translate into better examination and course grades and a higher Grade Point Average. One would expect that a mastery oriented student who monitors comprehension, connects new information with old, can discriminate more important information from less important information, who sets goals, uses elaboration and other adaptive learning strategies should attain higher levels of academic success than a performance oriented student who procrastinates and self-handicaps. However, most studies employing an objective measure of academic performance in the classroom have not found a consistent or robust advantage. Only a third of these studies have reported higher grades for mastery oriented students (Bouffard, Boisvert, Vezeau, & Larouche, 1995; Eppler & Harju, 1997; Pintrich, Zusko, Schiefele, & Pekrum, 2001; Schraw, Horn, Thorndike-Christ, & Bruning, 1995; Wolter, Yu, & Pintrich,1996) and about two thirds have obtained null results (Beck, Rorer-Woody, & Pierce, 1991; Elliot & Church, 1997; Harackiewicz & Elliot, 1998; Harackiewicz, Barron, Carter, Lehto, & Elliot, 1997; Harackiewicz, Barron, Tauer, Carter, & Elliot, 2000; Harju & Eppler, 1997; Greene & Miller, 1996; Pintrich & Garcia, 1991; Pintrich & Garcia, 1993; Roedel & Schraw, 1995). No studies have found that pursuing mastery goals is associated with declining academic success. We have identified a set of conditions under which the relative pursuit of mastery and performance goals plays a potent role in academic success. One goal orientation theory (Dweck, 1999; Dweck & Leggett, 1988) suggests that the effects of these goals can be altered or amplified during stress or challenge. We can identify an obvious and frequent source of stress in Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 3 our students, failure on an examination. To see how relevant achievement goals are in this situation, we tracked students to see what happened on the subsequent examination after an objective failure. Students who endorsed mastery orientation enjoyed a 15-point increase on the next examination. Students who pursued performance goals suffered a 10-point decrease. Not only did different patterns of goal orientation lead to an effect on academic performance, but it was very robust and consistent. A 25-point difference in examination scores can lead to a difference of two or three letter grades. Further, ninety-five percent of the mastery-oriented students attained a higher examination grade on the subsequent test. About half of the performance-oriented students had a decrease on their test grade on the subsequent examination (Hoyert & O’Dell, 1999; 2000; 2001a; 2001b; 2004a, 2004b; O’Dell & Hoyert, 2000, 2002). As a result, we developed an intervention to aid struggling Introductory Psychology students. We measured goal orientation at the onset of the semester and invited students who primarily pursued performance goals to attend brief tutorial sessions after they had failed an examination. Student peers led the sessions. The goal of the intervention was to alter goal orientation in order to prevent the decrement sometimes seen in highly performance-oriented students. The tutors’ coached students on a variety of techniques including orientation modeling from several different perspectives, discussion of multiple study techniques, goal setting, and value referencing. The intervention provided students with knowledge about and practice expressing mastery goals. It must be pointed out that the tutorials only address motivational issues. We did not cover classroom material. The intervention produced clear benefits: students in the intervention began to endorse mastery goals to a greater extent and earned higher examination and course grades. The average student achieved a 15-point increase on their next examination. Further, the effects persisted over the duration of the semester even though the intervention ended. The D/F failure rate was nearly cut in half (to 47%). These improvements are even more impressive when their performance is compared to that of the control participants. One could view the control group as a predictor of what could have happened to the intervention group participants without the intervention. These two groups of participants had the same goal orientation profile and the same grades on the first examination. Both sets of students endorsed performance goals more than mastery goals. After the first examination, the control students= grades declined precipitously. The failure rate (D, F) was 78%. The successes of this intervention are heartening. However, the procedure reached a relatively small number of students and was labor intensive. Over the course of three semesters we were able to involve a little over 60 students. We believe that there are many more students who could benefit from the intervention. In any particular class, we estimate that about half of the students are predominantly performance-oriented and that about half of the students who are predominately performance-oriented will fail at least one examination. This amounts to several hundred students each year at our school. In this research, we explored two techniques for extending the intervention to reach all struggling performance-oriented students enrolled in sections of Introductory Psychology in an effort to alter the success profile in the class as a whole. II. Method A. Instruments Goal orientation was measured using a locally developed inventory modeled after Roedel, Schraw, and Plake’s (1994) Goals Inventory. This instrument consists of 24 statements Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 4 rated on a 5-point Likert-type scale for strength of agreement. Students were asked to consider how much each of the statements applies to themselves within the Introductory Psychology class. The statements assess attitudes and behaviors towards learning and performance goals as described by Dweck and Leggett (1988). B. Procedure Intervention 1: Guest Lecture. During the first day of the semester and after the final examination, the goal inventory was administered to all students. One hundred eighteen students were enrolled in the class. Many of these students experience an academic challenge over the course of the semester. Eleven students did not take the first examination. Twenty-eight students received a failing grade on the first examination. Sixty students failed at least one examination over the course of the semester. Eighty-three students received a grade that was less than a C on at least one examination over the course of the semester. The intervention occurred during the class immediately after the first examination and took the form of a lecture and discussion. In the intervention, we provided information about goals and the meaning of failure, and practiced setting mastery goals. All of the exercises were aimed at increasing the adoption of mastery orientation. The techniques included orientation modeling from several different perspectives, discussion of multiple study techniques, goal setting, and value referencing. Following the class, students were instructed to write a paper defining the two goals, describing individuals who exemplify the traits of each goal, and considering their own experience with goal orientation. Seventy-two students attended the class and 19 wrote and submitted the paper. At the end of the academic term, the students’ introductory psychology examination and course grades were obtained from the instructor. Intervention 2: Interactive Computerized Tutorial. This intervention occurred with a different class during a different semester. Following the first examination all students received a CD ROM containing the goals inventory and the intervention. The intervention provided students with information about goals and the meaning of failure and practiced setting mastery goals. It took the students 35 to 55 minutes to complete the tutorial. Two hundred-twelve students were enrolled in the class. Many of these students experienced an academic challenge during the semester. Eighty-two failed the first examination. One hundred fifty-six failed at least one examination over the course of the semester. One hundred eighty-three received a grade that was less than a C on at least one examination over the course of the semester. One hundred thirty-seven (65% of the class) students completed the intervention; 75 did not. There were 147 students who were performance-oriented and failed at least one examination. Eighty-two out of the 147 (56% of the group) target students completed the intervention. III. Results A. Intervention 1 Nineteen students completed the motivational intervention. Nine of these were not struggling or were not high-performance oriented students. Ten were students that we predict could struggle in the class. In this study, we are primarily interested in struggling students. We predict that a student will be at risk for academic failure when they pursue performance goals more than learning goals and when they fail an examination or are otherwise challenged. We will refer to these students as the “target” students. Because the 10 struggling students in the Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 5 intervention group and the 9 other students responded to the intervention approximately the same we will display the group data because it is based on more observations. Ninety-nine students did not participate in the intervention. Of these 72 were at risk for failure because of grades and goal orientation. Thus 12% of the target students completed the intervention. The students who completed the intervention did not differ from the students who did not complete the intervention in goal orientation or grades on examination 1. The students who wrote the paper had a mean Mastery score of 3.69 (SD=0.79) (both goal orientation scores can range from 1 to 5). The students who did not write the paper had a mean Mastery goals score of 3.86 (SD=0.88). These did not differ (t(94)=.612, p=.542). The same pattern holds for Performance goals (Intervention: M=3.59, SD=0.61; Comparison: M=3.75, SD=1.06; t(94)=.425, p=.672). The scores on the first examination also did not differ (Intervention Group: M=74.63, SD=14.00; Comparison: M=71.18, SD=14.19, t(105)=0.963, p=.338). Because the two groups were similar, the students who did not complete the intervention can be treated as a convenience control group or a comparison group. The intervention produced an increase in the pursuit of mastery goals. Before the intervention the Mean was 3.69 (SD=0.79) and after it had increased to 3.78 (SD=0.54) However, the increase was not statistically significant (t(18)=0.527, p=.604). The intervention also produced a decrease in performance goals (first day: M=3.86, SD=0.89; Last day M=3.77, SD=0.92; t(18)=.749, p=.463). In contrast, students who had not completed the intervention displayed dramatic decreases in both mastery (first day: M=3.58, SD=0.55; last day M=3.18, SD=1.00; t(37)=2.40, p=.021) and performance (first day: M=3.76, SD=1.07; last day M=3.37, SD=0.70; t(37)=2.31, p=.026) goals over the course of the semester. Students who completed the intervention displayed steadily but slightly increasing examination grades (exam 1 M=74.63, SD=14.00, exam 2 M=75.89, D=14.60, exam 3 M= 76.44, SD=12.53, exam 4 M=80.94, SD=12.79, exam 5 M=78.09, SD=10.11). The gradual increase is not statistically significant (F(4,64)=1.11, p=.358). The scores on Examination 1 for students who did not complete the intervention were very similar to the students who completed the intervention. However, after the first examination, the grades for the students who did not complete the intervention declined precipitously (exam 1 M=71.18, SD=14.25, exam 2 M=68.33, SD=15.98, exam 3 M= 65.42, SD=13.77, exam 4 M=64.75, SD=19.96, exam 5 M=57.96, SD=13.57). This decline is significant (F(4,144)=6.01, p<.001). The overall pattern of increasing grades for the students who completed the intervention is different from the pattern of decreasing grades for the comparison students (F(1,204)=3.42, p=.010). Figure 1 presents the difference in mean examination grades between students who completed the intervention and those who did not. On Examination 1 the difference was less than 3 points. With each subsequent exam, the disparity widened. By examination 5, there was a 20-point difference between the test scores of the two groups. The intervention also led to improvements in their final course grades. Table 1 displays the percentage of grades in the two groups. We compared the semester average for the two groups. Students who completed the intervention earned higher average grades than the other students (intervention: M=78.9, SD=19.47, comparison: M=46.89, SD=27.99; t(116)=4.75, p<.001). Because we are primarily interested in the effects of the intervention on the target students who are highly performance-oriented and who fail an examination, we report their scores separately. The pattern of results was similar for these target students as for the overall sample. The intervention led to slight increases in mastery goals (first class: M=3.62, SD=0.83; Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 6 Figure 1: Difference in Examination Scores between Intervention and Comparison Groups 0 5 10 15 20 25 1 2 3 4 5 Examination M ea n di ff er en ce be tw ee n sc or es Table 1: Grade Distributions for Students Completing Intervention 1 and for the Comparison Group Grade A B C D F/W Completed Intervention 26.3 31.6 31.6 5.3 5.5 Comparison Group 5.2 15.6 17.7 15.6 45.8 last class: M=3.74, SD=0.53; t(17)=0.594, p=0.560) and slight decreases in performance goals (first class: M=3.89, SD=0.90; last class: M=3.78, SD=0.84; t(17)=0.868, p=0.398). Target students who failed an examination and did not complete the intervention demonstrated considerable decreases in mastery goals (first class: M=3.58, SD=0.55; last class: M=3.18, SD=1.00; t(17)=2.40, p=0.021), and performance goals (first class: M=3.76, SD=1.07; last class: M=3.37, SD=0.70; t(17)=2.31, p=0.026). As can be seen in Table 2, the target students who completed the intervention also maintained higher grades. Table 2: Grade Distributions for Performance-Oriented Students Completing Intervention 1 and for the Comparison Group Grade A B C D F/W Completed Intervention 0.0% 30.0% 60.0% 0.0% 10.0% Comparison Group 0.0% 6.8% 23.3% 19.2% 50.7% Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 7 B. Intervention 2 One hundred thirty-seven students completed the motivational intervention. Fifty-five of these were not struggling or were not high-performance oriented students. We predict that 82 of these students would struggle in the class because of goal orientation. Seventy-five students did not participate in the intervention. Of these, 66 were “at risk” because of grades and goal orientation. Thus about 55% of the target students completed the intervention. The scores of students who completed the intervention did not differ from the students who did not complete the intervention on examination 1 (Intervention, M=54.5, SD=14.12; Comparison, M=55.0, SD=14.56; t(197)=0.23, p=0.749). Because the two groups were similar, the students who did not complete the intervention can be treated as a comparison group. The intervention produced an increase in the pursuit of mastery goals in both the target students as well as the general body of students who completed the intervention. Since our primary interest is in the target students we will present these data first. Before the intervention the Mean was 4.10 (SD=0.09) and after it had increased to 4.29 (SD=0.10) (t(80)=3.34, p=0.002). The intervention also produced a decrease in performance goals. Before the intervention, the mean was 4.07 (SD=0.42) and after it had decreased to 3.06 (SD=0.91) (t(80)=8.69, p<0.001). Target students who completed the intervention displayed steadily increasing examination grades (exam 1 M=54.4, exam 2 M=55.9, exam 3, M= 56.0, exam 4 M=57.9, exam 5 M=55.6). In contrast, the students who did not complete the intervention earned increasingly lower grades (exam 1 M=55.0, exam 2 M=47.3, exam 3, M= 43.8, exam 4 M=37.7, exam 5 M=31.0). The grade on examination 1 for the two groups is not statistically significant. However, the overall pattern of increasing grades for the students who completed the intervention and decreasing grades for the comparison group is significant (F(1,109)=43.06, p<0.001). Figure 2 presents the difference in mean examination grades between students who completed the intervention and those who did not. On examination 1, the students who did not complete the intervention maintained a higher mean exam grade than the students who would later complete the intervention. After completing the intervention, the intervention students outscored the comparison group by nine points. With each subsequent exam, the disparity widened. By examination 5, there was a 25-point difference between the two groups. Figure 2: Difference in Examination Scores between Intervention and Comparison Groups -5 0 5 10 15 20 25 1 2 3 4 5 Examination M ea n di ff er en ce be tw ee n sc or es Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 8 The intervention also led to improvements in their final course grades. The DFW rate in Introductory Psychology is typically similar to the percentage of students who received an F on any one test. In this class the percentage of students who received an F was 74%. Thus, we could expect a very high DFW rate. The comparison group resembled this historical pattern. Table 3 reports that 76% of these students failed the class. In contrast, in the intervention group, only 37% of the students received an F for the course. The pattern of grades in the intervention group is statistically different from the comparison group (X2(4)=12.38, p<0.001). Table 3: Grade Distributions for Students Completing Intervention 2 and for the Comparison Group Grade A B C D F/W Completed Intervention 1.2% 9.9% 21.0% 30.9% 37.0% Comparison Group 0.0% 1.5% 9.1% 13.6% 75.8% We are also interested in the overall effect on grades in the class. Table 4 displays the effects on course grades. Given the historical pattern and the pattern of goals and examination scores we would expect that 156 students would have earned less than a C. After the Intervention, we found that 123 students received a D, an F, or withdrew. Table 4: Overall Effect on the Grade Distribution Students Completing Intervention 2 and the Predicted Grade Distribution A, B, or C D, F, or W Intervention 42.0% 58.0% Predicted 26.4% 73.6% IV. Discussion Teachers are frequently puzzled by the disparate reactions of students to challenge. After failing an examination some students will react with despair and will give up. Other students will react to the same challenge by buckling-down, increasing their effort, and developing better study habits. The study of goal orientation seems to provide some insight into patterns of reactions such as these (Dweck, 1999; Dweck & Leggett, 1988; Hoyert & O’Dell, 1999; 2000; 2001a; 2001b; 2004a, 2004b; O’Dell & Hoyert, 2000, 2002). The interventions assessed here attempt to encourage students who may have been prone to despair to adopt a more positive approach to challenge. We have now attempted three variants of the intervention. The first involved peer-tutors. It was not successful in reaching large numbers of students. However, when students participated, they enjoyed changed goals and increased academic success (Hoyert & O’Dell, 2004b). To an extent, goal orientation theory suggests a reason for the lack of participation. Highly performance-oriented students tend to avoid evidence of a lack of competence. To a Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 9 performance-oriented student, seeking help from a tutor is an admission that they are not capable of completing the task without assistance, something that a truly competent student could have done. As a result, we developed two variants of the intervention that can be embedded in the course structure so that participation will not carry any perceived stigma. Providing the intervention through a guest lecture was similar in effect to the peer-tutors. Not very many students were willing to participate. However, when they completed the intervention, they enjoyed a changed pattern of goals, greater persistence, and higher grades. One unique finding occurred during this particular class. Mastery and performance goals tend to remain rather stable over time. Typically, when we measure these goals at the beginning and the end of a class without an intervention, they are approximately the same for each student and for the class as a whole. In this class, goals changed dramatically. Both mastery and performance goals declined significantly over the semester. We have not observed this before and suspect that it may have been related to messages provided by the instructor or to the pattern of academic success experienced by the students (Hoyert & O’Dell, 1999; 2000; 2001a; 2001b; 2004a, 2004b; O’Dell & Hoyert, 2000, 2002). Given the backdrop of declining goals, the increase in mastery goals observed in the students who completed the intervention provides strong evidence for the efficacy of the intervention. The intervention occurred once, early in the semester, and lasted about 45 minutes. However, the effects of the intervention persisted for the duration of the semester. It is noteworthy that the average examination grades of these students increased over the course of the semester in much the same way as the mastery goals. The students who completed the intervention passed the class and earned high grades at far higher rates than the students who did not. The major drawback of this intervention is that it did not attract many students. Only 12% of the target students completed the intervention. One of our goals in this research is to determine whether we can alter the pattern of academic success in a whole class. Because we did not attract many students to complete the intervention, we were not able to decrease the proportion of students who did not pass the class (DFW rate). The computerized tutorial has been the best procedure for involving students in the intervention to date. Over half of the students who we predicted could be at risk for academic failure participated in the intervention. After completing the brief intervention, they pursued mastery goals to a greater extent and were less interested in pursuing performance goals. The change in goals led to changes in studying and increased academic success over the duration of the semester. One of our interests is in trying to decrease the rate of failure in individual students. The students who completed the intervention earned failing course grades at less than half the rate of the historical pattern and at less than half the rate of the comparison group. Another interest is in altering the rate of failing grades in an entire class. It appears that the intervention was successful in decreasing the rate of the F grade by about 25%. The present interventions produce positive results. However, they still can be improved. Both procedures could be more fully integrated into the class structure to encourage more students to participate. For instance, making the programs mandatory components of the classes could be explored. In addition, the intervention has only been attempted within Introductory Psychology classes. An interesting extension could be to try to introduce the intervention in other introductory level courses in other disciplines. The success profiles in many mathematics and science classes are lower than introductory psychology. This would suggest that they are prime candidates for this type of intervention. Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 10 Finally, this study contained a troubling methodological flaw. Both interventions may have a selection problem. The students determined who would participate in the intervention. We have identified goal orientation and challenge as important variables in course success. We have observed that these variables are similar in the two groups. Thus, we believe that it is reasonable to use these students as a comparison group. To a certain extent, this was unexpected in the first intervention. However, we would predict that in the event of a low rate of participation, more learning-oriented students would participate in the intervention than performance-oriented students. Regardless, because students could determine participation, it could be that the differences in grades, persistence, and changes in grades are related to the self- selection. Another important future study could involve a procedure in which students are randomly assigned to the intervention or to one of several control conditions. We are optimistic that these changes will further strengthen our findings that the intervention positively alters goal orientation, examination scores and course grades in a college setting. References Albaili, M. (1998). “Goal orientations, cognitive strategies and academic achievement among United Arab Emirates college students.” Educational Psychology, 18, 195-203. Ames, C. (1992). “Classrooms: Goals, structures, and student motivation.” Journal of Educational Psychology, 84, 261-271. Ames, C., & Archer, J. (1988). “Achievement goals in the classroom: Students’ learning strategies and motivation processes.” Journal of Educational Psychology, 80, 260-267. Aronson, J., & Fried, C. (1998). “Reducing stereotype threat and boosting academic achievement of African Americans: The role of conceptions of intelligence.” Unpublished manuscript. Beck, H. P., Rorer-Woody, S., & Pierce, L. G. (1991). “The relations of learning and grade orientations to academic performance.” Teaching of Psychology, 15, 35-37. Blumenfeld, P.C. (1992). “Classroom learning and motivation: Clarifying and expanding goal theory,” Journal of Educational Psychology, 84, 272-281. Bouffard, T., Boisvert, J., Vezeau, C., & Larouche, C. (1995). “The impact of goal orientation on self-regulation and performance among college students.” British Journal of Educational Psychology, 65, 317-329. Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia: Psychology Press. Dweck, C. S. & Leggett, E. L. (1988). “A social-cognitive approach to motivation and personality.” Psychological Review, 95, 256-273. Elliot, A. J., & Church, M. A. (1997). “A hierarchical model of approach and avoidance achievement motivation.” Journal of Personality and Social Psychology, 72, 218-232. Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 11 Eppler, M. A. & Harju, B. L. (1997). “Achievement motivation goals in relation to academic performance in traditional and non-traditional college students.” Research in Higher Education, 38, 557-573. Graham, S., & Golan, S. (1991). “Motivational influences on cognition: Task involvement, ego involvement, and depth of information processing.” Journal of Educational Psychology, 83, 187-194. Greene, B. A., & Miller, R. B. (1996) “Influence on achievement: Goals, perceived ability, and cognitive engagement.” Contemporary Educational Psychology, 21, 181-192. Harackiewicz, J. M., Barron, K. E., Carter, S. M., Lehto, A. T., & Elliot, A. J. (1997). “Predictors and consequences of achievement goals in the college classroom: Maintaining interest and making the grade.” Journal of Personality and Social Psychology, 73, 1284-1295. Harackiewicz, J. M., Barron, K. E., & Elliot, A. J. (1998). “Rethinking achievement goals: When are they adaptive for college students and why?” Educational Psychologist, 33, 1-21. Harackiewicz, J.M., Barron, K.E., Tauer, J. B., Carter, R. A., & Elliot, A. J. (2000). “Short-term and long-term consequences of achievement goals: Predicting interest and performance over time.” Journal of Educational Psychology, 75, 183-205. Harackiewicz, J.M., & Elliot, A.J. (1993). “Achievement goals and intrinsic motivation.” Journal of Personality and Social Psychology, 65, 904-915. Harackiewicz, J.M., & Elliot, A.J. (1998). “The joint effects of target and purpose goals on intrinsic motivation: A mediational analysis.” Personality and Social Psychology Bulletin, 24, 675-689. Harju, B. L. & Eppler, M. A. (1997). “Achievement motivation, flow and irrational beliefs in traditional and nontraditional college students.” Journal of Instructional Psychology, 24, 147- 157. Hidi, S. & Harachiewicz, J.M. (2000). “Motivating the academically unmotivated: A critical issue for the 21st Century.” Review of Educational Research, 70, 151-179. Hoyert, M.S. & O’Dell, C.D. (1999). “Achievement motivation in traditional and non-traditional students.” Poster presented at the Annual Meeting of the Midwestern Psychological Association, April 29 - May 1, 1999, Chicago, IL. Hoyert, M.S. & O’Dell, C.D. (2000). “Achievement Motivation and the Impostor Phenomenon.” Poster presented at the Annual meeting of the Midwestern Psychological Association, May 5, 2000, Chicago, IL. Hoyert, M.S. & O’Dell, C.D. (2001a). “Goal Orientation and Challenge in College Students.” Paper presented at a symposium entitled “Toward a scholarship of teaching and learning in developmental psychology” at the Biennial Meeting of the Society for Research in Child Development, April 10-22, 2001, Minneapolis, MN. Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 12 Hoyert, M.S. & O’Dell, C.D. (2001b). “Achievement Motivation and Excuse Making.” Poster presented at the Annual meeting of the Midwestern Psychological Association, May 3-5, 2001, Chicago, IL. Hoyert, M. S., & O’Dell, C.D. (2004). “The impostor phenomenon and goal orientation.” Journal of the Indiana Academy of Social Sciences, 7, 1-10. Hoyert, M.S., & O’Dell, C.D. (2004). “Altering achievement motivation to enhance classroom learning.” Selected Papers from the 15th International Conference on College Teaching and Learning, 15, 119-135. Kong, C., & Hau, K. (1996). “Student achievement goals and approaches to learning: The relationship between emphasis on self-improvement and thorough understanding.” Research in Education, 55, 74-85. Midgley, C. (1993). “Motivation and middle level schools”, In Advances in Motivation and Achievement (Vol. 8, pp. 217-274). Greenwich, CT: JAI Press. O’Dell, C.D., & Hoyert, M.S. (2000). “Achievement Motivation and Academic Success in Traditional and Non-Traditional Aged College Students.” Poster presented at the 16th Biennial Meeting of the Conference on Human Development, April 14-16, 2000, Memphis, TN. O’Dell, C. D., & Hoyert, M.S.(2002). “Altering achievement motivation and the response to failure.” Poster presented at the CTUP Creative Classroom Poster Session held during the 73rd Annual Meeting of the Midwestern Psychological Association. Pintrich, P. R., & Garcia, M. (1991). “Student goal orientation and self-regulation in the college classroom.” In Advances in Motivation and Achievement (Vol. 11, pp. 123-158). Greenwich, CT: JAI Press. Pintrich, P. R., & Garcia, M. (1993). “Intraindividual differences in students= motivation and self-regulated learning.” German Journal of Educational Psychology, 7, 99-107. Pintrich, P. R., & Schunk, D. H. (1996). Motivation in Education: Theory, Research, and Applications. Englewood Cliffs, NJ: Prentice Hall. Pintrich, P. R., Zusho, A., Schiefele, U. & Pekrun, R. (2001). “Goal orientation and self- regulated learning in the college classroom: A cross-cultural comparison.” In F. Salili, C. Y. Chiu & Y. Y. Hong (Eds.), Student motivation: The culture and context of learning. New York: Plenum. Roedel, T. D. & Schraw, G. (1995). “Beliefs about intelligence and academic goals.” Contemporary Educational Psychology, 20, 464-468. Roedel, T. D., Schraw, G., & Plake, B. S. (1994). “Validation of a measure of learning and performance goal orientations.” Educational and Psychological Measurement, 54, 1013-1021. Hoyert, M.S. and O’Dell, C.D. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 13 Ryan, A.M., Gheen, H. H., & Midgley, C. (2000). “Why do some students avoid asking for help? An examination of the interplay among students’ academic efficacy, teachers= social- emotional role, and the classroom goal structure.” Journal of Educational Psychology, 90, 528- 535. Schraw, G., Horn, C., Thorndike-Christ, T., & Bruning, R. (1995). “Academic goal orientations and student classroom achievement.” Contemporary Educational Psychology, 20, 359-368. Somuncuoglu, Y. & Yildirim, A. (1999). “Relationship between achievement goal orientations and use of learning strategies.” Journal of Educational Research, 92, 267-277. Wolters, C. A., Yu, S. L., & Pintrich, P. R. (1996). “The relation between goal orientation and students= motivational beliefs and self-regulated learning.” Learning and Individual Differences, 8, 211-238. Urdan, T. C. (1997). “Examining the relations among early adolescent students’ goals and friends’ orientation toward effort and achievement in school.” Contemporary Educational Psychology, 22, 165-191. Urdan, T. C., Kneisel, L., & Mason, V. (1999). “Interpreting messages about motiation in the classroom: Examining the effects of achievement goal structures.” In Advances in Motivation and Achievement (Vol. 11, pp. 123-158). Greenwich, CT: JAI Press. Urdan, T.C., & Maehr, M. L. (1995). “Beyond a two goal theory of motivation: A case for social goals.” Review of Educational Research, 65, 213-243. Urdan, T., Midgley, C., & Anderman, E. (1998). “The role of classroom goal structure in students= use of self-handicapping strategies.” American Educational Research Journal, 35, 101-122. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006, pp. 14 – 38. Students’ Race and Participation in Sociology Classroom Discussion: A Preliminary Investigation Jay Howard1, Aimee Zoeller2 , and Yale Pratt3 Abstract: This study utilizes observation, survey and interview methodologies to investigate the impact of student race on participation in discussion in introductory sociology courses at a large Midwestern US university with a minority enrollment of approximately 15 percent. While results are mixed there is some evidence that white students participated at a higher rate than minority students. However, in certain circumstances (e.g., discussion of racism), minority students became the “experts” during particular class sessions and participated at a greater rate than did white students. Key Words: Discussion, College Students, Race, Learning, Interaction I. Introduction and Literature Review The 2003 Supreme Court decision on race sensitive college admission policies at the University of Michigan once again focused attention on minority students in higher education. Higher education researchers have long been interested in the impact of race on end-of-first-year degree plans (Pascarella, Wolniak, and Pierson, 2003), development of problem-solving and group skills (Terenzini, Cabrera, Colbeck, Bjorklund, and Parente, 2001), preferences towards collaborative learning (Cabrera, Crissman, Bernal, Nora, Terenzini, and Pascarella, 2002), and adjustment to college (Cabrera, Nora, Terenzini, Pascarella, and Hagedorn, 1999; Schwitzer, Ancis, and Griffin, 1999). Researchers have also compared the experiences and perceptions of African-American students who attend historically black colleges and universities with African- American students attending predominately white institutions (Terenzini, Yaeger, Bohr, Pascarella, and Amaury, 1997). Feagin, Cera and Imani (1996) concluded that African-American college students continue to face many obstacles in higher education. Feagin (2003) found that black college students face a continuum of discriminatory practices that included aggression, exclusion, dismissal of subculture and typecasting which may be responsible, in part, for declining college enrollment and graduation for black Americans. One largely unaddressed issue is whether these obstacles and discriminatory practices impact minority students’ participation in college classroom discussions. Antonio, et al (2004) demonstrated that white college students display higher levels of complex thought when they are placed in discussion groups with a black student. Given that participation in classroom discussion has also been associated with learning (Astin, 1985; Johnson and Johnson, 1991; Kember and Gow, 1994; McKeachie, 1990), critical thinking 1 Professor of Sociology and Head of Division of Liberal Arts, Indiana University Purdue University Columbus, 4601 Central Ave., Columbus, IN 47203, jhoward@iupuc.edu. This research was supported by grants from the Indiana University Purdue University Indianapolis Office for Professional Development and the Mack Center at Indiana University for Inquiry on Teaching and Learning. 2 Adjunct Lecturer in Sociology, Indiana University Purdue University Columbus, 4601 Central Ave., Columbus, IN 47203, anzoelle@iupui.edu 3 Yale Pratt was graduate research assistant at Indiana University Purdue University Indianapolis during the data gather portion of this project. He was tragically killed in an automobile accident before the project was completed. Howard and Zoeller wish to acknowledge his contributions to this study by including him as a co-author. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 15 (Garside, 1996; Smith, 1977; Weast, 1996), and degree completion (Tinto, 1975; Tinto, 1997), it makes sense to ask, does race matter in participation in classroom discussion? This study seeks to address this void in the literature on student participation by addressing the role of student race in classroom discussion in introductory sociology courses at a large urban university with a racially mixed student population. Studies of student participation in discussion in the college classroom have addressed a number of variables thought to have significant impact including student gender, student age, instructor gender, class size, instructor traits, student traits, and classroom environment. The variable most often examined is student gender. This line of research springs from Hall and Sandlers’ (1982) “chilly climate” thesis which postulated that patterns of interaction and behavior in the college classroom create a climate that is less hospitable to female students than to male students. Despite the ongoing concern with student gender in classroom participation, the research support has been mixed. A number of studies have found that males participate more frequently than females (Auster and MacRone, 1994; Brooks, 1982; Crawford and MacLeod, 1990 [in their small college sample, but not in their university sample]; Crombie, Pyke, Silverthorn, Jones, and Piccinin, 2003; Fassinger, 1995; Karp and Yoels, 1976; O’Keefe and Faupel, 1987; and Statham, Richardson, and Cook, 1991). Brooks (1982) concluded that males participate more only in courses taught by female instructors. Other studies have suggested the opposite – males participate more frequently in male taught courses (Pearson and West, 1991; Sternglanz and Lyberger-Ficek, 1977) but not in courses taught by female instructors. Fassinger (1995) and Karp and Yoels (1976) found that females participate more in courses taught by female instructors than in courses taught by male instructors. A large number of studies found no significant difference in participation based on student gender (Boersma, 1981; Constantinople, Cornelius, and Gray, 1988; Corneilius, Gray, and Constantinople, 1990; Crawford and MacLeod, 1990 [in their university sample]; Heller, Puff, and Mills, 1985; Howard, James, and Taylor, 2002; Jung, Moore, and Parker, 1999). One possible explanation for the lack of significance of student gender is the presence a high percentage of females in the classroom. However, in their study which utilized a survey methodology, Crombie, Pyke, Silverthorn, Jones, and Piccinin (2003) failed to find a significant relationship between percentage of female students in a class and students’ participation. Several studies have demonstrated that student age has a stronger impact than student gender on participation in classroom discussion. Nontraditional students (25 years of age or older) have been consistently shown to participate more frequently than traditional students (less than 25 years of age) (Howard, Short, and Clark, 1996; Howard and Henney, 1998; Howard and Baird, 2000; Howard, James, and Taylor 2002; Jung, Moore, and Parker, 1999). However, Crombie, Pyke, Silverthorn, Jones and Piccinin (2003) found no differences in participation by student age in a study that utilized self reports rather than observation. One study by Faust and Courtenay (2002), who observed only a single section of a single course, found the opposite – that traditional students contributed to class discussion more frequently than did nontraditional students. Instructor gender is another variable that has been examined in relation to students’ participation in classroom discussion. The results have again been mixed. Some studies have found that there is more discussion in courses with female instructors (Canada and Pringle 1995; Constantinople, Cornelius, and Gray, 1988; Crawford and MacLeod, 1990; Fassinger, 1995; Howard and Baird, 2000; Howard, James, and Taylor, 2002; Karp and Yoels, 1976; Pearson and Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 16 West, 1991; and Statham, Richardson, and Cook, 1991). While none of the studies has suggested that students participate more frequently in courses with male instructors, numerous studies have failed to find a difference based on instructor gender (Auster and MacRone, 1994; Cornelius, Gray, and Constantinople, 1990; Crombie, Pyke, Silverthorn, Jones, and Piccinin, 2003; Heller, Puff, and Mills, 1985). Of these studies that failed to find an effect of instructor gender all but one (Cornelius, Gray, and Constantinople 1990) relied on student self reports via survey rather than observation. This may account for the lack of significant findings. Karp and Yoels (1976) reported that while students reported no effect of instructor gender in their survey responses, based on observations of actual classroom behaviors female students participated significantly more in female taught courses than in male taught courses. Howard and Baird (2000) and Howard, James and Taylor (2002) had the same result with survey responses failing to find a relationship between instructor gender and participation, but observations of classroom behavior revealing that students participate more frequently in courses with female instructors. Class size is another variable frequently found to have a significant impact on student participation in discussion. Most studies have found that more interaction occurs in smaller classes (Auster and MacRone, 1994; Constantinople, Cornelius, and Gray, 1988; Cornelius, Gray, and Constantinople, 1990; Crawford and MacLeod, 1990; Fassinger, 1995; Howard, Short, and Clark, 1996; Howard and Henney, 1998; Neer and Kircher, 1989). However, Crombie, Pyke, Silverthorn, Jones, and Piccinin (2003) and Karp and Yoels (1976) failed to find a significant impact of class size. Fassinger (1995) argued that instructor traits (e.g., gender) have little impact on student participation. Instead, student traits (confidence, comprehension, interest, preparation) and class traits (size, emotional climate, interaction norms, frequent large group discussions) were more important influences on participation. Likewise, Aitken and Neer (1993) concluded that it is a student trait (motivation or the lack thereof) that best explains students’ lack of participation. However, it is clear that instructor behaviors can influence student traits like comprehension and interest and can influence class traits such as emotional climate and interaction norms. Nunn (1996) argued that it is instructor teaching techniques (such as praise, posing questions, asking for elaboration, and using students’ names) that significantly improve levels of discussion. Thus Nunn concludes that instructors do play an important role in student participation. Fritschner (2000) found that students were more likely to participate in 300-400 level courses than in 100- 200 level courses. Despite this extensive research examining numerous variables, to date research focusing on classroom discussion has not directly addressed the impact of race on college students’ participation. However, some have argued, based on personal experience rather than systematic research, that minority students at predominately white institutions manifest a fear of failure that may cause them to participate in class discussions less frequently than white students (see for example, Saufley, Cowan, and Blake, 1983). Likewise, Asian students enrolled in Australian universities have been presumed to bring learning experiences that favor passive rote reproduction and teacher centered learning in contrast to the active learning and critical thinking required in class discussions which are more typical in Australia and the West (see for example, Ballard and Clanchy, 1991). Adams (1992) argued that men and women of culture have alternative cultures which imply a need for more collaborative and less competitive instructional design. One possible interpretation of Adams’ (1992) argument is that students of color may be better prepared for participation at least in collaborative classroom discussions. Hardiman and Jackson (1992) argue that instructors’ failure to understand and respect the racial identity of Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 17 students can lead to volatile situations in the classroom and on campus. Weinstein and Obear (1992) suggest that majority group students can “trigger” (p. 44) defensive and intense emotional reactions from minority group members that can easily silence classroom discussion altogether. Each of these studies of the minority students rely primarily on personal experience and reflection rather than systematic research to determine whether minority students participate in classroom discussion at a different rate than majority students. This investigation seeks to fill this void. II. Methodology This study was conducted at both [identifying information removed] and [identifying information removed]’s satellite campus in [identifying information removed]. A triangulation of research methods was utilized to examine students’ participation in classroom discussions in introductory sociology courses. As noted above, multiple methods are important when examining student participation in the college classroom. While surveys allow researchers to access students’ attitudes and beliefs about their own participation, often students’ self reports are not supported by observations. Thus this study utilizes observation, survey and interview methodologies to provide the fullest possible picture of student participation in the college classroom. Both for convenience and in order to limit the effect of variation in the curriculum on participation in discussion we chose to limit our investigation to introductory sociology courses. All eleven instructors teaching in the fall 2003 semester at [identifying information removed] were invited to participate in the study. Nine instructors (six males, three females) agreed to participate. Two instructors (one male, one female) chose not to participate. One section of each of the nine instructors’ introductory sociology courses were observed for four class meeting sessions. The observations were spread over the course of the semester with one observation occurring approximately every four weeks. During our observations, we kept track of student participation in discussion by using a seating chart to note students’ gender, approximate age (traditional or nontraditional), and race (African-American, Asian, Hispanic, Mixed, or White) as it appeared to the observer. Any verbal response by students regardless of length or content was counted an instance of participation. Thus a brief response (e.g., “Can you repeat the definition?”) counted equally with a longer comment or question that demonstrated critical thinking. We also kept more general field notes regarding activity in the classroom. Eight of the nine courses were “regular sections” with a maximum enrollment of 45 students. One of the observed courses was a “mass lecture” section with an enrollment of 182 students at the start of the semester. A total of 36 class meetings were observed with 1402 students in attendance (15.5 percent non-white). During the last three weeks of the semester all students in attendance in 15 sections of Introduction to Sociology (three instructors taught more than one section) were given a survey to assess students’ perceptions of their participation in classroom discussion and their reasons for participation and for non-participation. A total of 441 students completed the survey (13.2 percent non-white). Finally, the researchers interviewed the nine instructors and ten students from the courses observed. We sought to interview an equal number of white and non-white students. This proved to be a since there were relatively few non-white students. Also because [identifying information removed] is a commuter campus, it was difficult to schedule interviews with students who frequently left campus whenever they were not in class. Numerous students agreed Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 18 to be interviewed, but then failed to appear at the agreed upon time. Eventually, were are able to interview five white and five non-white students, a significantly fewer than the 20 students we had hoped to interview. In the interviews, students were asked about the effectiveness of discussions for facilitating learning and their perceptions of the students who participate most frequently in class discussion. III. Results and Discussion Table 1 presents the mean number of interactions per student per class session by student race, student gender, and student age. The typical 75 minute class session averaged almost 50 interactions from the 39 students in attendance. This resulted in a mean of 1.27 interactions per student. However, a caution is necessary. Computing mean interactions can be somewhat misleading because as Table 1 indicates over two thirds of all students present fail to contribute to discussion in a typical class session. On average, around 12 students, or 30 percent of those present, participate in discussion for a mean of 4.3 interactions per student participant. Table 1 also presents results by student race. An ANOVA comparison of means is used to test for significant differences in mean interactions per student and Kendall’s tau is used to test for significant differences in the percentage of students participating. The vast majority of nonwhite students were African American (72.8%). Because there were so few Asians (15.6%), Hispanics (6.0%), and mixed-race (4.6%) students in the sample, meaningful comparison of minority racial groupings were not possible. Therefore, students were grouped by whites (84.5 percent of those present) and non-whites (15.5 percent of those present). The results reveal no significant differences between whites and non-whites. The percentage of whites and non-whites participating in classroom discussion is nearly identical (29.6 percent of whites compared to 29.5 percent of non-whites). White students had a higher mean interaction per student (1.31 to 1.05); however this difference was not statistically significant. The lack of significant findings may be due to the nature of the sample. If it were possible to separate the various minority groups, the results may have varied. For example, including Asians in the same category with African Americans and Hispanics may be masking differences between racial groupings. Interview evidence indicated that students themselves were uncertain whether minority students participated at the same rate as did white students. In their comments students would quickly note that because there were so many more white students in class, most contributions to class discussion came from white students. But they were unsure whether minority students participated at a rate that was proportional to their number. For example, a nontraditional black female student noted, “There were not too many minorities in the class. There were more whites to speak out. The minorities speak out just as much. But it doesn’t look like it because…there were not many minorities in the class.” Instructors, on the other hand, tended to perceive minority students as less frequent contributors to discussion. One female instructor remarked, “I think it is always hard to have minorities be comfortable enough to speak up.” Another female instructor stated, “I feel that there is less verbal discussion among minorities. I don’t know why. African American students speak and respond less.” In our observations, the participation of white and non-white students most often was very similar. However, there were occasions when non-white students became the class “experts” on a given topic, such as police profiling. On these occasions, the non-white students became the dominant talkers for the class session. One nontraditional white male student commented, “When we were discussing race, I noticed that a lot of people of color really seemed Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 19 Table 1: Interactions per class session (ANOVA Comparison of Means) and percent of students participating (Kendall’s tau) by student race, student gender, and student age. Mean Interaction Per Session Mean Attend Mean No. Students Participate Percent Students Participate Mean Interaction Per Student N All 49.4 38.9 11.5 29.6 1.27 1402 Whites (84.5%) 43.1 32.9 9.8 29.6 1.31 1185 Non-whites (15.5%) 6.3 6.0 1.8 29.5 1.05 217 Males (28.9%) 12.3 11.3 2.9 25.4* 1.09 405 Females (71.1%) 37.1 27.7 8.7 31.3* 1.34 996 Traditional (89.7%) 33.2 34.9 9.1 26.1*** .95*** 1258 Nontraditional (10.3%) 16.3 4.0 2.4 60.4*** 4.08*** 144 White Males (24.3%) 11.3 9.5 2.5 26.7 1.19 341 Non-white Males (4.6%) 1.0 1.8 .3 18.7 .55 64 White Females (60.2%) 31.8 23.4 7.2 30.8 1.36 843 Non-white Females (10.9%) 5.4 4.3 1.4 34.0 1.25 153 White Traditional (77.3%) 30.1 30.1 7.9 26.1 1.00 1084 Non-white Traditional (12.4%) 3.1 4.8 1.3 25.9 .64 174 White Nontraditional (7.2%) 13.1 2.8 1.9 67.3** 4.67*** 101 Non-white Nontraditional (3.1%) 3.2 1.2 .5 44.2** 2.70*** 43 N 1779 415 1779 1402 *** Significant at the p < .001 level ** Significant at the p < .01 level * Significant at the p < at .05 level Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 20 to get involved on that topic. It seemed to hit more close to home.” Instructors noted the same tendency, “When we were talking about racism and talked about Hispanics, he [Hispanic student] talked more than he usually does (female instructor).” Table 1 also reveals that female students had a slightly higher, but not statistically significant, mean number of interactions per class session compared to 1.09 for male students (1.34 to 1.09). Contrary to the chilly climate thesis, a significantly higher percentage of female students participated in class discussion than male students (31.3 to 25.4%). Students differed significantly in their rates of participation by age (see Table 1). Nontraditional students, those the observers judged to be age 25 or older, had a mean number of interactions per class session which was over four times that of traditional students (4.08 to .95) and the percentage of nontraditional students who participated in discussion was more than double that of traditional students (60.4 to 26.1 percent). Table 1 also presents a comparison of students by both race and gender. White male students had a higher mean number of interactions per class session (1.19 to .55) and a higher percentage of those present participating (26.7 to 18.7) than their non-white male counterparts. However, neither difference was statistically significant. Again, the lack of significance may be due to sample size. Only 64 non-white males (4.6% of all students) were in the sample, making demonstrations of statistical significance difficult. White and non-white female students were much closer in terms of mean interactions per class session (1.36 to 1.25). While the difference is small and is not statistically significant, it is interesting to note that the percentage of non- white females participating in discussion was slightly higher than that of white females (30.8 to 34.0). When we compared students by both race and age (see Table 1), we again found white traditional students had a higher mean interaction per student (1.00 to .64) when compared with non-white traditional students. The percentage of students participating was nearly identical (26.1 to 25.9%). These differences were not statistically significant. For nontraditional students, however, whites had a significantly higher mean interaction per class sessions than did non- whites (4.67 to 2.70). The nontraditional whites also had a significantly higher percentage of students participating compared to nontraditional non-whites (67.3 to 44.2%). Table 2 presents a comparison of mean interactions per student per class session and the percentage of students participating in discussion by student race and instructor gender. Nearly 80 percent of students observed, including those in the mass lecture class, were in courses taught by male instructors. A significantly higher percentage of students in female instructor courses participated in discussion as compared to students in male instructor courses (47.0 to 25.2%). Students in female taught courses also had a significantly higher mean number of interactions per class sessions (2.69 to .91). These results support the previous findings that students participate more in female taught courses than in male taught courses. However, female instructors had the advantage of teaching smaller classes (23.8 to 39.9 students), a variable shown to significantly effect students’ interaction in previous studies (see, for example, Auster and MacRone, 1994). Female instructors also had a higher percentage of nontraditional students in their courses (26.0 to 6.3%). As Table 1 demonstrated, nontraditional students have a significantly higher mean number of interactions per class session. These differences in mean class size and percentage of nontraditional students may account for the differences in student participation in male and female instructors’ courses. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 21 Table 2: Interactions per class session (ANOVA Comparison of Means) and percent of students participating (Kendall’s tau) by student race and instructor gender Mean Interaction Per Session Mean Attend Mean No. Students Participat e Percent Students Participate Mean Interaction Per Student N Male Instructor (79.7%) 36.3 39.9 10.0 25.2*** .91*** 1117 Female Instructor (20.3%) 64.0 23.8 11.2 47.0*** 2.69*** 285 White Students Male Instructor (67.5%) 38.7 39.5 10.2 25.9 .98 947 Non-white Students Male Instructor (12.1%) 3.5 7.1 1.5 21.2 .49 170 White Students Female Instructor (17.0%) 51.9 19.8 8.8 44.5 2.62 238 Non-white Students Female Instructor (3.4%) 11.9 3.9 2.3 59.64 3.04 47 N 1779 415 1779 1402 *** Significant at the p < .001 level While white students in male instructors’ courses had a higher mean interaction per class session (.98 to .49), the difference was not statistically significant. The percentage of white and non-white students who participated in male instructors’ courses was very similar (25.9 to 21.2%). Interestingly, non-white students in female instructors’ courses had both a higher mean interaction per student (3.04 to 2.62) and a higher percentage of non-white students in female instructors’ courses participated relative to white students (59.6 to 44.5%). However, neither difference was statistically significant. Regardless of instructor gender, in our observations we could quickly identify which instructors’ classrooms included significant amounts of student discussion. When an instructor whose teaching style included significant use of discussion entered the classroom, informal interactions would begin immediately. Sometimes it was the instructor greeting and engaging individual students, but most often it was students initiating interactions with the instructor. Often the topic was related to course administration (e.g., “Do you have our papers graded yet?”), but just as frequently the conversation was unrelated to the course. These instructors had clearly built relationships with their students so that the students felt very comfortable interacting with them. Other instructors would enter the room and either be greeted with silence or would 4 Significant at p. < 0.07 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 22 face the challenge of gaining students’ attention when they were busy talking among themselves. This was particularly a problem in classes that consisted predominately of traditional college age students. For example, in a class that was 97 percent traditional students we observed the following. The instructor is having trouble getting students’ attention at start of class. He begins his lecture even though many students are still engaged in side conversations. The conversations at the rear of room continue very audibly even after class has clearly started. A student sitting in the front complains to the instructor that she cannot hear. The instructor chides those still chatting, “Your talking is preventing classmates from being able to hear.” The talking softens but doesn’t completely stop. Not only did the classes for these instructors not begin with productive student interaction, it was often 30 or more minutes into the session before the instructor first posed a question to the class. By this point, students appeared to have already gotten the, intended or unintended, message that their participation was neither needed nor desired. These late attempts to engage students in discussion were frequently met with silence from the students. Instructors would pause awkwardly for a second or two, answer their own question, and return to their lecture. We observed both male and female instructors whose classes resembled the above, but most frequently larger (30 or more students) male taught courses with very high percentages of traditional students were the classes with the least interaction. Table 3 presents a comparison of mean interactions per student and the percentage of students participating by student race and class type (regular session versus mass section). Despite the fact that the mass class section had a very interactive instructor whose efforts resulted in more interactions per class meeting session and a higher mean number of students participating, the mean interaction per student was significantly lower than that of the regular sessions (1.83 to .40) as would be expected. Likewise, the percentage of students participating in the regular sections was significantly higher than that of students in the mass section (37.3 to 17.8%). These results are simply a reality of mass sections. It would be difficult, if not impossible, for all 138 mass section students present to make even a single comment in a course meeting. When comparing white and non-white students by class type, we found no significant differences. White students in regular sections had a slightly higher mean interaction per student when compared with non-whites (1.89 to 1.50), but the percentage of students participating was nearly identical (37.6 to 36.2%). Whites and non-whites in the mass class section had identical mean interactions per student (.40) and a very similar percentage of whites and non-whites participated (17.3 to 20.0%). In sum, while we found significant differences in the participation by student age, student gender, instructor gender and class type. Significant differences by student race were relatively few. White non-traditional students had a higher mean number of interactions and a higher percentage of participation in discussion compared to non-white nontraditional students. However, reversing the trend, non-white female students participated at a slightly higher, but not significant rate than did white females. White students in courses taught by female instructors had a lower mean number of interactions and a lower percentage participated in discussion compared to non-white students in courses taught by female instructors. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 23 Table 3: Interactions per class session (ANOVA Comparison of Means) and percent of students participating (Kendall’s tau) by student race and class type (regular versus mass section) Mean Interaction Per Session Mean Attend Mean No. Students Participate Percent Students Participate Mean Interaction Per Student N Regular Section (60.6) 48.7 26.6 9.9 37.3*** 1.83*** 850 Mass Section (39.4%) 55.8 138.0 24.5 17.8*** .40*** 552 Regular Section Whites (85.1%) 42.7 22.6 8.5 37.6 1.89 723 Regular Section Non-whites (14.9%) 6.0 4.0 1.4 36.2 1.50 127 Mass Section Whites (83.7%) 46.2 115.5 20.0 17.3 .40 462 Mass Section Non-whites (16.3%) 9.0 22.5 4.5 20.0 .40 90 N 1779 415 1779 1402 *** Significant at the p < .001 level Table 4 presents the results of another way to test for differences in participation - linear regression of numerous variables on mean student interactions in order to more directly assess the impact of these variables. The first regression model (I) presents results of an analysis that includes the students enrolled in the mass section of Introduction to Sociology. Because the dynamics of interaction in the mass class may be significantly different from that in the regular sections, Table 4 also presents an analysis that excludes the students in the mass section (II). The adjusted R square for the first analysis (including the mass section) was only .146. Despite the lack of significance when we compared mean interactions by white and non-white students (see Table 1), we found that when we controlled for other variables in the regression analysis, being white has a significant positive affect on the mean number of student interactions per class session. Other variables with significant positive effects include student age (nontraditional = 1), percentage of non-traditional students in the class, and front third seating. The percent of non- white students in the class had a significant negative effect. This finding suggests that when more non-white students are in the class, students as a whole may be less willing to participate. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 24 Observations provided some further insight. As noted above, there were occasions when non-white students became the “experts” on a given topic in class and became the dominant talkers. Interestingly, this participation was not prompted by “triggers” (Weinstein and Obear 1992) which provoked defensive and emotional reactions from non-white students. In each case, the non-white students readily volunteered to share their experience with and knowledge of the topic. During one class session when the topic was racial stratification, four non-white students became the dominant talkers for that class session. However, some white students, through their body language, shaking of heads, sighs, and crossing of arms, seemed to disagree with the perspective of their non-white classmates, but did not verbally challenge what was being said. Thus it may be that when minority students speak up regarding controversial topics, white students, out of fear of appearing to be racist, stop participating. In an interview a white male instructor commented: When we get to the units having to do with race and ethnicity, I’ve noticed that most of the white students don’t want to hear it anymore. They really don’t. When I am talking about race, or homosexuality, they are mostly silent. If anyone is going to talk, it’ll be the African Americans. ….The white students usually back down on their views. I think that part is unfortunate. They pick up that I am a liberal. Therefore, my views on race are going to be closer to the African- American students’ views. So it would be silly to alienate your instructor. If that is true, it is a shame. The effect of student gender (female = 1), instructor gender (female = 1), and attendance in session were negative in direction, but not significant in model I. The negative direction of the effect of instructor gender suggests that the higher mean interactions and higher percentage of students participating in female taught courses (see Table 2) are more likely due to the smaller class size and the higher percentage of nontraditional students present than to instructor gender. The adjusted R square for the second analysis (excluding the mass class section) was .130. Being white had a significant positive impact on mean student interactions. Student age (non-traditional = 1) and percent nontraditional also had significant positive effects. Attendance in the session had a significant negative effect. As class size went up, the amount of student participation went down. The impact of percent non-white went from negative to positive and was not significant in the second model. Front third seating, while significant in model I, was not significant in model II. The effects of student gender and instructor gender remain negative but not significant. Percent female remains positive, but not significant. As noted above, there is a need for caution in examining mean interactions per student. As previous research has demonstrated (see for example, Howard and Baird 1998 and Karp and Yoels 1976) there are no mean students. There are only talkers (students who participate frequently) and non-talkers (students who only rarely participate in discussion). Therefore, it is important not only to examine mean interactions per student, but also to look at which students are most likely to become talkers. A. Talkers and Non-talkers The norm of the consolidation of responsibility was first identified by Karp and Yoels (1976). The consolidation of responsibility suggests that in the typical college classroom a small number of students will assume responsibility for speaking on behalf of the entire class. For the purposes of this study, we refer to these students as “talkers.” The rest of the students will Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 25 Table 4: Linear regression of mean student interactions by student gender, age, and race, percent female, percent non-white, percent non-traditional, instructor gender, attendance in session, and seating (I) including mass class and (II) excluding mass class. I II Includes Mass Section Excludes Mass Section Variable B Beta B Beta Student Race (White =1) .63* (.26) .062 1.08* (.43) .084 Student Gender (Female=1) -.12 (.21) -.015 -.16 (.29) -.018 Student Age (Non-traditional=1) 1.71*** (.34) .141 1.91*** (.49) .145 Percent Female .0275 (.02) .070 .00 (.02) .008 Percent Non-white -.06* (.02) -.067 .00 (.4) .001 Percent non- traditional .07*** (.01) .246 .03* (.02) .113 Instructor gender (Female=1) -.32 (.30) -.035 -.26 (.38) -.026 Attendance in session -.00 (.00) -.041 -.13*** (.04) -.173 Front Third Seating .99*** (.20) .125 1.24 (.31) .131 Constant -1.15 (1.32) 3.24 (2.29) Adjusted R Square .146 .130 N 849 1402 *** Significant at the p < .001 level * Significant at the p < at .05 level remain either non-contributors to class discussions or only occasional contributors. These students we label “non-talkers.” Table 5 presents a comparison of the percentage of students making two or more interactions per class session (talkers) and a comparison of the percentage of students making two or more interactions per class session by student race, gender and age. In the typical class meeting session, seven to eight students accepted the consolidation of responsibility and became talkers, accounting for 92 percent of all student interactions. We found no significant difference in the percentage of whites and non-whites who were talkers (19.7 to 18.4%). However, white talkers had a significantly higher mean number of interactions per class session than did non- white talkers (6.17 to 5.07). We also found that females were significantly more likely than males to be talkers (20.9 to 16.3%). However, male talkers had a slightly, but statistically significant, higher mean number of interactions per class session when compared to female 5 Significant at P < .08 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 26 talkers (6.14 to 5.93). When it comes to student age, non-traditional students were almost three times as likely as traditional students to be talkers (47.9 to 16.3%). Non-traditional talkers also had a significantly higher mean number of interactions per class session compared to traditional talkers (8.26 to 5.21). As Table 5 demonstrates, further comparisons by race and gender revealed few significant differences. A higher percentage of white males compared to non-white males were talkers (17.3 to 10.9%) and white male talkers had a higher mean number of interactions per class session (6.36 to 4.29), however, the differences were not statistically significant. Interestingly, a slightly higher percentage of non-white females were talkers compared to white females (21.6 to 20.8%). However, white female talkers had a somewhat higher mean number of interactions per class session (6.06 to 5.24), but neither difference was statistically significant. There were also no significant differences when comparing white traditional students with non-white traditional students by either percentage of students who were talkers or mean interactions per class session by talkers. A significantly higher percentage of white non- traditional students were talkers compared to non-white non-traditional students (53.5 to 34.9%). However, the difference in mean interactions per class session by white and non-white non- traditional students was not significant. Thus while statistically significant differences were few, the direction of the advantage most often favored white students over their non-white counterparts. Table 6 presents a comparison of mean interactions by students making two or more interactions per class session and the percentage of students who are talkers by student race and instructor gender. A significantly higher percentage of students in courses with female instructors were talkers compared with students in courses with male instructors (36.5 to 15.2%). These talkers in female taught courses also had a higher mean number of interactions per class session (7.08 to 5.31). A significantly higher percentage of white students in male taught courses were talkers compared to non-white students in male taught courses (16.3 to 9.4%) and the white talkers spoke up more often during the class period (5.44 to 4.00). The pattern differed in female taught courses. A significantly higher percentage of non-white students were talkers (51.1 to 33.6%). However, the white talkers in female taught courses had a higher mean number of interactions per class session (7.46 to 5.79). The findings suggest that non-white students are more likely than whites to become talkers in courses taught by female instructors, but the pattern is reversed in courses taught by male instructors with white students more likely to become talkers. Given that the courses taught by females were both smaller and had a higher percentage of nontraditional students, and given the small size of the sample of courses, it is not possible to determine whether these differences are due to instructor gender or the size and composition of the classes. B. Students Perceptions of Classroom Discussion Table 7 presents students’ self reports via survey of characteristics of talkers and non- takers. As was the case in previous research (reported above), students’ self reports of frequency of participation in class discussion exceeded what was observed. In their self reports, 75% of students reported contributing to discussion twice or more in the typical class meeting. Therefore, when reporting results from survey data, we chose to define talkers as those who reported making three or more contributions to discussion per session. The resulting percentage of students who were then defined as talkers (27.6%) was still higher than percentage of talkers observed (19.5%). Using survey data we were also able to make comparisons of talkers and non- Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 27 Table Five: Mean interactions by students making two or more interactions per class session (ANOVA Comparison of Means) and percent of students making two or more (Twoplus) interactions per class session (Kendall’s tau) by student race, student gender, and student age No. Students Making Two Plus Interactions % Students Making Two Plus Interactions Mean Interactions by Two Plus Students No. Interactions by Two Plus Students % all Interactions by Two Plus Students N All 7.61 19.5 5.98 45.5 92.1 274 Whites (84.5%) 6.50 19.7 6.17*** 39.9 92.5 234 Non-whites (15.5%) 1.11 18.4 5.07*** 5.6 89.4 40 Males (24.1%) 1.83 16.3* 6.14*** 11.3 91.6 66 Females (75.9%) 5.78 20.9* 5.93*** 34.3 92.2 208 Traditional (74.8%) 5.69 16.3*** 5.21*** 29.7 89.7 205 Nontraditional (25.2%) 1.92 47.9*** 8.26*** 16.3 96.9 69 White Males (24.3%) 1.64 17.3 6.36 10.4 92.1 59 Non-white Males (4.6%) .19 10.9 4.29 .8 85.7 7 White Females (60.2%) 4.86 20.8 6.06 29.4 92.6 175 Non-white Females (10.9%) .92 21.6 5.24 4.8 90.1 33 White Traditional (77.3%) 5.00 16.6 5.43 27.1 90.5 180 Non-white Traditional (12.4%) .69 14.4 3.64 2.5 82.0 25 White Non- traditional (7.2%) 1.50 53.5* 8.48 12.7 97.0 54 Non-white Nontraditional (3.1%) .42 34.9* 7.47 3.1 96.6 15 N 1638 274 *** Significant at the p < .001 level * Significant at the p < at .05 level Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 28 Table 6: Mean interactions by students making two or more interactions per class session (ANOVA Comparison of Means) and percent of students making two or more (Twoplus) interactions per class session (Kendall’s tau) by student race and instructor gender No. Students Making Two Plus Interactions Percent Students Making Two Plus Interactions Mean Interactions by Two Plus Students No. Interactions by Two Plus Students Percent all Interactions by Two Plus Students N All 7.61 19.5 5.98 45.5 92.1 274 Male Instructor (79.7%) 7.08 15.2*** 5.31*** 37.6 89.0 170 Female Instructor (20.3%) 8.67 36.5*** 7.08*** 61.3 96.1 104 White Students Male Instructor (67.5%) 6.42 16.3** 5.44*** 34.9 90.2 154 Non-white Students Male Instructor (12.1%) .67 9.4** 4.00*** 2.7 76.2 16 White Students Female Instructor (17.0%) 6.67 33.6* 7.46*** 49.8 95.8 80 Non-white Students Female Instructor (3.4%) 2.00 51.1* 5.79*** 11.6 97.2 24 N 1638 274 *** Significant at the p < .001 level ** Significant at the p < .01 level * Significant at the p < at .05 level talkers by their seating (front third versus back two thirds), class standing (freshmen and sophomores versus juniors and seniors), and by expected grades. Table 8 presents a comparison of students’ perceived responsibilities by race (whites versus non-whites) and level of participation (talkers versus non-talkers). Of the seven responsibilities at least 86 percent of all students agreed that six were part of their responsibility. There was a strong consensus that attending class, completing assigned tasks, studying for exams and quizzes, paying attention in class, learning the material and asking for help when needed were each a part of the students’ responsibilities. However, when it came to responsibility for participation in class discussion, less than 71 percent agreed this was part of the student’s responsibilities. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 29 Using an ANOVA comparison of means, students who reported they were talkers had a significantly higher mean age than self reported non-talkers (24.7 to 21.0 years). This is consistent with our observations. Non-traditional students were more likely to be talkers than traditional students (see Table 5). Using Kentall’s tau, we found no significant self reported differences in the percentage of students who were talkers between whites and non-whites, females and males, juniors and seniors versus freshmen and sophomores, nor by student grades. However, significantly more non-traditional students reported being talkers compared to traditional students (49.2 to 23.1%) and significantly more students in female taught courses reported being talkers when compared to students in male taught courses (34.9 to 24.3%). Finally, significantly more students seated in the front one third of the classroom reported being talkers than students in the back two thirds of the classroom (36.3 to 21.6%). These results are consistent with our observations, nontraditional students and students in female instructors’ courses were more likely to be talkers. Differences by instructor gender, however once again, must be interpreted with caution because female instructors taught smaller courses with more nontraditional students compared to the courses taught by male instructors. When we compared white and non-white students, we found no significant differences in students’ perceived responsibilities except for responsibility to “learn the material.” A significantly higher percentage of white students agreed this was a student responsibility than did non-white students (97.6 to 88.3%). Further examination revealed it was the non-talkers who differed in their responses on this responsibility. While white and non-white non-talkers had similar levels of agreement on five of the responsibilities, White non-talkers were significantly more likely than non-white non-talkers to agree students had a responsibility for learning the material (96.5 to 84.2%). Another difference was asking for help from the instructor when needed (85.3 to 71.1%). This difference would be significant at p < .08. A greater percentage of non-white talkers reported their agreement with responsibility for participation in classroom discussion (64.3 to 52.6%), but the difference was not statistically significant. These findings are difficult to interpret. Consistent with Ballard and Clanchy’s (1991) argument, we hypothesize that they may be due to prior experience in the educational system. Non-white students’ primary and secondary experience may have emphasized more rote and teacher-centered learning than that of white students. As such non-white students may have been socialized to take a more passive approach to learning, seeing the teacher as more responsible than the student for learning and ensuring that each student understood what was being taught. Non-whites may have also experienced a primary and secondary educational setting where teachers emphasized control and order in the classroom to a greater extent than they emphasized creativity and initiative in learning. Further research will be necessary to test this hypothesis. White and non-white talkers agreed on each of the student responsibilities except participation in classroom discussion. Interestingly, 100 percent of the small number of self- reported non-white talkers (N=16), indicated their agreement with responsibility for participation compared to almost 91 percent of white talkers. While this difference was statistically significant, both groups had a very high level of agreement. Table 9 presents a comparison of students’ reasons for participation in discussion by level of participation (talker versus non-talker) and race (white versus non-white). We found no significant differences in responses of whites and non-whites. Again, perhaps because of the small number of non-white non-talkers (N=38) and non-white talkers (N=16), statistically significant results were difficult to demonstrate. Reasons for participation by white and non- white non-talkers were very similar. The top reasons cited for participation by non-talkers were Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 30 Table 7: Survey characteristics of talkers and non-talkers (Kendall's tau - except where indicated) Characteristic Non-talkers (72.4%) Talkers (27.6%) N Mean Age (Oneway ANOVA) 21.0 24.7*** 347 Percentage of White Students (86.8%) 72.9 27.1 354 Percentage of Non-white Students (13.2%) 70.4 29.6 54 Percentage of Female Students (75.1%) 73.1 26.9 305 Percentage of Male Students (24.9%) 71.4 28.6 98 Percentage of Traditional Students (82.4%) 76.9 23.1*** 286 Percentage of Non-traditional Students (17.6%) 50.8 49.2*** 61 Percentage in Female taught courses (31.2%) 65.1 34.9* 129 Percentage in Male taught courses (68.8%) 75.7 24.3* 284 Percentage of Front Third Seating (41.5%) 63.7 36.3*** 171 Percentage of Back Two-thirds Seating (58.5%) 72.5 21.6*** 241 Percentage Junior/Senior (17.6%) 70.4 29.6 71 Percentage Frosh/Soph (82.4%) 72.6 27.4 332 Percentage Self-defined A student (25/4%) 66.7 33.36 96 Percentage Self-defined B student (51.9%) 70.9 29.1 196 Percentage Self-defined C student (20.1%) 78.9 21.1 76 N 298 114 412 *** Significant at the p < .001 level ** Significant at .01 6 Grade differences significant at .065 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 31 Table 8: Students' perceived responsibilities by race and level of participation and race (Kendall's tau) My responsibilities as a student include : (Circle all that apply) All White Non- white White Non- talker Non-white Non- talker White Talker Non- white Talker attend class 98.9 98.9 98.3 99.2 97.4 100 100 complete assigned tasks 98.6 98.8 96.7 98.8 94.7 100 100 study for exams/quizzes 97.3 97.6 95.0 96.9 92.1 100 100 pay attention in class 97.1 97.6 93.3 97.7 89.5 100 100 learn the material 96.4 97.6* 88.3* 96.5* 84.2* 100 93.8 ask for help from the instructor when I need it 86.5 87.8 80.0 85.3 71.17 95.8 93.8 participate in class discussion 70.7 72.1 66.7 64.3 52.6 90.6** 100** N 438 378 60 258 38 96 16 * significant at .05 ** significant at .01 “I have something to share,” “I need clarification,” “participation may help my grade,” and “My instructor creates a comfortable atmosphere by sharing about him/herself.” The least frequently cited reasons for participation in discussion by non-talkers were “It is required,” “If I don’t, no one else will,” and “I disagree with something the instructor said.” Non-white non-talkers more frequently cited “It makes the class more interesting” as a reason for participation (31.6 to 24.8%), but the difference was not statistically significant Among talkers, again there were no statistically significant differences in reasons cited for participation in classroom discussion. However, non-white talkers cited “I need clarification” more often than did white talkers (75.0 to 54.2%), a difference that would be significant at p < .1. Non-white talkers also more frequently stated “I learn more when I participate” than did white talkers (81.3 to 61.5%), a difference that would be significant at p < .09. But the larger picture is one of agreement between white and non-white talkers on their reasons for participation in discussion. Table 11 presents reasons why students choose not to participate by their level of participation (talker versus non-talker) and race (white versus non-white). We again found no significant differences by race. Whites and non-whites were very similar in the degree to which they cited the four top reasons “I am shy,” “the feeling that I don’t know enough about the subject matter,” “I have nothing to contribute,” and “my ideas are not well enough formulated.” While the differences were not statistically significant, non-whites more frequently indicated they did not participate because “Of the chance I would appear unintelligent to other students” (28.3 to 21.2%), “of the chance I would appear unintelligent to the instructor” (25.0 to 18.5%), and “I have not completed the assigned tasks” (26.7 to 17.2%). The only significant difference between white non-talkers and non-white non-talkers or white talkers and non-white talkers was that among talkers whites were more likely to avoid participation because of the perception that the instructor does not want participation or discussion (7.3 to 0.0%). 7 Significant at p < .08 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 32 Table 10: Reasons for participation in discussion by level of participation and race (Kendall's tau) In this class, I participate in discussion because: (Circle all that apply) All White Non-white White Non- talker Non-white Non- talker White Talker Non-white Talker I have something to share 52.5 51.2 58.3 44.0 44.7 75.0 87.5 I need clarification 46.0 45.0 53.3 42.6 47.4 54.2 75.08 participation may help my grade 40.6 41.3 36.7 34.1 28.9 63.5 56.3 My instructor creates a comfortable atmosphere by sharing about him/herself 38.4 37.8 43.3 31.0 34.2 62.5 68.8 I learn more when I participate 37.2 35.7 45.0 25.6 28.9 61.5 81.39 It makes the class more interesting 34.8 34.4 35.0 24.8 31.6 62.5 43.8 I disagree with something another student said 25.1 25.1 23.3 19.4 18.4 42.7 37.5 I am familiar and comfortable with my classmates 23.3 23.8 20.0 19.8 13.2 36.5 37.5 the instructor calls one me 21.9 22.5 18.3 24.4 18.4 20.8 25.0 I don't participate in discussion 18.3 24.4 23.7 2.1 6.3 I disagree with something the instructor said 15.8 15.3 16.7 12.4 7.9 25.0 37.5 I am trying to help other students 12.6 12.7 11.7 9.3 7.9 25.0 25.0 if I don't, no one else will 12.0 12.7 8.5 12.0 8.1 17.7 12.5 It is required 10.2 10.6 8.3 10.9 10.5 11.5 6.3 N 408 378 60 258 38 96 16 8 Significant at P <, .10 9 Significant at P < .09 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 33 Table 11. Reasons why students choose not to participate by level of participation and race (Kendall's tau) In this class, when I choose NOT to participate in discussion I do so because: (Circle all that apply) All Whites Non- whites White Non-talker Non-white Non-talker White Talker Non-white Talker I am shy 42.0 43.4 35.0 50.8 47.4 24.0 12.5 of the feeling that I don't know enough about the subject matter 33.0 34.7 25.0 35.7 23.7 33.3 18.8 I have nothing to contribute 29.6 30.2 28.3 30.4 26.3 33.3 31.3 my ideas are not well enough formulated 22.9 22.8 25.0 22.9 21.1 22.1 25.0 someone else will participate therefore I don't need to. 22.8 23.3 20.0 28.3 18.4 13.5 31.3 of the chance I would appear unintelligent to other students 21.9 21.2 28.3 26.0 26.3 12.5 25.0 of the chance I would appear unintelligent to the instructor 19.2 18.5 25.0 22.9 26.3 7.3 12.5 I have not completed the assigned tasks (I am not prepared for class) 18.5 17.2 26.7 16.3 23.7 19.8 31.3 the class is too large 14.7 14.3 18.3 15.9 21.1 7.3 18.8 the course is not interesting to me 7.7 8.0 6.7 9.7 5.3 4.2 0.0 the instructor does not want participation or discussion 4.5 4.5 5.0 3.5 5.3 7.3* 0.0* of the possibility class may end early if no one participates 3.8 4.0 1.7 4.7 2.6 3.2 0.0 N 408 378 60 258 38 96 16 Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 34 IV. Conclusion While Antonio, et al’s (2004) work has demonstrated the benefits of multi-racial group discussion for white students, our research raises concerns about the participation of non-white students. The conclusions of this preliminary investigation, of course, are limited because it was conducted at a single university with non-white enrollment of approximately 15 percent and because the sample included only introductory sociology courses. Clearly further research at a variety of institutions with a range of non-white enrollments and including a variety of disciplines is needed. While the results of this case study are mixed with regard to the impact of race on participation in classroom discussion, there is evidence that presents cause for concern. Based on our regression analysis, white students are likely to participate at a significantly higher rate than non-white students. While there are occasions when the non-white students become the dominant participants in discussion (e.g., when discussing topics related to race), we also found evidence that white students may choose to disengage from these very discussions. Likewise, we found some evidence that an increase in the percentage of non-white students may decrease overall participation. These preliminary findings require further investigation of the type described above. A major area for further examination is the impact of race on the participation of various minority groups. For example, we need to ask, do the interaction patterns of Asians, for example, differ significantly from those of African Americans or Hispanics? Do white students tend to withdraw from discussion only when large numbers of particular minority groups are enrolled in the course? Or are they likely to be silent regardless of which minority group is represented? We also need to further investigate the topics which spark the participation of non- white students in class and determine how to encourage their interaction without closing the door to discussion and debate among students. Does the percentage of minority students matter in this regard? Are the classroom interaction dynamics different on a campus with 50 percent minority enrollment versus 15 percent minority enrollment? Our study also failed to find a difference in the reasons why whites and non-whites chose to participate or not participate. Survey studies will need to directly address the attitudes and behaviors of whites and non-whites with regard to one another as well as the usual reasons given for participation and non-participation. Finally, we need to address the role of instructor gender. Studies that include a larger number of courses with a larger number of instructors are necessary before any firm conclusions can be drawn with regard to the relationship between instructor gender and race. Given the increasing attention that is being paid to race in higher education by academics and non-academics, continued study of the actual experiences of minority students within higher education is clearly warranted. As Dedlacek (1983) suggested, different teaching methods may be necessary to facilitate the success of minority students. Thus in order to be able to see the world from the viewpoint of minority students, as Wu and Morimoto (1983) argue is necessary, investigations at a wide range of campuses with a wide range of minority students will be necessary to capture the experiences of minority students in American higher education. References Adams, Maurianne (1992). “Cultural Inclusion in the American College Classroom.” New Directions for Teaching and Learning, Vol. 49, pp. 5-18. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 35 Aitken, Joan E., & Neer, Michael R. (1993). “College Student Question-Asking: The Relationship of Classroom Communication Apprehension and Motivation.” Southern Communication Journal, Vol. 59, pp. 73-81. Antonio, Antonio, Chang, Mitchell J., Hakuta, Kenji, Kenny, David A., Levin, Shana, & and Milem, Jeffrey F. (2004). “Effects of Racial Diversity on Complex Thinking in College Students.” Psychological Science, Vol. 15, No. 8, pp. 507-510. Astin, Alexander W. (1985). “Achieving Educational Excellence.” San Francisco: Jossey- Bass. Auster, Carol J. & MacRone, Mindy (1994). “The Classroom as a Negotiated Social Setting: An Empirical Study of the Effects of Faculty Members’ Behavior on Students’ Participation.” Teaching Sociology, Vol. 22, pp. 289-300. Ballard, Brigid & Clanchy, John (1991). “Teaching Students from Overseas.” Cheshire, Melbourne: Longman. Boersma, P. D., Gay, D., Jones, R.A., Morrison, L., & Remick, H. (1981). “Sex Differences in College Student-Teacher Interactions: Fact or Fantasy?” Sex Roles, Vol. 7, pp. 775-84. Brooks, V. (1982). “Sex Differences in Student Dominance Behavior in Female and Male Professors’ Classrooms.” Sex Roles, Vol. 8, pp. 683-90. Cabrera, Alberto F., Crissman, Jennifer L., Bernal, Elena M., Nora, Amaury, Terenzini, Patrick T., & Pascarella, Ernest T. (2002). “Collaborative Learning: Its Impact on College Students’ Development and Diversity.” Journal of College Student Development, Vol. 43, No. 1, pp. 20- 34. Cabrera, Alberto F., Nora, Amaury, Terenzini, Patrick T., Pascarella, Ernest T., & Hagedorn, L. (1999). “Campus Racial Climate and the Adjustment of Students to College: A Comparison Between White Students and African-American Students.” Journal of Higher Education, Vol. 70, No. 2, pp. 134-60. Canada, Katherine & Pringle, Richard (1995). “The Role of Gender in College Classroom Interactions: A Social Context Approach.” Sociology of Education, Vol. 68, pp.161-186. Constantinople, Ann, Cornelius, Randolph, & Gray, Janet M. (1988). “The Chilly Climate: Fact or Artifact?” The Journal of Higher Education, Vol. 59, pp. 527-50. Cornelius, Randolph, Gray, Janet M. & Constantinople, Ann (1990). “Student Faculty Interaction in the College Classroom.” Journal of Research and Development in Education, Vol. 23, pp. 189-97. Crawford, Mary, & MacLeod, Margo (1990). “Gender in the College Classroom: An Assessment of the “Chilly Climate” for Women.” Sex Roles, Vo. 23, pp. 101-22. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 36 Crombie, Gail, Pyke, Sandra W., Silverthorn, Naida, Jones, Alison, & Piccinin, Sergio (2003). “Students’ Perceptions of Their Classroom Participation and Instructor as Function of Gender and Context.” The Journal of Higher Education, Vol. 74, pp. 51-76. Dedlacek, William E. (1983). “Teaching Minority Students.” New Directions for Teaching and Learning, Vol. 16, pp.39-50. Fassinger, Polly A. (1995). “Understanding Classroom Interaction: Students’ and Professors’ Contributions to Students’ Silence.” The Journal of Higher Education, Vol. 66, pp. 82-96. Faust, Debra F., & Courtenay, Bradley C. (2002). “Interaction in the Intergenerational Freshman Class: What Matters?” Educational Gerontology, Vol. 28, pp. 401-422. Feagin, Joe R., Hernan, Vera, and Imani, Nikitah. (1996). “The Agony of Education: Black Students at White Colleges and Universities.” New York: Routledge. Feagin, Joe R. (2003). “Race and Its Continuing Significance on Our Campuses: An Interview with Dr. Joe R. Feagin.” Black Issues in Higher Education, Vol. 19, No. 24, pp. 24-27. Fritschner, Linda M. (2000). “Inside the Undergraduate College Classroom: Faculty and Students Differ on the Meaning of Student Participation.” The Journal of Higher Education, Vol. 71, pp. 342-62. Garside, Colleen (1996). “Look Who's Talking: A Comparison of Lecture and Group Discussion Teaching Strategies in Developing Critical Thinking Skills.” Communication Education, Vol. 45, pp. 212-27. Hall, Roberta M. & Sandler, Bernice R. (1982). “The Classroom Climate: A Chilly One for Women?” Project on the Status and Education of Women. Washington, D.C.: Association of American Colleges. Hardiman, Rita & Jackson, Bailey W. (1992). “Racial Identity Development: Understanding Racial Dynamics in College Classrooms and on Campus.” New Directions for Teaching and Learning, Vol. 52, pp.21-38. Heller, Jack F., Puff, C. Richard, & Mills, Carol J. (1985). “Assessment of the Chilly College Climate for Women.” Journal of Higher Education, Vol. 56, pp. 446-461. Howard, Jay R., & Baird, Roberta (2000). “The Consolidation of Responsibility and Students’ Definitions of the College Classroom.” The Journal of Higher Education, Vol. 71, pp. 700-721. Howard, Jay R., & Henney, Amanda L. (1998). “Student Participation and Instructor Gender in the Mixed Age College Classroom.” The Journal of Higher Education, Vol. 69, pp. 384-405. Howard, Jay R., Short, Lillard B. & Clark, Susan M. (1996). “Student Participation in the Mixed-Age College Classroom.” Teaching Sociology, Vol. 24, pp. 8-24. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 37 Howard, Jay R., James, George, & Taylor, David R. (2002). “The Consolidation of Responsibility in the Mixed-Age College Classroom.” Teaching Sociology, Vol. 30, pp. 214- 234. Johnson, David W., Johnson, Roger T., & Smith, Karl A. (1991). “Cooperative Learning: Increasing College Faculty Instructional Productivity.” ASHE_ERIC Higher Education Report No. 4. Washington, D.C.: George Washington University. Jung, Jan, Moore, Rosemary L., & Parker, Jim (1999). “A Study of Older Students’ Participation in the College Classroom.” Unpublished. Karp, David A., & Yoels, William C. (1976). “The College Classroom: Some Observation on the Meaning of Student Participation.” Sociology and Social Research, Vol. 60, pp. 421-39. Kember, David, & Gow, Lyn (1994). “Orientations to Teaching and Their Effect on the Quality of Student Learning.” Journal of Higher Education, Vol. 65, pp. 58-74. McKeachie, Wilbert J. (1990). “Research on College Teaching: The Historical Background.” Journal of Educational Psychology, Vol. 82, pp. 189-200. Neer, Michael R., & Kircher, W. Faye (1989). “Apprehensives’ Perceptions of Classroom Factors Influencing Their Class Participation.” Communication Research Reports, Vol. 6, pp. 70-77. Nunn, Claudia E. (1996). “Discussion in the College Classroom: Triangulating Observational and Survey Results.” The Journal of Higher Education, Vol. 67, pp. 243-66. O’Keefe, Tim F., & Faupel, Charles E. (1987). “The Other Face of the Classroom: Student Ethnography.” Sociological Spectrum, Vol. 7, pp. 141-155. Pascarella, Ernest T., Wolniak, Gregory C., & Pierson, Christopher T. (2003). “Influences on Community College Students’ Educational Plans.” Research in Higher Education, Vol. 44, No. 3, pp. 301-314. Pearson, Judy, & West, Richard (1991). “An Initial Investigation of the Effects of Gender on Student Questions in the Classroom: Developing a Descriptive Base.” Communication Education, Vol. 40, pp. 22-32. Schwitzer, Alan M., Ancis, Julie R., & Griffin, Oris T. (1999). “Validating a Proposed Model of African-American Students’ Social Adjustment.” Journal of the First-Year Experience and Students in Transition, Vol. 11, No. 1, pp. 77-102. Smith, Daryl G. (1977). “College Classroom Interactions and Critical Thinking.” Journal of Educational Psychology, Vol. 69, pp.180-90. Statham, A., Richardson, L., & Cook, J.A. (1991). “Gender and University Teaching: A Negotiated Difference.” Albany: SUNY. Howard, J., Zoeller, A., and Pratt, Y. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 38 Saufley, Ronald W., Cowan, Katherine O. & Blake, J. Herman (1983). “The Struggles of Minority Students at Predominately White Institutions,” New Directions for Teaching and Learning, Vol. 16, pp. 3-15. Sternglanz, Sarah Hall, & Lyberger-Ficek, Shirley. (1977). “Sex Differences in Student-Teacher Interactions in the College Classroom.” Sex Roles, Vol. 3, pp. 345-52. Terenzini, Patrick T., Cabrera, Alberto F., Colbeck, Carol L., Bjorklund, Stefani A., & Parente, John M. (2001). “Racial and Ethnic Diversity in the Classroom: Does It Promote Student Learning?” Journal of Higher Education, Vol. 72, No. 5, pp. 509-531. Terenzini, Patrick T., Yaeger, Patricia M., Bohr, Louise, Pascarella, Ernest T., & Nora, Amaury (1997). “African American College Students’ Experiences in HBCUs and PWIs and Learning Outcomes.” Paper presented at the Annual Meeting of the Association for Institutional Research (Orlando, FL, May, 1997). Tinto, Vincent (1975). “Dropout from Higher Education: A Theoretical Synthesis of Recent Research.” Review of Educational Research, Vol. 45, pp. 89-125. Tinto, Vincent (1997). “Classrooms as Communities: Exploring the Educational Character of Student Persistence.” The Journal of Higher Education, Vol. 68, pp. 599-623. Weast, Don (1996). “Alternative Strategies: The Case for Critical Thinking.” Teaching Sociology, Vol. 24, pp. 189-194. Weinstein, Gerald & Obear, Kathy (1992). “Bias Issues in the Classroom: Encounters with the Teaching Self.” New Directions in Teaching and Learning, Vol. 52, pp. 39-50. Wu, Jean & Morimoto, Kiyo (1983). “On Careful Listening.” New Directions for Teaching and Learning, Vol. 16, pp. 51-57. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006, pp. 39 - 55. Metacognitive Knowledge Monitoring and Self-Regulated Learning: Academic Success and Reflections on Learning Randy M. Isaacson and Frank Fujita1 Abstract: During the past decade the relationship of self-regulated learning (SRL) to academic success has been extensively explored but the impact of metacognition in this process has not been thoroughly examined. This study examined the relationship of metacognitive knowledge monitoring (MKM) to classroom performance. Eighty-four undergraduate students in an introductory educational psychology class completed ten weekly in-class tests in which they were allowed to choose test questions. Students were asked to identify the number of hours they studied, their level of confidence, and to predict their test results after completing the test but before it was graded. High achieving students were: more accurate at predicting their test results; more realistic in their goals; more likely to adjust their confidence in-line with their test results; and more effective in choosing test questions to which they knew the answers. The study supports the relationship of metacognitive knowledge monitoring to self-regulated learning and academic success. Keywords: self-regulated learning, metacognitive knowledge monitoring I. Introduction. The application of self-regulation to learning is a complicated process involving not only the awareness and application of learning strategies but also extensive reflection and self- awareness. Pintrich (1995) describes self-regulation as the “active, goal-directed self-control of behavior, motivation, and cognition for academic tasks by an individual student.” (p. 5) Students who are skillful at academic self-regulation understand their strengths and weaknesses as learners as well as the demands of the specific tasks. They approach learning with an assortment of strategies they might apply to achieve their goals and an understanding of when and how to implement their plan. But students who are expert learners have more than an arsenal of study strategies and the ability to regulate academic resources, they also know when they have mastered, or not mastered, the required academic tasks. That is, expert learners are also skillful at metacognitive knowledge monitoring (MKM). Self-regulated learners are skillful at monitoring their learning and comprehension which has a direct effect on each step in the self-regulation process. Pintrich et al. (2000) compares monitoring to the thermostat of a furnace. When the temperature falls below a specified level the thermostat tells the furnace to turn on the heat; when a learner is confused or does not comprehend what they are studying the monitor tells the learner to regulate their behavior, cognitive strategies, or motivation and affect to increase learning. To be effective learners, students must adjust their efforts based on their awareness of their own understanding and the level of difficulty of the upcoming task. One of the critical barriers to success for many students may be their inability to objectively assess their mastery of the academic tasks they are facing. Accurate monitoring of learning can impact self-regulation throughout the learning 1 Respectively, Professor of Education, Indiana University South Bend, risaacso@iusb.edu and Associate Professor of Psychology, Indiana University South Bend, ffujita@iusb.edu. Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 40 process. Zimmerman (1998) proposes three phases to self-regulation which incorporate metacognition. The first phase is forethought which includes goal setting, strategic planning, and self-efficacy; students identify their goals, their plans for achieving them, and consider how likely it is they will achieve their goals. The second phase is performance or volitional control which includes attention focusing, self-instruction, and self-monitoring; students attempt the learning tasks and monitor what they are learning. The third phase is self-reflection which focuses on comparing self-monitored information with a standard or goal and reactions to the results. During the reflection stage students assess their success or failure, modify their self- efficacy, make causal attribution, and adapt for future learning. In all three of these phases students are using academic goals as the yardstick against which they assess their learning; using the thermostat metaphor, goals are the set-temperature the thermostat uses to judge whether to turn-on the furnace. In each phase, self-reflection and self- monitoring are critical to master the skills of self-regulation as well as the content being learned. In each phase students who are skillful at self-regulation are cognizant of their understanding and adjust their goals and self-efficacy based on internal as well as external feedback on their mastery of the tasks. Students’ ability to monitor their learning is one of the key building blocks in self- regulated learning; students who are aware of the level of their mastery of material can adjust their study time and strategies. Over the past decade a number of concepts have been used to describe students’ awareness of their learning. These concepts were originally investigated at a micro-level focusing on metamemory using an experimental format. Ease-of-Learning (EOL) judgements refer to a student’s inferences about how easy or difficult a task will be to learn (Nelson & Narens, 1990; Nelson & Leonesio, 1988). EOL judgments occur in advance of actual learning, but clearly are a stumbling block for students who underestimate the difficulty of the material or the level at which it must be mastered. Judgments of learning (JOL) occur during or after learning and are predictions of future test performance (Kelemen, 2000; Koriat, 1997; Nelson & Narens, 1990). Students whose JOL overestimate their actual learning are also likely to terminate their studying prior to mastering the material and fall short of their goals, especially when the demands of the tasks exceed the levels at which they have learned the required material. These assessments of learning have important implications for self-regulated learning as each influence the thermostat that adjusts how much time and effort a student devotes to studying. But the primary focus of the research on EOL and JOL has been on metamemory with very little attention to the higher level of concepts and problem solving which occurs in classrooms, particularly in postsecondary classrooms. In these studies metamemory is not a measure of understanding or comprehension, but rather the ability to retrieve information from long term memory. While obviously not irrelevant to classroom learning, these measures are primarily applicable to the most basic learning which occurs in school. The connection to the self-regulation of cognitive strategies is probably most suitable to rehearsal or the most rudimentary forms of elaboration. Skillful self-regulators should be able to go beyond the assessment of their ability to recall facts; effective self-regulators should be able to estimate how well they have mastered a body of knowledge and how well they will be able to demonstrate their mastery. Skillful self-regulators should be able to predict how well or poorly they will do, and have done, on a test while naive self-regulators should be less able to estimate their academic success. A series of studies by Maki and colleagues has shown the significance of metacognition Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 41 on the comprehension of college students in experimental situations. In a study by Maki and Berry (1984) students who scored above the median on multiple choice items of text showed more accuracy in their test prediction than students who scored below the median. Students who had done well on the test were also better at predicting their test scores: greater metacomprehension could be inferred to influence self-regulation. Students who were asked to process information at high levels were also found to be more accurate in their test predictions (Maki, et.al., 1990) as were students who had increasingly more information about the text (Maki and Serra, 1992). Maki invited the connection between metamemory, metacomprehension, and classroom learning when she found an experimental relationship between higher order thinking questions and prediction accuracy. Studies of metamemory had used simple recall questioning, but Maki (1995) found that there was greater accuracy of predictions when students were asked higher order questions that did not require verbatim recall of information. This was in support of the finding of Weaver (1990) who had proposed that multiple questions had increased the reliability and calibration of comprehension assessment. These finding have important implications for post-secondary education which places greater value on higher level thinking skills. Do successful college students have better metacognitive awareness on tasks that require higher level thinking skills? Can they adjust their study skills in response to the increasing demand of these tasks? The transition from high school to college puts many demands on young adults. In the classroom the greatest challenge may be the move from the declarative knowledge emphasized in high school to the higher level thinking skills typically required in college. When students do not recognize that these new demands require new approaches to learning and studying they may be unwilling and/or unable to make the necessary changes. At the core of this problem may be that students do not realize that their learning does not match the demands of the task. They assume they have learned the material if they can recall the important terms. They do not recognize that different academic tasks (e.g., different test formats) demand different levels of learning. They overestimate their understanding because they do not recognize the implication of different levels of learning and varying levels of task difficulty. To be successful in college, students may need to have a variety of SRL and metacognitive skills that were not necessarily essential in high school. First, successful college students recognize that professors expect more than the memorization of declarative knowledge. Second, successful college students use accurate MKM while studying to assess their mastery of the required material, particularly in relation to what will be require on the performance task (e.g., test). Third, successful college students have an arsenal of SRL strategies they can choose from to match their level of learning to the demands of the performance task. And finally, successful students are able to self-monitor their understanding and the demands of the performance task during evaluation to adjust their demonstration of the learning (e.g., to choose the right questions to answer during a test.) While it may be very difficult to assess metacognition before or during learning, it is possible to assess MKM during the assessment of their learning. While taking a test are expert students better at identifying what they know and what they do not know? For the past decade a program of research by Tobias and Everson (2000, 2002) has examined learners’ ability to differentiate between what they know and do not know. Their findings indicate learners of all levels of ability and developmental stages are affected by their ability to monitor their learning. In dozens of studies with students of all ages and abilities, Tobias and Everson have found that students who are able to differentiate between when they know and when they do not know are more likely to excel than students who are not able to Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 42 distinguish their level of comprehension. The studies by Tobias and Everson have focused on the correlation between knowledge monitoring and student’s academic performance. But very little evidence exists which explores the relationship between knowledge monitoring and academic choices within classes: How do students who possess effective knowledge monitoring skills use these skills to make decisions which impact their academic success? These experimental studies of metamemory and metacomprehension are an open invitation to naturalistic studies performed in actual classrooms. Studies on EOL (Nelson & Narens, 1990) suggest that students who underestimate the difficulty of classroom content may abandon their learning efforts before they have mastered the material. Experimental studies on JOL (Kelemen, 2000; Koriat, 1997) indicate that students overestimate their understanding which can lead to discontinuing learning efforts prior to mastery. Schommer & Surber (1986) demonstrated an Illusion of Knowing (IK) when students believe that only shallow processing is necessary when the material is actually difficult, which could lead some students to underestimate the necessity of learning classroom content to the degree which will be required for success. Each of these issues have clear implications for self-regulation in the classroom context. When students in post-secondary education are presented with learning tasks that require higher level thinking they must accurately judge the difficulty of material to be learned, accurately judge the level of their own understanding, and accurately judge the requirement of the performance task (e.g., test) they will be given in class. While laboratory predictions of test performance are revealing of student’s self- monitoring there is little classroom research on this topic. In the first study of classroom confidence, Shaughnessy (1979) reported high achieving students as being better able to distinguish between known and unknown information. In a study of self-efficacy, Sinkavich (1995) reports a significant difference between high and low achieving students on their confidence on individual test items. Hacker et al. (2000) report similar findings; high performing students were accurate in predicting their test results with their accuracy improving over multiple exams, while low performing students were poor at predicting test results. Hacker et al. (2000) found significant differences between low performing students and high performing students in relation to their ability to predict (before taking a test) and postdict (after taking a test but before receiving their grade) their test results. The implication being that students who were doing poorly in a college course were unable to monitor their knowledge of the course material (i.e., they overestimate their test results in both prediction and postdiction) and therefore were unable to regulate their studying to assure mastery of the course material. The student sample in the Hacker et al. study were college students in an undergraduate educational psychology class in which the students took three tests. On each test the highest achieving students on that test were more accurate in their predictions and postdictions, but the analysis did not focus on the overall achievement of students across the semester. Would the highest achieving students for the semester be more accurate in their postdictions and would they be more likely to accurately adjust their test postdictions on a test-by-test basis? The present study examines students across ten tests during a one semester undergraduate educational psychology course comparing intra-individual differences for low and high preforming students. All three levels of Zimmerman’s (1998) academic self-regulation learning cycle emphasize the importance of goals setting. In the forethought stage students set goals, in the performance and volitional control stage students monitor their learning in relation to their goals, and in the reflection stage students assess their success or failure in relation to their goals. Extensive research has focused on goal setting (Locke & Latham, 1990) but no classroom Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 43 research to date has examined the impact of goal setting and goal achievement on self-regulation. The present study will explore the relationship among the variables of goal setting, self- monitoring, performance, and self-regulation. In the performance and volitional control stage of Zimmerman’s (1998) learning cycle students are attempting the learning task and monitoring their mastery of the task. In this cycle students who possess good MKM should be able to reflect on the application of their learning to the task and choose appropriate tasks. When given choices of tasks of varying difficulty, such as a selection of diverse test questions, students with good metacognitive awareness should be capable of choosing tasks in which they will succeed and avoid tasks in which they are less likely to succeed. The present study will examine the ability of students to use MKM to choose between tasks of varying difficulty. The self-reflective stage in Zimmerman’s (1998) learning cycle examines the outcomes when students compare self-monitored information to their goals, especially in relation to the impact on self-evaluation, self-efficacy, and adaptation of strategies. The combination of goals with knowledge of performance impacts self-efficacy which heightens motivation (Bandura, 1997). Skillful self-regulators effectively monitor their progress in relation to their goals and then adjust their self-efficacy and future strategies. The present study will begin to explore how the feedback students receive from completing tasks of varying difficulty (e.g., tests) impacts their self-efficacy as they progress through a college course. This study will explore the metacognitive differences between high achieving students and low achieving students. High achieving students, who have good MKM, should be more reflective and thoughtful about decisions they make in their studying, test taking, and self- efficacy. Are students who excel across the semester more likely to be more accurate in their estimation of their individual test grades and do they accurately adjust their estimations based on their mastery of the material? Do students who excel have goals that are more consistent with their academic performance? Do students who excel have realistic self-efficacy and how does that self-efficacy change over the course of a semester? Are students who excel more likely to make accurate task choices based on their understanding of the required material. II. Method A. Participants The participants were 84 undergraduate college students (59 females, 25 males) enrolled in an introductory educational psychology course on a commuter campus of a mid-western university. All students were university students enrolled as education majors. B. Procedures Participants took weekly objective tests (true-false and multiple choice) and completed a questionnaire for each test. Part of the questionnaire was completed before the participants took the test and part immediately after taking the test but before scoring the test. Prior to taking the test each student was asked to report the number of hours they had studied, how many points they would have to achieve to be satisfied with their performance (satisfaction goal), how many points they would have to achieve to be proud of their performance (pride goal), and how confident they were about achieving their satisfaction goal (pre-test self-efficacy). After completing the test, but before it was graded, each student was asked to identify Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 44 how many points they believed they would achieve on the test2 and how confident they now were about their achieving their satisfaction goal (post-test self-efficacy). Then, tests were graded and were returned to the student for review before the student was dismissed from class. Each of the weekly tests included 40 objective test questions: 18 lower level test questions which emphasized knowledge and comprehension and were worth 1 point each; 18 moderately difficult questions which emphasized application and were worth 2 points each; and 4 difficult test questions which emphasized analysis and synthesis and were worth 3 points each. For each weekly test, students were allowed to answer only 30 of the 40 test questions; their grade being dependant on both the accuracy of their answers (number of questions correct) and the type of test questions they chose and answered correctly. To earn an A in the class, students had to choose more difficult test questions (worth 2 or 3 points) and get them correct. To earn a lower grade, students could either take more difficult questions and get a lower percentage correct, or take less difficult test questions (worth 1 or 2 points) and get a higher percentage correct. Therefore, the key to success in the course was not only correctly answering test questions, but also choosing the test questions you could answer correctly. The weekly tests were designed to reveal and substantiate student metacognitive awareness during testing. Ten tests were administered during the semester (approximately one per week). III. Results This study examines the differences between high and low achieving students on a number of metacognitive variables: What are the long-term changes between their estimations of test grades early to late in the semester?; How are their satisfaction and pride goals different from their actual performance?; How does their self-efficacy change and how does that compare to their actual test score?; and, Are students able to make academic choices based on their MKM? A. Test Scores Postdiction Following the format used by Hacker (2000) we examined the hypothesis that high performing students would be more accurate in predicting their test scores than low achieving students. Using Hacker’s terminology, our participants made “postdictions” because they took the test before estimating their score on the test. For each of the ten tests, a correlation was computed between the test score and the squared error of the student’s postdiction [(test score – expected test score) 2]: let us call this type of correlation the matched-score format. Students who have good MKM should be better at postdicting their test scores which would result in smaller squared error scores and a negative correlation between test scores and squared error scores in the matched-score format. The average (median) of these ten correlations was -.26 (all listed correlations are significant at p < .05 unless noted otherwise). For the three most difficult tests, the correlations were -.27, -.49, and -.65. Thus, the students who were most accurate in their postdictions (having a low squared error of postdiction) tended to have higher test scores and this was particularly true on the most difficult tests (see Table 1). From the previous analysis (matched-score format) it is possible that all students are 2 Hacker (2000) makes the distinction between predictions and postdictions. Predictions are a student’s estimates of their test scores before they take a test. Postdictions are a student’s estimates of their test scores after they have taken the test but before their tests are graded. In the present study the students estimated their results after having taken the test but before the test is graded. The students were not asked to make predictions before taking the test. Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 45 equally good (or poor) at postdicting their test scores, they all postdict that they will do well, and that on any particular test a few students do poorly at random. To insure that it was the students who consistently performed well on their tests who were making the best postdictions, we also computed correlations between the total of the ten test scores (total points) and each of the ten squared error of postdictions. For example, for the first correlation, we took the squared error of postdiction for Test 1, and correlated it with the student’s total points for the semester. Let us call this type of correlation the total score format. This calculation was done for each of the ten tests. The median of these ten correlations was -0.18. For the three most difficult tests, the correlations were -0.27, -0.24, and -0.53. Thus, the students who have the highest achievement across the semester are better at postdicting test results, and their postdiction accuracy is most pronounced on the most difficult tests. This confirms the findings by Hacker (2000) and others (Maki and Berry, 1984; Maki, et.al., 1990; Maki and Serra, 1992) that demonstrates that high performing students are better at metacognitive awareness, knowledge monitoring, and calibrating how they will do on tests in college. Are there also differences in the goals students set and the changes that occur over time during a semester? Table 1: Correlations Between Test Score and the Squared Error of Predicted Test Score Matched-Score Format Total-Score Format Chapter Mean StdDev r N p r N p 2 3 4 6 7 8 20 11 14 15 43.0 41.2 43.7 43.5 38.4 44.3 41.7 43.5 37.1 37.1 5.2 6.7 5.0 5.7 6.8 6.8 6.2 7.0 6.3 7.4 -0.270 -0246 +0.054 -0.392 -0.274 -01.54 -0.163 -0.055 -0.492 -0.651 81 78 81 81 77 81 80 77 80 75 0.02 0.04 0.64 0.00 0.02 0.17 0.15 0.64 0.00 0.00 -0.228 -0.190 -0.014 -0.133 -0.273 -0.180 -0.161 -0.238 -0.527 -0.160 76 77 78 79 75 78 77 76 77 74 0.05 0.10 0.91 0.24 0.02 0.11 0.16 0.17 0.04 0.00 Mean Median -0.264 -0.258 -0.210 -0.185 B. Group Differences by Goals across Time Four roughly equally sized groups were created based on the total performance across the ten tests. To examine the differences between high achieving groups and low achieving groups in relation to goal setting, expected performance, and actual performance across the ten tests during the semester, a repeated measures MANOVA was performed. The three factors were performance group (four levels), measure (satisfaction goal points, pride goal points, expected points, actual points), and time (the ten different tests). The important significant difference for this analysis was the group by measure interaction (F(9,107.2) = 5.76, p<.05, partial 02 = .274). The highest performing group has small differences between goal points, expected points, and actual points. The lowest performing group had very large differences between satisfaction and pride goal points (high), expected points (high), and actual points (low). The two intermediate groups had appropriately intermediate differences between the two extremes. The three-way Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 46 interaction was not significant. Thus, the lowest performing group of students did not adjust their satisfaction or pride goal points, nor their expected points, to the reality of their actual points. This further supports the hypothesis that low achieving college students are less likely to use metacognitive awareness to make adjustments as they are learning in a college course. The actual test scores of the highest achieving students are very similar to their satisfaction goals and their expected points, with their pride goals Figure 1: Examination Points as a function of Type of Points and Performance Level of Student 35.00 38.00 41.00 44.00 47.00 50.00 53.00 Actual Points Expected Points Satisfaction Goal Pride Goal Measure E x a m in a ti o n P o in ts A Students B Students C Students D & F Students approximately 4 points above their achievement. When the highest achieving students are incorrect in their postdiction they are more likely to under-postdict their score which may have led to more extensive studying before the test. As shown in Figure 1, he actual test score of the lowest achieving students are significantly less than their expected points and their goals across the entire semester which may have led to less studying before their test. Expected points, goals, and self-efficacy can vary dramatically both across individuals and within individuals across the semester. When a high achieving student is able to make accurate postdictions, is that just because the high achieving student lives up to the universal expectation of good performance, or does the high achieving student appropriately lower expectations when they are going to perform at a lower level in a way that the low achieving student does not? That is, in a course which has a weekly test even high achieving students occasionally have a bad day and if the hypothesis about metacognitive awareness is correct these students should make adjustments to their postdictions, self-efficacy, and goals as the semester progresses. C. Intra-individual Differences across Time To focus on differences within individual students across the semester, we calculated Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 47 within-subject correlations of the test score, postdicted test score, hours studied, pre-test self- efficacy, and post-test self-efficacy within each individual student across the ten tests. Each student will have, for example, a correlation between his or her ten test scores and his or her ten postdicted test scores to help explore how changes in test scores compare to changes in postdictions which indicate whether changes in a student's postdictions across the semester reflect changes in their test scores across the semester. The postdictions of students with effective MKM should rise and fall in concert with their test results indicating that they knew when they did well or poorly on test. These individual correlations are characteristics of each student, and as such can be considered a type of individual difference variable. These individual difference variables were themselves correlated across students with each student’s final grade. These correlations have a number of important implications for self-regulation. Intra-individual Post-diction Accuracy across Time. Each student has a within-subject correlation between his or her postdicted points and actual points across the ten tests, which reflects the extent to which each student adjusts their postdictions to match their actual score across the test tests during the semester. Let’s call this within-subject correlation “relative postdictive accuracy.” The mean relative postdictive accuracy across all students is 0.24. Thus, using the weekly test scores and the student’s postdictions for each test, the average student is able to make a somewhat accurate relative postdiction of whether he or she will do better (or worse) on this exam than on the other nine exams. The between-subjects correlation of the relative postdictive accuracy with total points is 0.26 (p=.056) meaning that students whose relative postdictive accuracy is greater than the mean tend to score higher across the ten tests during the semester. Students who are more accurate in adjusting their estimation of how well they have done on tests from week to week are more likely to achieve more total points during the semester. Reliance on Effort for Pre-test Self-Efficacy. Each student has a within-subject correlation between his or her number of hours studied and pre-test self-efficacy. Let’s call this within-subject correlation “reliance on effort.” The mean reliance on effort is 0.30. The average student is more confident the more hours he or she studied for that test. But the between-subject correlation of reliance on effort with total points is -0.24 (p=0.12) meaning that students whose reliance on effort is greater than the mean are less likely to do well in the class. This could be interpreted to mean that the more a student depends on the number of hours they have studied to decide on their confidence for success, rather than on MKM to decide on how confident they are about the test, the less likely they are to do well on a test. It is good to be able to weigh metacognitive feedback during test preparation rather than being forced to depend primarily on the amount of effort expended when making a prediction on how well you’ll be able to do on an exam. Metacognitive Changes in Post-test Self-Efficacy. Each student has a within-subject correlation between his or her pre-test self-efficacy for achieving their satisfaction goal and their post-test self-efficacy for achieving their satisfaction goal across the ten tests. Essentially this measures how much each student is likely to change their self-efficacy for achieving their satisfaction goal from before they take the test to after having taken the test. Let’s call this within-subject correlation “self-efficacy constancy.” The mean self-efficacy constancy is 0.30. But the between-subjects correlation of self-efficacy constancy with total points is -0.39 (p<.05). Thus, students whose self-efficacy constancy is greater than the mean tend to score lower than students whose self-efficacy constancy is less than the mean. Students who use the feedback they receive from taking a test to adjust their self-efficacy are more likely to do better on tests Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 48 across the semester. This is consistent with what we would expect from students with good metacognitive awareness in that they are aware of how they have done after taking a test but before it is graded. These three individual difference variable all support the theory that high achieving students are consistently monitoring their understanding of their learning and adjusting their postdictions (relative postdictive accuracy), adjusting the time they spend studying (reliance on effort), and are better judges of how well they have done after completing a test (self-efficacy constancy) than low achieving students. This begs the question: Can metacognitive knowledge monitoring be taught to students and will improvements in MKM lead to improvements in learning? D. Metacognitive Impact on Choosing Test Questions Each of the tests in the course used a variability difficulty - variable weight test format where students were given choices about which questions they selected to be graded. This test format allowed students to eliminate questions to which they did not know the answer but at a cost: choosing more difficult questions earns more points but only if you get the correct answer. For each of the 10 exams, students received a number of questions correct out of 30. Each student also had a number of points earned for the exam. Since students are allowed to choose 30 questions from 40 available questions, and since their total points are dependant upon choosing 30 questions that they are likely to get correct, it is critical that students choose the appropriate questions from week to week depending upon their mastery of the material. That is, some weeks students have clearly mastered the material and can choose questions which are more difficult and worth more points. Other weeks when students have not mastered the material as well, the students with good MKM can choose less difficult questions to which they know the answers while students with poor MKM are likely to guess at the answers and guess at which questions to choose to answer. To assess the accuracy of students' choices we correlated the number of points earned with the number of questions answered correctly within student so that each student had a within-subject correlation coefficient that measured the relationship between the number of questions answered correctly and the student’s score on the exam. We will call this within-subject correlation question dependency (QD). If all of the questions on the exam were worth the same number of points, QD would be 1.0 for all students. Because students chose which questions to answer, and different questions were worth differing numbers of points, the average QD was 0.94, the minimum QD was 0.74, and the maximum QD was 0.99. While it is clear that the number of questions answered correctly is the key variable in any student’s score, some students are able to assess their understanding of the material and the test question and choose the appropriate test questions to positively influence their grade. Students with high metacognitive skill should have a lower QD, because when they know that they do not understand the material well they will choose easier questions to answer, when they understand the material well they will choose more difficult questions which are worth more points, lowering their QD. Students with high metacognitive skills know when they understand the material and are more likely to be able to adjust their choice of test questions based on accurately reflecting upon their learning. Since the test format allows them to not answer some questions they can make their choices based on their assessment of their understanding. Their test points will depend relatively less on the number of questions answered correctly and relatively more on the difficulty of the questions chosen. Students with high metacognitive skill should also, on average, score higher on the exams. Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 49 A correlation across students was calculated between each student’s average test score and that student’s QD. The between-subjects correlation was negative ( r = -0.33, p < .01) as expected. Students with low QD had higher average test scores which is evidence of their metacognitive awareness; they used metacognitive strategies to adjust their test item selection based on their knowledge monitoring. The student whose test scores are more question dependant are most likely to do well based on whether they get questions correct, independent of which questions they choose. Expert college students in the present study adjusted their choice of test questions to enhance their test scores based on knowing-when-they-knew. It can be inferred that low achieving students were less accurate in selecting the questions they knew based on their metacognitive awareness of their understanding of the material and their comprehension of the test questions. IV. Discussion The literature on metamemory and metacomprehension has demonstrated that better learners are able to make more accurate judgments about their learning. The classroom application of these findings for self-regulated learning has not been thoroughly explored although the work of Hacker et.al. (2000) begins to shed light on these possibilities. Our findings begin to make a connection between the accuracy of students’ metacognitive judgments of their learning and a number of variables related to self-regulation. The initial step, in Zimmerman’s three step academic learning cycle, is forethought which includes goal setting and self-efficacy, the second step is performance and volitional control which is guided by self-monitoring, and the third stage is reflection including assessing success or failure and modifying self-efficacy. To be an effective self-regulated learner a student must use MKM to guide this process and make adjustments in goals, judgments of learning, self- efficacy, and task choice. This study begins to explore these relationships within individual learners. The first study of metacomprehension within a classroom context (Hacker, et.al., 2000) confirmed the finding that high achieving students are better at predicting and postdicting their learning. Our study supports this finding in relation to student postdictions. Students’ postdictions correlated significantly to their test scores. Of special interest was the finding that this correlation was greater for tests that were more difficult which supports the finding of Maki (1995). This relationship has potential implications for metacognition and self-regulation. If difficult tests are better at differentiating high achievers from low achievers the reason may be that difficult tasks require a higher level of metacognition and self-regulation. Given the demands for higher level thinking skills that occur when students transition from high school to college, it would be revealing to explore if effective MKM becomes increasing important in academic tasks that require these thought processes. The groups in the Hacker et.al. (2000) study were assigned based on their results on each individual test which leaves open the possible interpretation that the reason for the discrepancy between groups could have been a regression to the mean. If the test goals of most students were approximately 80%, and each student were to predict achieving their goal, the group discrepancy between predictions and actual score would be attributable to their initial goals. Since the group assignment changed across each of the three tests it is possible that group membership changed while goals, predictions, and postdictions stayed the same. In our study the group assignments were made based on the students’ accumulated test points across all ten exams. Group assignment based on accumulated points across the entire Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 50 semester rules out the possible regression to the mean explanation. The twenty students in the top group were the students with the highest scores across all ten tests. The twenty students in the bottom group were the students with the lowest scores. The repeated measures MANOVA demonstrates that the top group of students had the smallest differences between their satisfaction and pride goals, expected points, and actual points across the course of the semester while the lowest group of students had the widest discrepancy between these variables. Future research should examine whether students’ MKM changes over time, what factors might impact these metacognitive skills, and whether these skills can be taught to lower achieving students. The results from Hacker et.al (2000) also suggest that high achieving students are not only more accurate in their judgements but more likely to under-estimate their results (leading to under-confidence) while low achieving students were more likely to grossly over-estimate their results (resulting in overconfidence.) In these conditions self-regulation theory would predict that high achievers who are under-confident might be defensive-pessimists (Garcia and Pintrich, 1994) which would lead to an increase in their efforts and their success. Students who overestimate their result may be self-handicapping which may result in a decrease in their efforts which would lead to failure. The relationship between calibration, confidence, and self-efficacy will need to be more fully explored since self-protective perceptions may influence the accuracy of calibration (Dembo and Jakubowski, 2003). What is the impact when students underestimate or overestimate their mastery of the course material? How does this impact studying before a test? What impact does this underestimation and overestimation have on motivation to study? According to Zimmerman (1998), adjusting goals, expectations, and self-efficacy over time is a critical skill in self-regulated learning. The weekly class test format in our study allowed students to set and readjust goals and expectations before each test during the course of the semester. The pattern of pride and satisfaction goals, expected points, and achievement for the high achieving group is consistent with self-regulation, while the pattern for the low achieving students does not reflect the reality of their actual test scores. Low achieving students maintain the same general expectations which may lead them to learned helplessness. Future studies should explore the affective reactions of students who do not adjust their goals, their expectations in the face of continual failure, and their attributions for success and failure. One of the advantages of the present study is the longitudinal nature of the data set. Since students are given weekly tests and the same data is collected for each test, we can examine the intra-individual differences for a number of variables. Self-regulation involves small changes over time within, as opposed to across, individuals. To explore changes in self- regulation it is important to examine how the changes of one student week-to-week compare to the changes within another students. These intra-individual relationships can be examined as individual difference which can then be compared to variables such as final course grade. Our study explored three such relationships. The first intra-individual difference we are calling “relative postdiction accuracy.” This is a measure of whether a student is able to predict whether they will do better on this test relative to other tests. The average student is somewhat accurate in estimating whether they will do better on this test compared to other tests, but the highest achieving students are better at judging when they will do better or worse on a test compared to their own performance across the semester. While teachers at all levels may view this as "common sense", the ability of students to judge how they have performed on this test compared to other tests is indicative of the MKM that is critical to academic success in college. High achieving students are not only better at estimating their score on an individual test, they are also sensitive to whether they will Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 51 do better or worse than they usually perform on the class tests. Whether the reason is that they were not able to study as much as usual or they estimate that the material will be more difficult, high achieving students are aware that on this test they are likely to perform better or worse than they usually perform. Self-regulated learning in Zimmerman’s performance stage (1998) is dependent upon on-line monitoring of performance and when students are not able to assess whether they are doing better or worse than they normally do they are less likely to adjust their behavior. The second intra-individual difference is “self-efficacy constancy.” Students were asked to estimate their confidence for achieving their satisfaction goal before they took the test, and after they took the test but before the test was graded. Students who are metacognitive about the feedback they receive from taking a test adjust their self-efficacy based on this feedback. Students who lack metacognitive awareness do not change their self-efficacy. High achieving students are more likely to modify their confidence for an individual test after having taken the test; "That test was harder/easier than I thought it would be." This is also evidence of the metacognitive awareness necessary for self-regulation. When students do not, or cannot, adjust their self-efficacy after taking a test, it is likely they are not metacognitively aware of how they have done. The third intra-individual difference is “reliance on effort.” Many students link their test confidence (i.e., pre-test self-efficacy) to the number of hours they study for a test. When a student has studied four hours for a test they are usually more confident than when they have only studied one hour. But students who are truly self-regulating do not tie their confidence solely to the number of hours they have studied. Self-regulating students make judgments of when to stop studying based on how well they know the material and whether they believe they will be able to achieve their goals. Many students seem to have a pre-designated number of hours they plan on studying and if they complete those hours they believe they are sufficiently ready for the test. When students rely primarily on time to regulate their learning they may be less likely to succeed. The correlation of “reliance on effort” with total points indicates that the more a student relies on time to determine their pretest self-efficacy the lower their total points in the class. The relationship of metacognitive awareness, pre-test confidence, and self-regulation is an important variable to explore. Many students depend on time as the leading indicator of learning - - - and the result is often failure. How students decide they have spent enough time learning is an important question to be explored. The process of self-regulation depends on intra-individual changes in expectations, goals, and self-efficacy. Students who are skillful at self-regulation modify what they expect based on the feedback they receive from self-monitoring and external input (e.g., tests). They also adjust their goals and self-efficacy based on these results. The process of learning self-regulation requires extensive time and feedback. The present study begins to follow the development of this process in a college classroom with frequent feedback over an entire semester. The variable weight - variable difficulty test format in this study places a strong emphasis on student’s metacognitive awareness. Given this test format it is possible for a student to get all 30 of the questions they choose correct and still only earn a C, if they choose the easiest test questions. Students are told, and quickly learn, that their grade is dependant not only on the number of questions they get correct, but also choosing the correct test questions. This test format encourages students to take questions that are worth more points, which require higher level thinking skills, with the expectation that they will eventually learn to regulate their study time and strategies to go beyond the simple memorization of facts to the application, analysis, Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 52 and synthesis of information. But learning at this higher level is not the only skill necessary to succeed with this test format. It is also important for students to know-when-they-know. Students who used elaboration and organizational strategies in learning would be more likely to do well if all students were required to take all the questions of highest value. If all students took all the higher level questions, the student who got the most questions correct would have the highest score. But since students can choose which questions to take their score is also dependant upon their ability to choose the question they will get correct. The question dependency (QD) variable helps to reveal a student’s ability, across the semester, to choose questions that will enhance their test score based on their MKM. Students who lack MKM are less likely to be able to assess whether they understand, and can answer, each individual test item. For these students their test score is entirely dependent upon the number of questions they get correct because they do not make metacognitive choices on individual items on the test. For students with high MKM their test score is influenced by the questions they choose as well as the number of items they get correct. For example, a student with high MKM who, in a given week, is not able to devote as much time as usual to studying for the higher level thinking test questions is likely to choose lower level difficulty test questions on that test. These students are aware that they do not know the answer to the more difficult test questions and, instead of guessing, choose the easier questions for which they know the answer. Their MKM influences their total score which makes them less question dependent. But students with poor MKM cannot make these choices because they do not know-when-they-know and therefore are guessing: guessing not only on the answer to the question, but also on whether they should choose the question. This finding invites further exploration of the relationship of metacognitive awareness and learning particularly when higher level thinking is required. Many undergraduate students have difficulty when they first enter college because they are not familiar with the academic demands of higher level thinking. If they were effective in memorizing information in high school they were likely to have received good grades and also were likely to have believed their were good at learning. This would lead them to believe they would be successful in college if they were to use the same learning strategies. When they enter college and fail to meet their own expectation, they are then more likely to externalize the blame for their failure to the teacher or an unfair task. This may keep them from engaging in the reflection which is essential to changing their study behavior and improving their metacognitive awareness. Are high achieving college students better able to predict their scores on difficult tests because the questions are more difficult (Maki, 1995) or because the test requires higher level thinking skills? This study also raises the question of whether metacognitive awareness can be improved over time. It is clear that the highest achieving students in this study were better able to predict their test scores and also choose the right test questions. Are the metacognitive skills that allow these students to choose the right test questions a stable characteristic, or can these skills be improved over time? If these skills can be improved, what instructional approaches would facilitate an improvement in their metacognitive awareness that would impact their self-regulated learning and success in school? This study and others (e.g., Hacker et.al., 2000; Tobias and Everson, 2000, 2002; Maki, 1995) are demonstrating a strong relationship between MKM and academic performance but the causal relationship is unclear. Can an improvement in MKM lead to academic improvement or is improved MKM a result of improved academic performance? This study demonstrates that expert students are effective at estimating their understanding (postdicting their test scores) and they are more inclined to vary their goals and Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 53 self-efficacy based on past results and the feedback they receive from taking a test. Expert students in this study were also more likely to make choices which demonstrate mastery and non-mastery of tasks of varying levels of difficulty (choosing the appropriate test questions.) This has important implications for the teaching-learning process in higher education and secondary education. As students are required to take on academic tasks of increasing difficulty it is critical that they have the metacognitive skills to assess their mastery of the material on a variety of levels. This metacognitive self-assessment is essential to the application of self- regulated learning. This study also invites a thorough examination of the relationship of MKM to learning. It seems clear that expert students are skilled in reflecting on their own learning but the origin and nature of these skills is not clear. Does MKM facilitate student learning or does the mastery of a body of knowledge assist students in their judgement of their mastery of the material? Does MKM change over time and can it be taught? If MKM is a skill that can be learned, what pedagogical changes in classroom practice are most likely to encourage students to be more reflective of their own learning? What types of assessment practices are most likely to encourage students to be more metacognitively aware? Are there procedures that can be implemented in most postsecondary classrooms (e.g., frequent evaluation and immediate feedback) that would encourage students to be more metacognitively aware? Can technology be used outside the classroom to assist students to improve their MKM? The author is presently implementing pedagogical approaches that are designed to make students aware of the impact of metacognitive knowledge monitoring and assist them in improving their self-awareness of their learning. This study also raises questions about how to assess metacognitive knowledge monitoring. This study demonstrates the challenges inherent in assessing the metacognitive awareness of students while they are taking a test which reveals the even more difficult task of assessing metacognitive decision making while students are studying - which clearly is the more crucial connection between metacognitive knowledge monitoring and self-regulated learning. This study clearly demonstrates that low achieving students frequently over-estimate how well they know a body of information which leads to disengagement early during studying. This process typically results in failing the test which often leads them to blame the teacher/test for their failure instead of examining their own learning. A thorough examination of the impact of metacognition, and pedagogical approaches that might increase metacognition, are an important issue that should be addressed in the scholarship of teaching and learning. References Bandura, A. (1997). Self-efficacy: The exercise of control.. New York, NY: W. H. Freeman. Dembo, M., T.G. Jakubowski (2003, April). “The influence of self-protective perceptions on the accuracy of test predictions.” Presented at the Annual conference of the American Educational Research Association, Chicago, IL. Garcia, T., P. R. Pintrich (1994). “Regulating motivation and cognition in the classroom: The role of self-schemas and self-regulatory strategies.” In D.H. Schunk & B.J. Zimmerman (Eds.), Self-regulation of learning and performance: Issues and educational application. (pp. 127- 153). Hillsdale, NJ: Erlbaum Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 54 Hacker, D. J., Bol, L., Horgan, D. D., & Rakow, E. A. (2000). “Test prediction and performance in a classroom context.” Journal of Educational Psychology, 92, 160-170. Kelemen, W. L. (2000). “Metamemory cues and monitoring accuracy: Judging what you know and what you will know”. Journal of Educational Psychology, 92(4), 800-810. Koriat, A. (1997). “Monitoring one's own knowledge during study: A cue-utilization approach to judgements of learning.” Journal of Educational Psychology, 126(4), 349-370. Locke, E. A., & Latham, G. P. (1990). A theory of goal setting & task performance. Englewood Cliffs, NJ: Prentice-Hall. Maki, R. H. (1995). Accuracy of metacomprehension judgments for questions of varying importance levels. American Journal of Psychology, 108(3), 327-344. Maki, R. H., & Berry, S. (1984). Metacomprehension of text material.” Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(4), 663-679. Maki, R. H., J. M. Foley, W. K. Kajer, R. C. Thompson, & M.G. Willert. (1990). “Increased processing enhances calibration of comprehension.” Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(4), 609-616. Maki, R. H., & M. Serra. (1992). “The basis of test prediction for text material.” Journal of experimental psychology: Learning, memory, and cognition., 18(1), 116 - 126. Maki, R. H., & Serra, M. (1992). The basis of test prediction for text material. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(1), 116-126. Nelson, T., & Narens, L. (1990). “Metamemory: A theoretical framework and some new findings.” In G. Bower (Ed.), The psychology of learning and motivation. San Diego, CA: Academic Press. Nelson, T. O., & Leonesio, R. J. (1988). “Allocation of self-paced study time and the "labor-in- vain" effect.” Journal of Experimental Psychology, 14(4), 676-686. Pintrich, P. R., Walters, C., & Baxter, G. P. (2000). “Assessing metacognition and self-regulated learning.” In G. Schraw and J. C. Impara (Ed.), Issues in the measurement of metacognition (pp. 43 - 97). Lincoln, NE: Buros Institute of Mental Measurement. Pintrich, P., R. (1995). “Understanding self-regulated learning”. In P. R. Pintrich (Ed.), Understanding self-regulated learning (pp. 3 - 12). San Francisco, CA: Jossey-Bass. Schommer, M., & Surber, J. R. (1986). “Comprehension-monitoring failure in skilled adult readers.” Journal of Educational Psychology, 78(5), 353-357. Shaughnessy, J. J. (1979). “Confidence-judgment accuracy as a predictor of test performance.” Journal of Research in Personality, 13, 505-514. Isaacson, R.M., and Fujita, F. Journal of the Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 55 Sinkavich, F. J. (1995). “Performance and metamemory: Do students know what they don't know?” Journal of Instructional psychology, 22(1), 77. Tobias, S., & Everson, H. (2000). “Assessing metacognitive knowledge monitoring.” In G. Schraw & J. Impara (Eds.), Issues in measurement of metacognition (pp. 147-222). Lincoln, NE: Buros Institute of Mental Measurement. Tobias, S., & Everson, H. (2002). Knowing what you know and what you don't: Further research on metacognitive knowledge monitoring. College Board Report No. 2002-3. College Board, NY. Weaver, C. A., III. (1990). “Constraining factors in calibration of comprehension.” Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(2), 214-221. Zimmerman, B. J. (1998). “Developing self-fulfilling cycles of academic regulation: An analysis of exemplary instructional models.” In D. Schunk & B. Zimmerman (Eds.), Self-regulated learning: From teaching to self-reflective practice (pp. 1 - 19). New York, NY: Guilford. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006, pp. 56 – 74. Connecting, Making Meaning, and Learning in the Electronic Classroom: Reflections on Facilitating Learning at a Distance Charlene Johnson and William Brescia, Jr.1 Abstract : The increasing use of technology to meet the vast educational needs of our expanding world has led to heightened concerns about learning experiences within educational environments that are removed from the immediate purview of instructors (Duffy & Kirkley, 2004). Recent calls for use of more collaborative environments in which students interact with their instructors as well as with fellow students, have become more pronounced because of the purported learning benefits for students (Bonk, 2002). Constructivism, a theory of learning that is based on collaboration and interactions, provides such an environment (Jonassen, 1991). We report on the experiences of one professor who maintained a constructivist approach while teaching a foundation course in education (classroom learning theory) in a distance education setting. Emphasized are the challenges associated with creating the appropriate conditions for learning when moving from the face-to-face interactions of the regular classroom to the setting of compressed video. The implications of the medium for her role as facilitator, the establishment of a learning community, techniques of questioning and inquiry, and group collaboration are addressed. The impact of the medium and the greater cultural diversity of the distance education classes on how the tenets of constructivism are manifested and experienced by the students is also discussed. I. Introduction There is considerable use and application of technology for instructional purposes in the electronic classroom. Distance education including video conferencing, Web-based courses, and compressed video, has greatly increased the breadth and scope of educational outreach (Mangan, 2001). Educators use distance education to span the distance between groups and to ensure equitable access to educational opportunities for those interested in receiving additional training and/or expanding their educational repertoire in a number of disciplines (Mangan, 2001; Raymond, 2000). Accompanying the increased use of distance education and its technological tools is the clarion call for effective pedagogical strategies to ensure that the learning processes within this medium are as authentic and effective as those provided in traditional on-site, face-to-face (F2F) classrooms. To ensure that the learning experiences are equitable to those offered in F2F classrooms, Bernard, deRubacava & Pierre (2000) recommend that courses offered via distance education be comparable in format and content to the F2F courses. Given the structural and technological realities of teaching at a distance, adhering to this recommendation can be 1 Assistant Professor of Preventive Medicine and Director of Instructional Technology, University of Tennessee Health Science Center, 930 Madison Avenue, Room 836, Memphis, TN 38163, 901/448-5506, bbrescia@utmem.edu. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 57 challenging. Fostering and maintaining the interactions that are endemic to F2F classrooms further compounds the difficulty. Interactive instructional approaches that complement and build upon students’ existing knowledge base and experiences are touted as being highly effective for enhancing the learning experiences for a wide range of students, and for fostering higher-order thinking and problem- solving skills (Abdal-Haqq, 1998; Keiny, 1994). The student’s role in interactive instructional environments is to work with others to discover, construct, and participate in social collaborations that bring about meaning (Crumpacker, 2001). This is true for courses offered F2F as well as those offered via distance (McAlpine, 2000). Constructivism as a philosophy and approach to learning embodies these axioms. This article examines one instructor’s quest to remain true to her constructivist beliefs while leading a classroom of aspiring teachers via compressed video, an interactive video format. Her experiences while teaching the course via distance to three groups of students over four years are compared across the three classes as well as with those of the students she taught F2F. The foci of the discussion are the contributions, challenges, cultural and logistical implications arising from maintaining an effective constructivist environment via distance; and, the influence these experiences have had on the professor’s beliefs regarding teaching and learning processes. The questions addressed in this article are: 1. Can an instructor guided by constructivism remain true to the major precepts of this approach when physically distant from students? 2. How are learning community, collaboration, question/inquiry, and student voice manifested in distance-learning environments? 3. Does distance differentially affect students’ reactions to a theoretical, foundational course that is taught based on constructivism? 4. How do students’ reactions to and understanding of material delivered via distance compare to that of students in the typical, F2F classroom setting? 5. at are the implications of cultural understanding and compatibilities for reactions to and learning within constructivist classrooms via distance? II. Constructivist Theory The traditional transmission model of education views learning as an infusion of facts and information from one individual to another (Abdal-Haqq, 1998; Keiny, 1994). Constructivism diverges from this pedagogical model with its emphasis on collaboration and active participation of students as they seek to understand the material and resolve any inner conflicts it may cause. Students play an active role in constructing meaning from material studied based on their experiences and background knowledge (scheme). Their prior conceptions form a basis for determining the meaning of new knowledge. Collaboration and interaction among peers to “test” and mediate the knowledge process is integral to the process of making meaning from new experiences. Learning is considered a long-term phenomenon that requires discussion, debate, and opportunities to reconstruct ideas (Brooks & Brooks, 1999; Watts & Pope, 1989). Because knowledge is generated in collaboration rather than transmitted from one (a teacher) to another (a student), the role of the instructor is one of facilitator as opposed to “transmitter” of knowledge (Brescia, 2003, Abdal-Haqq, 1998). To effectuate these principles, collaborative learning is a major practice within constructivist approaches (Bonk, 2002). These ideas concerning learning and its interactive nature are founded and endorsed by a number of theorists including Jean Piaget, Lev Vygotsky and Paulo Friere. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 58 For Piaget, the development of scheme, the cognitive mental organization within the minds so fundamental to learning, hinges on experiences and the meanings individuals attribute to them (Wadsworth, 1996). These meanings are mediated by interactions with others. Vygotsky’s view on cognitive processes differed from Piaget’s in that he did not see cognition as being influenced by interactions, but as being determined by them. The implications of culture for these interactions and the meanings given to them are an integral part of his ideas on learning (Wertsch, 1985). Freire (1970) brought another dimension to the interactive nature of learning by proposing the idea of critical pedagogy. Rather than education serving as a mechanism to assimilate youth into the existing order of things, critical pedagogy seeks to provide young people with the mechanism to build anew and revise the existing order of ideas and concepts. Freire was critical of the “banking” paradigm in education which asserts that we seek to “deposit” knowledge into children, with little to no recognition of their culture, i.e., what they bring to the educational experience, or the relevance of the new information for what they already know. This process of depositing is considered detrimental to the authentic, contextual learning which Freire contended is necessary for the incorporation of information, lifelong learning, and praxis – action based on knowledge. For Freire, teachers facilitate learning and do not determine or provide the learning. As with constructivism, the trend or tradition of teachers as transmitters of knowledge is antithetical to Freire’s beliefs about learning and the transformative nature of the learning process. Each of these theorists emphasized raising the level of cognitive processing by building and expanding on what students bring with them experientially, culturally, and educationally so that they become learners capable of analytical reasoning in any learning environment. These ideas/concepts are especially pertinent for distance education given the nature of the medium and the variety of contexts that converge in the virtual classroom. Likewise, educators who believe in these principles should endeavor to ensure that authentic, transformative learning processes are germane to their instructional approaches. Facilitating is an instructional technique which is essential to effectuate these principles. For preservice teachers, coursework that reflects these principles enhances their ability to make meaning of foundational theories and furthers their understanding, incorporation and subsequent use of these principles when they construct educational practices (Keiny, 1994; Abdal-Haqq, 1998). To ensure that students/preservice teachers are exposed to these principles in action, instructional strategies such as group work and focusing on major ideas are regularly used (Muirhead, 2001). In groups, ideas are discussed, debated, and negotiated with peers and instructors. These class dialogues are integral to students processing and formulating meaning of the material across varied perspectives. Focusing on major concepts rather than facts in isolation and ideas out of context allows students to incorporate information in a more meaningful and contextualized manner (Abdal-Haqq, 1998; Van Looy, Callaert, Debackere, & Verbeek, 2004). Both practices reflect the belief that knowledge is constructed and predicated on the relevance or meaningfulness of the knowledge for the student. For distance learning these practices are even more pertinent given the nature of the medium with students who differ by location, experience, and educational background. Negotiation of meaning is more complex and richer given these differences but its success hinges on the learning community that is developed. Establishing a learning community is critical to the success derived from these interactive practices for student learning. The community ethos is an essential component of a constructivist classroom. It provides the safety net that frees students to share their experiences and ideas in an Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 59 otherwise risky environment. Establishing a distance environment in which students feel safe makes it possible for learners to share their knowledge across the various sites (Brown, Collins & Duguid, 1989). These principles and practices are standard within classes taught by the authors whether they are F2F or via distance. In the next sections, the context, i.e., course, program and class activities; students and their reactions, including those on campus as well as those at a distance; and the instructor and her reactions are discussed. III. Constructivism F2F A. Course Information This course is part of a five-year, Masters of Arts in Teaching (M.A.T.) licensure program. It is an upper-level (junior or senior) educational psychology/learning theory course required for all education majors. It is taken after a student has completed the introductory course in education. The instructor taught this course three years in a F2F setting; was among the inaugural group who developed and taught the course; and has been integrally involved in its development and evolution over the years. The instructors who collaborated on the design of the course ascribe to constructivist principles and agreed that the course would reflect these ideas. The activities were developed for F2F classes based on constructivist ideology. B. Students The majority of students in the F2F classes are traditional in that they range in age from 19 – 23 years old, are single, have limited experiences outside high school, and have few responsibilities beyond themselves. Most of the students are of European descent and grew up in ethnically homogenous areas of the state. The course is a core course for all students majoring in education, and, as a result, each class contains students from various concentrations: elementary, secondary, music, art, agricultural, and special education. C. Classroom Practices – Student Reactions Over the years, it has become evident that students are basically unfamiliar with anything which approximates constructivism in an educational environment. To expeditiously introduce them to this approach, the following paragraph is included in the instructor’s syllabus for the course. Course Requirements: Class Attendance/Participation: A constructivist approach is employed in class whereby students are encouraged to be actively involved in and responsible for their learning. Students are encouraged to read the material, note any questions or areas where concepts were not fully understood, and raise these questions during the class. However, the readings will not be “rehashed” during class. The focus in class is on students’ interpretation and synthesis of the reading material; different dimensions of the issues/ideas will be explored. A variety of learning approaches are used within the class for interpreting and analyzing the material. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 60 During the first days of class, this passage is discussed including what is explicit in this paragraph and the implied expectations. To further familiarize students with the approach and its tenets, one of the topics assigned for group presentation is constructivism. Peers (group members) give their interpretations of the approach and the underlying rationale for the instructor’s seemingly ambiguous responses and unstructured activities. Although largely unfamiliar with the constructivist approach, some students do report they have “heard about” the instructor from other students who have taken the course from her (mixed reviews). Based on these reports, the students have some preconceptions about the instructor. However, the instructor has no knowledge of them as students/learners. Every class is novel with its own unique personality and needs; the ongoing challenge is to develop each distinctive personality into a community of learners whereby the principles of learning within constructivism’s parameters can be manifested. The development of a learning community is integral to the success of this approach because it is vital to elicit sincere, reflective thinking on issues (Bonk, 2002). To facilitate this type of environment, a significant amount of attention is given to “reading” and knowing the students. The first and most basic way of becoming familiar with the students is to recognize them by name. Becoming familiar with names and hearing students’ voices are initial steps to building community among the students. Name cards are created (students write their names on folded index cards– 5” x 8”) the first day of class. These cards are placed on the students’ desks during the first few classes, until the instructor recognizes them by name. Another practice that contributes to the association of names with faces is calling on everyone at least once during the course of a class session. In addition to gaining familiarity with the students’ names, this practice indicates that value is placed on hearing the different “voices” within the class. However, a student always has a right to “say no.” That is, if they do not wish to “share” when called upon, a student can say he or she does not wish to share their perspective on that issue. This norm is established at the beginning of the semester and adhered to throughout the term. Groups are another means to build community and they also contribute greatly to the mediated process of learning. For each class, there is a major assignment to be part of a group presentation for the class. This assignment reflects collaborative learning’s principle of interdependence in that the same presentation grade is given to each member of a group. Individually, students write a paper or an article critique related to the research done for their contribution to the presentation. This ensures some measure of individual accountability (Johnson, Holubec, & Roy 1984). In addition to this assignment, groups are formed within class on a regular basis to discuss or explore the concepts being studied. The format and manner for responding within the groups varies. An example of an in-class group activity that is used during the study of Piaget and his theoretical principles follows. Students are placed into groups and directed to depict on newsprint (markers are provided), without using sentences or paragraphs (narrative), Piaget’s major theoretical precepts concerning thinking and cognition (assimilation, accommodation, organization, scheme, equilibrium, and disequilibrium) (Woolfolk, 2004). The idea is not to just restate a precept (e.g., “assimilation is fitting new information into existing schemes,”– repeating a statement from the book [Woolfolk, 2004, p. 31]). Students are required to somehow demonstrate/illustrate the meanings and relationships of Piaget’s theoretical precepts for cognitive development. Initially, most students express apprehension about this assignment and Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 61 cite a lack of drawing ability and/or uncertainty about how to create a depiction as reasons why they lack the skill or expertise necessary to participate in the activity. However, with prodding and encouragement, they slowly begin to discuss the meaning of the terms. Eventually the class is buzzing with ideas as students become engaged in the activity. Discussions concerning the meanings of the terms/precepts and the best way to illustrate and present these meanings emerge and enable students to illustrate their interpretations of the precepts in insightful ways. This activity reflects constructivist principles in that students’ backgrounds, majors, interests and educational experiences are incorporated in these depictions. A wide range of motifs including animals, knitting, sports, farming, etc., become a backdrop for explaining these concepts and their relationships. Group processes are monitored by checking with each group, asking clarifying questions about the depiction and its meaning, and addressing questions the group members could not resolve for themselves concerning the precepts and/or the appropriateness of their drawings. The activity and the illustrations allow the instructor to make a firsthand assessment of students’ understanding and interpretations of the material. Peers also assist in addressing misconceptions. This activity also incorporates another supporting belief of constructivism: the importance of recognizing and valuing multiple intelligences, i.e., the varied ways in which students process information (Gardner, 1983). Often students who struggle with traditional learning activities which emphasize memorization and recitation excel in this activity. On the other hand, many students who excel in these traditional activities are challenged by this exercise. Another area of angst for students is the inquiry nature of the course. Students in general are initially uncomfortable with nature of the professor’s questions and her responses to their inquiries. Questions that focus on the major concepts and how students incorporate them into their practices prevail in the classes. Questions are asked concerning the relevance of the assigned readings (What idea or concept in the chapter “spoke” to you and why?); familiarity with the practices shown (Have you ever experienced this practice? If so, what were your reactions?); and the implications of these ideas for educational practices (Given the research on the effectiveness of this approach/idea/concept/practice, will you incorporate it into your practice?). Follow-up questions are often asked to further explore students’ meanings, perceptions, and interpretations of the material. The intent of these questions is to get students to grapple with the ideas and assess the implications for them as educators. Often, the professor responds to students’ questions with a question: “What do you think?” Initially, students are frustrated with this response because they want the instructor, to “just tell me the correct answer.” They want to be imbued with the instructor’s understanding of a concept rather than develop their own. Instead, they are told that “correct” is contextual and that, although they are expected to follow certain analytical processes when studying material, they are expected to arrive at their own conclusions concerning how effectively a model explains phenomenon. As facilitator, the instructor places the onus of responsibility for addressing their issues and concerns with students – which adds to their frustration. They are given assistance in the process of making decisions about the relevance and implications of the material studied, but this is not determined for them. The question and answer procedure allows the community – both instructor and students – to better recognize any confusion or misinterpretation of material. When a student explains his/her thoughts on an issue, others gain insight into how someone else interprets the concept. Listeners can then assess that interpretation in light of their own perceptions of the concept. As Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 62 the semester progresses, students tend to: become increasingly active in their learning process; address each other’s questions or concerns; and accept that answers are contextual and may differ based on perspectives. IV. Constructivism at a Distance A. Program/Course Information The off-campus licensure program was initiated to support the training and local employment of new teachers, particularly those of African descent (because of the largely African American population of the area). Reflecting national trends in education, the number of African American teachers in the state’s schools has declined (American Council on Education, 2000). Rural, economically depressed regions such as the one targeted by this off-campus M.A.T. program, are usually the first to feel the effects of this decline. The region has difficulty attracting and creating employment opportunities to replace the agriculture-related jobs lost because of technological innovation. The program also contributed to the flagship university’s stated mission to meet the needs of all the citizens of the state. The program was set up so that students completed their first two years of coursework at the regional community college. The upper-level courses for the undergraduate degree leading to the M.A.T. program and the graduate courses in the M.A.T. program were offered via compressed video at a location 300-plus miles from the state’s flagship university which was the instructor’s home campus. Initially, it was expected that the students would travel to the flagship university to complete the last two years of undergraduate coursework and courses for M.A.T. degree. However, most of the students were nontraditional (over 25 twenty-five years of age), had families, were employed full-time, and were not able or willing to leave their families and jobs to complete the degree program in residence. Distance education (interactive video) was a viable alternative for the students to complete the degree program. The only degree/licensure program offered via distance education was elementary education. Instructor The instructor shared a common ancestry, of African descent, with the majority of students within the program, but was not born or raised in the state. During her time at the university, she had worked as a consultant with the schools in the area where the M.A.T. program was directed. However, the distance education course was her inaugural instructional experience teaching a university course to a class composed entirely of students from the region. When first asked to teach this theoretical course using distance education, the instructor was hesitant, resistant, and apprehensive. Could a constructivist approach be implemented effectively without physical presence and proximity? The opportunity to interact with a population of students from another part of the state who did not otherwise have access to the course was a strong incentive that overrode her initial hesitancy and trepidation. The challenge was to use the advantages provided by this medium without compromising the integrity of her beliefs about the roles of learner and teacher. The instructor’s initial impression of distance education and its learning expectations (lecture, transmission model) was based on the setup of the room from which the video was to originate. The setup resembled that of a principal’s office. There was a chair set up at a table with controls and a microphone. After a consultation with the technical-support people, the setup was changed to be more in line with the instructor’s preferred mode of instruction. The chair and a podium were placed next to the controls, which eliminated the appearance of a Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 63 barrier separating the instructor from the students. It was a seemingly minor change, but it gave the instructor the freedom to move around during the class which conveyed a more interactive persona. A. Students Over the course of four years, the course was offered three times via distance education. Although similar in some respects, each group has its own unique quality. The first class (Group 1) was predominantly African American (12 of 15) students; nontraditional in that 80% were over 25 and had families and jobs. These students were selected based on their excellent academic records and high grade-point averages. They were the program’s pioneers who experienced the usual intricacies of a new program. These common experiences and frustrations bonded them to each other. The second class (Group 2) was similar to the first in terms of ethnicity (16 of 19 students were of African descent) but differed in terms of age and experiences; less than 60% were under 25, had families and outside jobs. In addition, the educational backgrounds were less stellar than those of the first group. For the inaugural group, high performers were selected to ensure the success of the program. The program received a significant amount of publicity and public response which was very positive. People from the community, the region, and the state were impressed with the program’s intent–providing schools in the area with well-prepared teachers– and with the distance-education format. This support was a catalyst for continuing the program for subsequent cohorts of students. Those who came after the first group were recruited less aggressively and with less stringent academic requirements. The third class (Group 3) differed from Group 1 and Group 2 in terms of ethnicity. Ethnically, the class was more diverse with only nine of the eighteen students being of African descent and nine of European descent). Ages and experiences resembled those of Groups 1 and 2; only three of the eighteen (17%) were traditional students. As with the F2F classes, each class had its own unique personality. The varied personalities affected reactions to the lessons and the climate that evolved in the class. B. Classroom Practices – Student Reactions The distance-education students were also unfamiliar with a constructivist approach to learning and initially had mixed reactions to it. Most students responded amicably to being asked to bring their realities into the classroom and share their opinions and ideas about the material and its relevance for them (Knowles, 1990). In general, there had been little (if any) attention given to their experiences or perspectives by other instructors (Delpit, 1995; Freire, 1970). They welcomed opportunities to relate the course material to their lives and varied realities. These opportunities albeit gratifying and somewhat unexpected, did not preclude the discomfort associated with the process. Overall, these were high-caliber students whose high grade-point averages (pervasive in the first class, somewhat common in the second and third classes), were indicative of their success with the traditional transmission model of learning. Similar to the students in the F2F classes, the students preferred the didactic approach to teaching and learning, which involved listening to lectures, taking notes, reading the assigned texts, studying the notes and readings for exams, and responding with memorized information to exam questions. Tolerance of ambiguity was minimal. The focus was on completing assignments and getting correct answers. Similar to Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 64 the F2F students, the explore-and-find-meaning approach emphasized by the instructor was different than what they were accustomed and, therefore, a source of great frustration. For the distance-education students, this frustration was further compounded by obligations to families and employment. They wanted (sometimes demanded) to know the exact, “correct” answers so they could recite them on cue just as they had done in the past with great success. They were not inclined to analyze and grapple with issues. Interacting with the material and analyzing concepts for relevance and meaning were not customary practices for most of the students and they were unsure of the steps in and/or the feasibility of the process. When confronted with this situation on campus, being physically present the instructor can note students’ reactions, more readily assuage any discomfort, and facilitate their thinking about and interacting with the material. Being 300 miles away and viewing students via a monitor presented a significant challenge to this mode of accommodating students’ needs. Establishing a learning community and nurturing risk-taking behavior within the classes via distance was a challenge that was met with mixed results. Methods used successfully in the F2F environment were initially employed for the distance-education classes. Name cards were constructed, but the faces and the cards were not always clearly visible on the monitor. As an accommodation, students were asked to state their names before speaking. This helped to familiarize the instructor with the names in a time comparable to that in F2F classes. For a short time, students were acknowledged via an attribute or characteristic – “the young man in the red shirt,” “the person sitting beside [student name],” or “the person sitting in the corner.” This method, due to its impersonal nature, was used only during the first class period for Group 1. Reactions to being called upon to share one’s perspectives were also mixed. Although they welcomed the opportunity to relate the material to their experiences, culturally, they did not appreciate or accept the process for communicating their understanding of material to the instructor. African American response style influences how constructivist ideology and practice is perceived and implemented. Delpit (1995) discusses the direct response style prevalent in African American communities and its incompatibility with obscure approaches to instruction, such as process approaches to literacy and constructivism. She posits that the ambiguousness of response that is endemic to constructivism is foreign to the direct/straight-forward style of communication and discussion prevalent in African American communities. This premise is mostly applicable to literacy approaches for children of color when trying to incorporate sounds and meanings of words that are incongruous with their backgrounds. The instructor was aware of and had reflected on this contradiction and its significance for her approach to teaching and learning. Previous to teaching the distant education class, she had not had an opportunity to really grapple with this enigma given the majority European American student population she usually teaches. However, with the distant education students who embodied these realities, her beliefs on culture and its implications for meeting student needs were under scrutiny in conjunction with her constructivist philosophy. Although somewhat in agreement with this premise, she had a different “take” on this seemingly contradiction. She believed that the constructivist approach to teaching and learning does not run counter to the response style prevalent in African American communities under certain conditions. Students of color are constantly dealing with ambiguities in language and meaning as a matter of cultural style (Delpit, 995). Translating this into classrooms and using it to enhance student learning was the challenge. Shared ancestral background helped in meeting this Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 65 challenge. Common understandings of words, interaction styles, and conventions, allowed the class and the instructor to interact directly as they capitalized on meanings and extended them – reflective of constructivist precepts. Multiple responses that were recognized and valued to the seemingly vague questions (more than one “correct” answer) empowered and furthered students’ “voice.” They explored personal meanings and expressed their feelings about the course and its requirements. Once the students were more familiar and comfortable with the instructor, they directly expressed their frustration with query and response in the expectation that she would be more forthcoming with the “correct answer.” When the instructor continued to probe and explore their meanings and interpretations of the materials, students addressed the questions and displayed their frustrations in varied ways. As in the F2F classes, many times when a student was asked a question, there was a seemingly long and uncomfortable silence while waiting for a response. Using wait time, the question was not passed to another student until the first student indicated he or she did not know the answer or did not wish to answer. Via distance, this uncomfortable silence hangs between the two locations and its manifestation is difficult to gauge with limited vision of students and their reactions. Frequently, students attempted to address the discomfort by assisting each other in providing an answer to the question. They offered clues, hints, and, on some occasions, answers based on what they perceived the instructor wanted. The combined efforts of students in this situation were crucial in furthering the analysis and understanding of material by the distance classes. Via distance, with the instructor limited to the capabilities of a monitor, students restating a question or providing cues/hints for a classmate greatly aided in the shared understanding of the material and its meaning. In addition, the instructor was able to hear how students collectively processed and interpreted the information. Similar ancestry helped mediate the understanding process. However, given the differing backgrounds of instructor (midwestern, urban) and students (southern, rural), these clarifications were instrumental to understanding the context and meanings given to concepts by the students. The students’ common experiential knowledge helped them negotiate meaning in a context that was relevant for them (Kim & Hannafin, 2004). This process evolved in Group 1 and was often observed in Group 2, but not in Group 3. The interdependence of Group 1 based on their similar backgrounds and goal to successfully complete the program, intensified their sense of community and served to define a common purpose for the class as they sought to make collective meaning of the material. Group 2 was not as solidified as a learning community, but their similar experiences within and outside the course served to mold them into more of a community than Group 3. Group 3 had little sense of community. It was more fractured on the basis of their experiences and backgrounds. Additional effort was required to get them to collaborate on the meanings of the material. These experiences with the three classes underscored the benefits of establishing a learning community for the class if one is not already in place. The community serves the needs of the class in a number of ways. Developing community is contingent on hearing and honoring student voice but more often than not, students resist this objective/focus. In Group 1 a student requested that she not be asked questions in class. When the instructor explained the need to hear her voice and interpretations of the material, the student countered with she was shy and wanted to hear everyone else and learn from them. A compromise was reached, in which an attempt would be made to forewarn her before calling on her. This compromise had been used in F2F classes on rare occasions for students with similar issues and/or diagnosed special learning needs. Over Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 66 time, the interactions enabled the student to become more vocal in class, offering responses based on her interests. She realized that by speaking voluntarily on issues which held her interest, she was able to choose when and how she participated. For other students who expressed similar concerns, a compromise was struck whereby their voices were heard during class without (as they perceived it) being “put on the spot.” Group presentations were required of the distance-education classes, but accompanied with additional challenges. The topics for the presentations were selected by the instructor and were broad in scope (Vygotsky, multiple intelligences, constructivism, etc.) to allow each group considerable latitude in determining their focus and areas of interest within the category for their presentation. Based on their collective interests, each group was to determine its focus; research the areas of interest; and, develop a presentation based on their collective research. Maintenance and monitoring of groups to ensure they function effectively proved to be a major challenge given the distance and the available resources. In F2F classrooms, if a group had problems or concerns, students requested assistance after class. Sometimes an intervention was made that same day or by the next class period. Typically, issues were minor and pertained to differences in working styles or opinions concerning the appropriate manner of presentation. In the distance environment, students’ concerns were more difficult to address. Students could not talk to the instructor after class because the audio and video feed connection between the locations was shut down at the ending time for the class. Students used email, but in many cases that did not provide the desired level of immediate and personal interaction preferred in these situations. Thus, for the distance learning classes, personality and working-style issues that were ameliorated early and easily in the F2F classes became major stumbling blocks to developing an effective presentation. An associated issue stemmed from the complex restraints on time and responsibility for the students with families and jobs. Finding time to meet outside of class to decide on the focus and manner of a group presentation is problematic even for relatively unencumbered traditional college students. However, when one works all day and attends class in the evenings – distance education classes were scheduled from 4:00 to 9:00 PM, two or three days per week – meeting this requirement becomes even more complicated. It became obvious after the experiences with Group 1 that more support from the instructor via email and phone was needed to ensure they were functioning adequately. Subsequent to Group 1, one or two persons in each group emerged as leaders and became the major contacts at the off-campus site through which the instructor monitored the process. Additionally, individual group members were contacted to discuss any evident frustrations or concerns in order to support and encourage their involvement with the group. These measures helped to make the experience less stressful and more viable for the students. A major issue was access to library resources necessary to adequately investigate a presentation topic. Technically the distance-education students were enrolled at the flagship university. However, for library access they were limited to the inadequate collection of reference books and periodicals at the local community college. Theoretically, students had access to the ample resources, i.e., educational literature and periodicals, at the flagship university. Realistically, students often did not have ready access to these collections due to problems with the local Internet service through which they obtained online access to the university library. After several unsuccessful attempts to resolve library-access issues, the group presentations for Group 1 were cancelled. Instead, students wrote about their experiences Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 67 working with their groups and trying to secure adequate literature for the presentation. For Groups 2 and 3, library issues were addressed early via email and consistent updates in class. Access to resources did improve in subsequent years, after the university library became involved and worked to ensure access, but the problem was never completely resolved. Some students had access to more technologically updated servers and Internet providers, while others remained on the wrong side of the digital divide. To address this challenge of equity and access, books were recommended by the instructor on the presentation topics which the group could consider reading and using as an alternative resource to research articles. These books were available at the local libraries, through purchase from the community campus bookstore and/or online sources. With these adaptations, the group presentations remained a requirement for Groups 2 and 3. Classroom activities requiring group collaboration, such as the one on Piaget’s theoretical precepts, proceeded as with F2F classes but with some adaptations. Displaying and presenting results for distance education classes were done via the document camera because the drawings were harder to decipher if just held up by students and shown on screen. Moreover, additional description and explanation were often required. Students explained their depictions with rich detail and defended them when questioned about whether there had been appropriate application of the theoretical precepts. Reactions to questions about the non-written depictions and their meanings (how reflective of the precepts and their relationships) varied by class. In Groups 1 and 2, students tended to display their drawings and have them questioned with little to no display of defensiveness or hard feelings. By this point in the term, the understood response style allowed students to take risks and directly state their perspectives on the instructor’s interpretations of their ideas. Generally, the clarifications and explanations enhanced the shared understanding of the class on concepts being discussed. However, Group 3 lacked a strong community ethos and generally were more defensive about any comments or questions that challenged their depictions and interpretations of concepts. Eliciting the higher-order thinking required to address the questions raised in class was also a challenge to maintain and foster. When asked a question, students were hesitant to address it because of the possibility of follow-up questions requiring application and analysis. They feared they would not be able to address them adequately, thereby seeming incompetent to their peers. The community (peer) assistance was helpful, but as the class endeavored to explore implications of theoretical precepts for their communities and its students, it was obvious that many were unaccustomed to these types of questions and/or how to address them. According to Freire (1970), the students were not accustomed to learning as a basis for praxis or action. In the F2F classes, the instructor walked around the classroom, stood next to students, probed and encouraged responses recognizing that students were not accustomed to investigative/analytical questions. They were most comfortable when repeating a concept or principle from the book than when thinking about its meanings for meeting differing students’ needs. With a sense of community established, students were more likely to move from their comfort zones and venture to answer questions and/or propose solutions to issues. When they shared, the instructor provided encouragement and assisted them in making connections between the theoretical underpinnings of the concepts and their practical experiences with them. Distance-learning environments which by definition make physical proximity impossible changed the dynamics of these strategies resulting in them being less personal in format and function for the students. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 68 These impersonal realities of the distance education environment impact the role of facilitator and prove challenging when handling students’ frustration. Similar world views and communication styles helped in finding the right degree of directness for addressing student concerns. This was not as successful with the F2F classes due to the ethnic and cultural differences between students (primarily European American) and the instructor (African American). With F2F classes, a more subtle approach was used to address student frustration. With the distance education students, frustration was discussed candidly. Students openly complained to the instructor about the requirements and her “hard” grading. Conversely, the instructor directly addressed the class regarding concerns she had about the level of response and their performance on assignments. The common understanding concerning directness of response without ill feelings served the instructor’s purposes with the class also. Once, when concerned about the overall quality of their work (poor, inarticulate, and careless), the instructor discussed her frustration with this issue in the class. What came from the discussion was enlightening and reflected research findings for student populations in lower socioeconomic status areas. The students explained that the constructivist approach with its emphasis on collaborative learning, processing of information, and reaching higher levels of thinking, was new and unfamiliar to them. Past educational experiences had emphasized the rudiments of learning and focused on lower levels of thinking – rote memorization, comprehension, etc., known to predominate in areas where the poor and lower socioeconomic levels of people preside (Oakes, 1985). They were not “slacking” in their work but were finding it extremely difficult to engage in the analytical processes expected. Their explanations helped the instructor better understand why their level of disequilibrium was high and somewhat debilitating. The class explored how and why these lower-level cognitive exercises prevailed in the locality as well as the implications of a link between low socioeconomic status and the predominance of these approaches to learning (Apple, 2004). One student shared that she had recently observed these practices in her daughter’s classroom and had become more cognizant of them as a result of the class. She pondered the implications of these practices for area students’ future academic achievement and analytical abilities given that these higher-order thinking skills are necessary in higher education and in life. This was an extraordinary educational moment: the class had examined an issue and its implications for them making a difference as educators with their training – praxis (Freire, 1970). Based on this feedback, the instructor provided more opportunities in class for discussing concerns with the coursework. Questions and issues were addressed as directly as possible, while still allowing students to analyze the material based on their own realities rather than those of the instructor. Additional time for processing of information was recognized as necessary for the students to better fulfill required assignments. Periodically conducting the class F2F with the distance-education students was significant for familiarizing the instructor with the students and the students with the instructor’s approach to teaching. During these visits the instructor walked around the classroom and asked questions just as she does in her classes on campus. Students experienced constructivism in close proximity as opposed to having it modeled from the podium via distance education. They appreciated and understood the approach much better after those classes. They became more understanding and comfortable with the instructor and her approach after the visits. She tried to make at least three visits to the area: one to observe the group presentations, another to teach a concept, and the third for the presentation of the portfolio at the end of the semester. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 69 After the semester ended, the instructor continued her affiliation with some of the students. Several emailed her to express gratitude and their thoughts about the value of the course. The most memorable email came from a student in Group 1; the sentiments expressed embody her goals for the course and the students. She wrote: I just wanted to let you know how much I have enjoyed taking this class under your instruction. Some of my fellow classmates feel that you expected too much from us. However, they fail to realize that through it all they met all deadlines and they fulfilled all requirements. Therefore, they should realize something about themselves. I have really and truly enjoyed taking Classroom Learning Theory. Dr. Johnson, you will never understand how much you have helped me this semester. You have helped me to explore avenues within myself that I never knew existed. You made me push myself to excel far beyond that of which I felt I was capable of. Reflecting back over the semester, I realize more so now than ever before, the reason why you pushed us the way you did is because you want us to become the best educators that we possibly can. You want us to do more than just memorize answers, you want us to comprehend and fully understand the concepts and ideas. In my opinion, I honestly feel that you are the type of educator that I hope to become. An educator that cares about her students, both inside the classroom and outside of the classroom. Even when I thought that no one noticed how I felt at the time, you noticed. You encouraged me to hold on. I wanted to say thank you. Thank you for caring, thank you for pushing me to do the things of which you felt I was capable of doing. Thank you for being “TRUE” and thank you for being “REAL.” This email was received after grades were posted and the semester was done. Other students have expressed similar sentiments, but not as eloquently. In subsequent years, some have emailed the instructor about the value of the course when preparing for the licensure exams, PRAXIS II. It seems that the skills gained from having their ideas, beliefs and perspectives challenged, and being prodded to articulate their rationales proved beneficial for analyzing issues that are a major emphasis of the test. Additionally, case study is a part of the class, so it provides some exposure to this type of assessment. C. Instructor’s Reaction As a constructivist, the instructor’s questioning techniques were enhanced, and her commitment to groups remained steadfast in spite of its challenges. Her initial reaction to the idea of distance education has changed. She sees the value and efficacy of distance education for reaching students who would not otherwise be able to fulfill degree-program requirements. The exposure to another reality–i.e., rural, economically depressed area – was enlightening and led to a careful reexamination of her perspectives on difference and its manifestations. She especially appreciated having an opportunity to become privy to and interact directly with the different contexts and realities of student learning. Every class began with Current Events – this is true in F2F as well as distance classes. The instructor’s rationale, which is shared with the class, is that events worldwide have Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 70 implications for classrooms. In both F2F and distance education, local and world news are discussed and evaluated for relevance to the classroom, students, and educational issues. However, for the distance-education students, this activity became a forum for assessing local news and events from their perspectives. Local realities and histories brought different dimensions to the material being studied. Some of these realities have become part of the instructor’s educational scheme and continue to be incorporated into course activities when teaching precepts F2F. In this instance, distance education has informed the practices on campus. Educationally, the instructor has benefited as a facilitator and as a learner. Her questioning, active listening, and facilitation skills were enhanced as a result of these experiences. She has learned that her methods for detecting and handling student discomfort and disequilibrium in a F2F setting are not as effective when used in a distance learning environment. However, with attention to students’ needs and realities, a few alterations, some flexibility, and considerable patience, it can be just as productive. A dominant area of intelligence for the instructor is interpersonal intelligence (Gardner, 1983). Although aware of its influence on her as a learner and an instructor, the distance education experience further validated that the feedback and energy from the class are fundamental to her functioning in the classroom. There is a symbiosis of energy and involvement between instructor and students. The distance caused the instructor to be livelier and more involved in an effort to invoke this symbiotic relationship with the students. This is an area warranting additional attention as it relates to distance education – how do different intelligences and learning styles impact receptiveness to this medium as a learner and as an instructor? Although F2F interaction is the instructor’s preferred style, she understands that the attributes she dislikes most about the medium – lack of personal interaction, anonymity, and exclusive use of written responses, are the ones that some students find most suitable for their learning. To better meet student needs and “hear” the multiplicity of voices and tones, more familiarity and ease with this medium is warranted. A major benefit of the instructor’s involvement with the distance-education classes is her increased use and familiarity with technology for educational purposes. She has developed listservs and uses email much more frequently than before becoming involved with distance education. The areas of interpersonal intelligence and technology converge when the instructor emails students. Humor is a major tactic used by the instructor to maintain focus and interest in material considered to be “dry”, i.e., theories and their tenets by most students. In F2F classes when students misunderstand the instructor’s humor, her response, or manner of responding; their body language usually signals this and she immediately addresses the confusion. Email and listserv posts do not include these clues. The instructor became acutely aware of the importance of “tone” in email. Based on students’ reactions to content of her emails and on students’ perceptions concerning the “tone” of her messages, she has discovered that her audience may receive a message other then the one she intended to convey. She has worked on this aspect of her communication with some of the technical advisors at the institution and implemented several of their recommendations to use emoticons as indications of humor (smile, ☺, grin, etc.) in email responses. Improvement in this area has enhanced her communication skills for all classes. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 71 V. Conclusion and Implications The initial hesitancy, frustration, and bafflement concerning the constructivist approach and its expectations were as common at a distance as F2F. With each succeeding class, the instructor learned more about the importance of the sense of community within the class for the student interaction and involvement which are fundamental to the effectiveness of the pedagogical methods and student assignments. Over the years she has learned how to be more effective at mediating interpersonal and systemic challenges, to ensure that students receive equitable assignments and exercises (Anderson, 2001; Hardwick, 2000). For each class the degree to which a learning community is established needs to be assessed, and in the case of there not being one, it needs to be facilitated. For distance education this challenge is especially cogent because the instructor is less knowledgeable about all the realities and/or experiences of the students who are taking the course. The experience of distance education students with the library sources is characteristic of the logistic problems and the “hidden curriculum” embedded within distance education that has received limited attention in the literature (Apple, 2004). To ensure that the experiences of distance education students especially those in rural, less-developed areas are equitable with students on campus, access arrangements must be made for technological linkages (Anderson, 2001). Calling on students and asking several questions related to or associated with the local realities helps to establish community and build interpersonal relationships. Listening and using information from previous responses regarding the area and its realities when referring to a theoretical idea also invoke insightful responses. Students are often surprised that an instructor listens, values their responses, and builds on them. Active listening in any class is important when sincere, continued responses from students are a goal. However, at a distance, this is even more critical to the success of the collaboration and sharing of ideas. Instructors need to receive focused training and be given opportunities to practice using the relevant technologies before they face students who are relying on them for instruction. In addition to practicing with the technology, practicing appropriate distance-learning techniques is necessary for all instructors who want to foster fruitful collaborations and interactions. The establishment of community is necessary for collaboration and for constructivism to be effective in classrooms whether they are F2F or distant (Bernard et al. 2000; Hardwick, 2000; Keiny, 1994; McAlpine, 2000; Raymond, 2000). The necessity and benefits of designing instruction for distance education with these philosophical beliefs in mind have been documented (Bernard et al. 2000; Hardwick, 2000; McAlpine, 2000; Raymond, 2000). Creating appropriate environments involves more labor and time, and has its own distinctive set of challenges. However, doing so is beneficial for higher-order learning. Ensuring that learning experiences develop analytical and problem-solving skills equitable to those nurtured in students in F2F classes is essential for those who do not have direct access to university classrooms. Otherwise, distance education becomes another means by which the societal hegemony is continued by providing less for those most in need. Given that the focus or purpose of distance education is to extend educational equity and access, assurances must be made that students who receive their education via this medium are sufficiently skilled to compete and achieve. It is expected that in the future there will be an even greater need for higher-order thinking skills, problem-solving skills, and effective interaction skills with a range of individuals. Concurrently, the use of technology to meet the growing needs Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 72 of our students is growing exponentially. Therefore, further study on how to use and implement collaboration and constructivism more effectively via distance is needed. Additionally, better understanding of the impact of culture on the distance education learning processes is integral to developing effective environments that are equitable for a wide range of students with differing realities. References Abdal-Haqq, I. (1998, December). “Constructivism in teacher education: Considerations for those who would link practice to theory.” ERIC Digest, ERIC Clearinghouse on Teaching and Teacher Education. American Council on Education (2000). Annual status report: Minorities in higher education. Washington, DC: Author. Anderson, T. (2001). “The hidden curriculum in distance education.” Change, 33(6) 28 – 35. Apple, M. W. (2004). Ideology and Curriculum, 25th Anniversary 3rd Edition. New York: Routledge. Baec, Y. K. & Westrom, M. (2001). “Building a cyber training center for teachers in Korea.” Journal of Technology and Teacher Education, 9(2), 171-179. Bernard, R.M., deRubalcava, B.R., & St. Pierre, D. (2000). “Collaboration online distance learning: Issues for future practice and research.” Distance Education, 21(2), 260 – 277. Bonk, C. J. (2002, November/December). “Collaborative tools for e-learning.” Chief Learning Officer, pp. 22-24, & 26-27. Retrieved October 1, 2004: http://www.clomedia.com/content/templates/clo_feature.asp?articleid=41&zoneid=30 Brescia, W. F. “A Support Taxonomy for Improving Online Discussions.” Journal of Public Affairs Education, 9(4) 289-298, 2003. Brooks, J.G. & Brooks, M.G. (1999). In search of understanding: The case for constructivist classrooms (2nd ed). Alexandria, VA: Association for Supervision and Curriculum Development. Brown, J. S., Collins, A., & Duguid, P. (1989). “Situated Cognition and the culture of learning,” Educational Researcher, 18(1) 32-42. Crumpacker, N. (2001). “Faculty pedagogical approach skill and motivation in today’s education milieu.” Online Journal of Distance Learning Administration, 4(4) Retrieved online October 19, 2004: http://www.westga.edu/~distance/ojdla/winter44/crumpacker44.html. Delpit, L. (1995). Other people’s children: Cultural conflict in the classroom. New York: The New Press. Duffy, T. & Kirkley, J. (2004). “Learning theory and pedagogy applied in distance learning: The case of Cardean University.” In T. Duffy & J.R. Kirkley (Eds.), Learner-Centered Theory and Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 73 Practice in Distance Education: Cases from Higher Education (pp. 107-141). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Freire, P. (1970). The pedagogy of the oppressed. New York: Continuum Publishing Gardner, H. (1983) Frames of Mind: The Theory of Multiple Intelligences. New York: Basic. Hardwick, S.W. (2000). “Humanizing the technology landscape through a collaborative pedagogy.” Journal of Geography in Higher Education, 24(1) 123 – 129. Johnson, D.W., Johnson, R., Holubec, E.J., & Roy, P. (1984). Circles of learning: Cooperation in the classroom. Alexandria, VA: Association for Supervision and Curriculum Development. Jonassen, D. H. (1991). “Objectivism versus constructivism: Do we need a new philosophical paradigm?” Educational Technology Research and Development, 39(3), 5-14. Keiny, S. (1994). “Constructivism and teacher’s professional development.” Teaching and Teacher Education, 10(2), 157 – 167 Kim, M. & Hannafin, M (2004). “Designing online learning environments to support scientific inquiry.” The Quarterly Review of Distance Education, 5(1), 1-10. Knowles, M.S. (1990). The adult learner: A neglected species (4th ed.). Houston, Gulf Publishing. Mangan, P. (2001). “What is distance learning?” Management Quarterly, 42(3), 30 – 35. McAlpine, I. (2000). “Collaborative learning online.” Distance Education, 21(1), 66 – 80. Muirhead, B. (2001). “Practical strategies for teaching computer-mediated classes.” Ed at a Distance. 15(5). Retrieved October 14, 2004 from http://www.usdla.org/html/journal/MAY01_Issue/article02.html Oakes, J. (1985). Keeping track: How schools structure equality. New Haven, CT: Yale University Press. Raymond, F.B. III (2000). “Delivering distance education through technology: A pioneer’s experience.” Campus-Wide Information Systems, 17(2), 49 – 55. Van Looy, B., Callaert, J., Debackere, K. & Verbeek, Arnold. (2004). “Patent related indicators for assessing knowledge-generating institutions: towards a contextualized approach.” Journal of Technology Transfer. 28(1) 53-61. Retrieved October 14, 2004 from http://journals.kluweronline.com/issn/0892-9912/contents Wadsworth, B.J. (1996). Piaget’s theory of cognitive and affective development: Foundations of constructivism (5th ed.). White Plains, NY: Longman Publishers. Watts, M. & Pope, M. (1989). “Thinking about thinking, learning about learning: Constructivism in physics education.” Physics Education, 24, 326–331. Johnson, C. and Brescia, Jr., W. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 74 Wertsch, J.S. (Ed.) (1985). Culture, communication and cognition: Vygotskian perspectives. New York: Cambridge Press. Woolfolk, A. (2004). Educational psychology (9th ed.). Needham Heights, MA: Allyn and Bacon. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006, pp. 75 – 87. Rapid Prototyping as Method for Developing Instructional Strategies for Supporting Computer-Mediated Communication Among University Students Dave S. Knowlton1 Abstract: Because rapid prototyping results in the quick development of curriculum, materials, and processes, it is a form of design that could be particularly useful to professors in higher education. Yet, literature documenting the use of rapid prototyping in higher education is scarce. This paper offers a case example of rapid prototyping being used as a design process. After presenting the case, the author points to necessary considerations for other faculty members who are considering using rapid prototyping. These considerations include the need to gain perspective on the roles of instructional strategies and computers within teaching and learning processes, understand the distinction between traditional research rigor and design rigor, and the importance of approaching design systematically. I. Introduction and Purpose Designing meaningful learning experiences is difficult for professors. True instructional design often is too expensive of a process to be viable in higher education; and while carefully- constructed constructivist learning environments are becoming more widely used across the academy, such environments, too, require a high level of detailed planning, particularly when computers are involved. College professors simply cannot enter each semester with a solid constructivist design of all assignments and course activities. Sometimes, then, the best a professor can do is to design “something” as a part of a new course preparation and tweak it over time. In general, this process of designing and tweaking is referred to as rapid prototyping (Reiser, 2001). While a professor’s efforts to use rapid prototyping can result in the quick development of instructional materials or activities (Resier, 2001), the quality of resulting materials and activities often is suspect. Why? Models of rapid prototyping are surprisingly complex and are largely based on “progressive refinement”—“putting a first version of a design into the world” and then revising that design “until all the bugs are worked out” (Collins, Joseph, & Bielaczyc, 2004, p. 18). The process is not complete, then, when materials have been developed. Revising implies a detailed and systematic process; it is the iterative nature of designing that makes rapid prototyping a successful design approach (Jones & Richey, 2000). In fact, rapid prototyping often involves an entire support team to manage the design process (cf., Lohr, Javeri, Mahoney, Gall, Li, & Strongin, 2003), but most professors do not have access to such a level of human capital. The purpose of this paper is to offer a case example of the rapid prototyping process that I used to develop assignment guidelines for supporting students’ use of an online discussion board. Notably, this paper emphasizes the systematic development of the assignment guidelines 1 Assistant Professor of Instructional Design & Learning Technologies at Southern Illinois University Edwardsville, www.siue.edu/~dknowlt. Knowlton, D.S. 76 across three semesters of implementation. Perhaps this type of case example can be illustrative for other professors who need to systematically prototype assignments, sans formal training in rapid prototyping and support team. This paper begins with a description of the context in which the assignment was prototyped and then temporally describes the prototyping process. In the last section of this paper, I offer generalized principles for using rapid prototyping to develop assignments for the higher education classroom. II. Context in which Rapid Prototyping Occurred Part of my teaching responsibilities within a School of Education at a Midwestern university included serving as a member of a faculty team that supported the efforts of preservice teachers (undergraduate students majoring in elementary or secondary education) in a two-year, field-based teacher-certification program. The preservice teachers who were enrolled in this program were assigned to K-12 classrooms in partnership schools. Because this was only the second implementation of the entirely field-based certification program, much of the context supporting the program was still developing. During the first semester of the two-year program, the preservice teachers often assumed a periphery role within the classroom—serving more as a teacher’s aide than as a practicing teacher. During the last semester of the two-year program, though, the student teachers participated in a formal “student-teaching” experience. The field- based program was designed to support the preservice teachers’ development from aide to professional teacher. Throughout the two years, a team of university faculty supervised weekly content seminars. Within the seminars, faculty members sometimes resorted to lecture as a means of orienting the preservice teachers to various educational theories and methods; more often, though, within these seminars, preservice teachers were given opportunities to discuss their experiences in the classroom. During each of the first three semesters of their field experience, the preservice teachers were enrolled for one credit hour of educational psychology—the content that I was responsible for overseeing. In principle, though, “courses” were non-existent. Instead, each courses’ content was integrated into seminar activities and discussions. While the faculty team and preservice teachers came together for the weekly seminars, communication throughout the rest of the week was difficult. Most of the professors on the faculty team had other responsibilities that prevented them from spending substantive time within the partnership schools, and the preservice teachers were placed in a variety of schools across three school districts. Therefore, the team of faculty determined that since WebCT’s (the university’s approved online course management tool) discussion board depended on neither face-to-face communication nor real-time interactions, it would be a useful and efficient tool to help the preservice teachers stay connected with each other and with the faculty team. The rapid prototyping process that is the basis of this paper involves the design of strategies to support the effective use of bulletin board discussions. Table 1 provides (a) an overview of the factors that influenced the development of each version of the discussion assignment, (b) the characteristics of each version, and (c) a summary of evaluation findings for each version. III. The First-Semester Use of the Electronic Bulletin Board The first-semester guidelines supporting the preservice teachers’ use of the bulletin board proved ineffectual as a tool for promoting communication, much less learning. Within this Knowlton, D.S. 77 Table 1: Factors contributing to assignment design, assignment characteristics, and evaluation. First Semester Version Second Semester Version Third Semester Version Factors Influencing Design Factors Influencing Redesign Factors Influencing Redesign • Need for flexible and efficient communication tool • Emerging nature of the field experience • Lack of information about the preservice teachers’ knowledge and skills • Need to introduce basic educational psychology principles • First version was ineffectual • Shifting Responsibilities of preservice teachers • Changes to the use of weekly seminar time • Need for preservice teachers to gain skill in using web-based communication tools • Evaluation of revised version • Elimination of seminar time for educational psychology • Continued shifting responsibilities of the preservice teachers Initial Design Characteristics Characteristics of Redesign Characteristics of Redesign • Laissez-faire • Preservice teachers were simply made aware that discussion board existed. • Preservice Teachers assigned to two groups • Discussion based on three-week cycles • Discussion centered on student-initiated problems and proposals for practical solutions • Addition of a Privacy Statement and job aid emphasizing conventions of CMC • Additional direction to focus on “instructional problems”; more scaffolding to support “good” contributions • Added reflection writing and self report form Evaluation of First Evaluation of Second Evaluation of Third • Ineffectual and rare use • Preservice teachers reported that they didn’t see practical value of using CMC • Problems were narrow in scope • Interaction among the preservice teachers was limited • Grading was cumbersome • Perservice Teachers noted workload was heavy and contrived • Scaffolding of third week contributions did broaden the types of input from the preservice teachers Knowlton, D.S. 78 section, the factors contributing to the first-semester guidelines and a description and evaluation of those guidelines are discussed. A. Factors Contributing to the First-Semester Guidelines Commonly, careful analysis of both the educational context and learners precedes rapid prototyping (Jones & Richey, 2000). Because the context of the partnership school was still emerging, analysis was based largely on generalization. From a macro-perspective, it seemed that the unique context of the field experience would continue to emerge as implementation progressed. This symbiosis between context and implementation required that I give the preservice teachers plenty of latitude in their use of the bulletin board, which might include their decision not to use it at all. Furthermore, as the initial guidelines needed to be in place the day that I met the preservice teachers, I had no knowledge of the skill of the learner for which I was designing. Had they used a bulletin board before? Did they even know how to find the university’s WebCT site and log on? I did know that these preservice teachers had never before taken educational psychology. Some content, then, needed to be transferred to these preservice teachers. In a pedagogical age of open-ended learning environments and within the context of a field experience, I recognize the vulgarity of suggesting the need for knowledge transfer. Nevertheless, because of both the school of education’s accreditation process and certification tests that the preservice teachers would need to pass, the preservice teachers needed to obtain a basic understanding of educational psychology concepts and principles. This requirement further accentuated the need to de-emphasize the use of the bulletin board (which often is more well- suited for promoting open exploration than for supporting direct concept attainment) and emphasize activities and assignments that were more likely to promote direct knowledge transfer. B. Characteristics of the First-Semester Guidelines At the start of the first semester of the field-based program, the faculty team simply made the preservice teachers aware that WebCT had a discussion board where they could share ideas with each other and ask questions in a forum that would expedite communication. Once the preservice teachers were aware of WebCT, I informally suggested that they might use the bulletin board to collaboratively make sense of assigned readings and prepare for seminar activities. Admittedly, this laissez-faire approach contradicts much of the practical advice for using bulletin board discussions. Some literature suggests that if professors do not scaffold the “hows,” “whens,” and “whys” of using asynchronous discussion then students will not use it effectively, or even at all (e.g., Knowlton, Knowlton, & Davis, 2000). C. Evaluation of the Discussion Board’s First-Semester Use Predictably, the bulletin board was used rarely. When it was used, the contributions were most often in the form of close-ended questions: “What chapters are we supposed to have read by next week’s seminar?” Several preservice teachers noted that it was nice to know the bulletin board was available, but they did not have a need to use it often. That is, they did not see how sharing ideas on the bulletin board would help them prepare for their day-to-day activities in the K-12 classrooms. After all, their argument went, they daily had access to their mentor teachers— Knowlton, D.S. 79 the full-time teachers in the classroom to which each preservice teacher was assigned—who could guide them in their decision-making processes. IV. The Second-Semester Use of the Electronic Bulletin Board The laissez-faire approach to support learning through the bulletin board was not effective. To aim for more educational effectiveness, I shifted the emphasis toward a computer- mediated communication (CMC) assignment by designing instructional strategies that would more likely secure the preservice teachers’ participation. The formalized design showed some promise, but evaluation suggested the need for refinements to the assignment’s design. A. Factors Contributing to the Second-Semester Guidelines Both the preservice teachers’ “readiness” for a higher level of professional thinking and their shifting responsibilities in the classroom necessitated formalized guidelines to support the use of the computer-mediated discussion. During the first semester of the partnership program, I had assigned readings from the adopted educational psychology book (see Eggen & Kauchak, 1997). These readings served the purpose of introducing the preservice teachers to the large issues that fall within the domain of educational psychology. Once the preservice teachers had been exposed to key educational psychology concepts, they needed experience applying those concepts by making connections between textbook theory and real-world classrooms. Such connections can be useful in supporting students’ problem-solving efforts in field experiences (Beckett & Grant, 2003). This shift from “knowing” to “applying” seemed further appropriate because it paralleled the preservice teachers’ shift within the partnership school. The preservice teachers slowly were moving from serving as paraprofessionals—by taking class attendance and organizing materials, for example—to participating as true professionals—by designing lesson plans and teaching the entire class. A second contextual factor also created the need for more exact guidelines to support the use of the discussion board. The team of faculty members who supervised the weekly seminars decided that more organization was needed within the seminars. No longer would the faculty team collectively guide discussion and facilitate activities; rather seminar time was divided among content areas—“Today is an Educational Psychology seminar; next week will be a reading methods seminar.” Such a shift was problematic because it violated one of the very foundations of a field-based program—that content should be integrated and directly based on the preservice teachers’ field experiences (cf., Beckett & Grant, 2003; Scanlon & Ford, 1998; Weber, 1996). Successful professionals must learn to think holistically about their experiences, not about “courses” from a program of studies. Designing and implementing more exact strategies to support CMC served as a means for prompting the preservice teachers to continue making integrated connections, even though seminar time was less integrated. B. Characteristics of the Second-Semester Guidelines Participants were divided into two groups and the electronic discussion was based on three-week cycles of sharing and response. Assessment of students’ efforts was integrated across the cycle. At the end of each cycle, roles were reversed so that preservice teachers in group one performed the responsibilities of the preservice teachers in group two and vice versa. This general approach has been supported elsewhere in the literature (cf., Knowlton, 2002). Knowlton, D.S. 80 Week One of the Discussion Cycle. Preservice teachers assigned to group one were responsible for describing a problem that they were experiencing within their partnership school. The assignment guidelines noted that the “problem might be interpersonal (e.g., a conflict with a mentor teacher or parent); instructional (e.g., students not meeting the objectives of a lesson); behavioral (e.g., a student who constantly ignores classroom rules); or contextual (e.g., a lack of adequate textbooks or other supplies).” This emphasis on a variety of problem types was purposeful. As I have noted, one goal of the assignment was to help the preservice teachers see the ways educational psychology was integrated with other “content,” such as cultural foundations of education, instructional methods, and classroom management. If the scope of the problems that students shared was broad, then opportunities for connections to content beyond educational psychology equally would be broad. Week Two of the Discussion Cycle. Preservice teachers in group two were responsible for using the index and table of contents of the educational psychology textbook to theoretically frame the problems that had been shared during week one. The textbook, then, became a learning-on-demand resource, where preservice teachers were self-selecting readings that would most likely contribute to an analysis of the problem-at-hand. In addition to making connections between the problem and educational psychology, preservice teachers were encouraged to make connections to content areas that were the basis of their weekly seminars. These connections were designed to help the preservice teachers understand that neither the problems that they encountered nor educational psychology were discrete. Instead relationships existed among problems encountered in classrooms, educational psychology, and other content areas. Week Three of the Discussion Cycle. All of the preservice teachers—regardless of whether they were assigned to group one or group two—were responsible for three contributions to the computer-mediated discussion. The assignment guidelines dictated that not all three contributions should be posted on the same day of the week. The purpose of this criterion was to build in reflection time for the preservice teachers. They were to consider the discussion in its entirety before adding further to the discussion. The assignment guidelines directed the preservice teachers’ efforts with week three contributions by noting that they should “further define and work to solve the problems-at-hand through dialogue.” They should “read what [their colleagues had written] within a ‘thread’ of discussion and interact by responding to [each other’s] ideas.” Assessment across the Discussion Cycle. The assignment guidelines noted that the preservice teachers would “receive most credit for the number of contributions that [they offered].” The rest of the credit would be earned by meeting the stated purpose of each week’s contribution. For example, a stated purpose of week two and week three contributions was that the preservice teachers should theoretically frame the problem with citations. The assignment guidelines did offer a caveat to this loose assessment structure, however: “After the first two cycles, if we all feel that we are doing more than ‘going through the motions,’ then the assessment criteria can stay equally ‘loose.’ That is, we all should be working as professionals to help classmates solve real problems. If, however, I sense—or we agree as a class—that the criteria are not rigorous enough to foster collaborative problem solving, I will [offer] additional criteria (for example, specifying the length of contributions) to enhance the educational potential of this assignment.” Knowlton, D.S. 81 C. Evaluation of the Second-Semester Design Two data sources served as a basis for evaluation. First, my assessment of the preservice teachers’ efforts served as a basis for determining additional changes that were needed to improve the efficacy of the assignment. Second, during a weekly seminar, I engaged the preservice teachers in a discussion about the use of CMC. Assessment as Evaluation Data. As I assessed the preservice teachers’ participation in the computer-mediated discussion, I made judgments about the design of the assignment itself. This approach of combining assessment with evaluation to determine the educational viability of CMC is not unprecedented. In fact, “only the integration of assessment [with] evaluation can produce a clear picture of an online discussion’s educational viability” (Knowlton, 2001, p. 164). Numerous observations suggested the need for additional change. First, I found that students were relatively successful at articulating problems that they were experiencing, but the problems were extremely narrow in scope. Of the approximately ninety posted problems, most focused on discipline problems among the K-12 students. One or two of the posted problems focused on interpersonal conflicts, such as conflicts with their mentor teachers or a parent. Two of the posted problems focused on instructional concerns. Second, most contributions during week three of the discussion were replies to the original problem posted during week one. In other words, the preservice teachers were not discussing the problems by interacting; they merely continued to offer solutions to the original problem. In fact, it often was unclear as to whether or not the preservice teachers were reading the threads of discussion in which they were responding. While repetition of various ideas across contributions within the same thread of discussion was common, consensus building and synthesis of ideas were scarce. Third, as I assessed the preservice teachers’ participation, I recognized the inordinate amount of time I was spending on grading, as opposed to engaging in activities that were related to assessment but more productive toward creating continued learning among students—such as reacting to their discussion contributions, highlighting common themes among their interactions, and offering contributions to the discussion as an authentic participant. Certainly, it was within my purview to grade the preservice teachers’ efforts, but grading should not dominate the assessment process (Bauer & Anderson, 2000). Input from Preservice Teachers. At the end of the second semester, I solicited input from the preservice teachers about the use of CMC. I used two questions as prompts to promote feedback from the preservice teachers. Notably, these prompts emerged as I assessed the preservice teachers’ success during the second semester: • How could the assignment guidelines be adjusted to emphasize the value of collaborative analysis and inquiry in an attempt to solve real-world problems? • How might the assignment guidelines be structured to foster an environment where the preservice teachers “forget” that CMC is being used as an “assignment” that will be graded and instead remember the need to act professionally and help their colleagues, even if that means offering more input than the minimum expectations require? During this discussion, two points emerged. First, the preservice teachers noted that many of them still usually were engaged in activities that did not directly relate to teaching. Certainly, they felt that by the third semester of their partnership experiences they would have shifted even more into a key role as the “teacher” of the class. This shift, they argued, would make it easier Knowlton, D.S. 82 for them to participate in the discussion because they would have richer experiences on which to base their contributions to the discussion. Second, the preservice teachers noted that criteria governing week two and week three contributions were hindering their participation. They urged me, for example, to reduce the number of required contributions in both weeks two and three. This would give them more time to research and find appropriate resources to support the perspectives that they offered within the discussion. One preservice teacher noted that there were so many contributions to each thread that there was nothing left to add for late-comers to the discussion; reducing the number of required contributions would solve this problem. Another preservice teacher agreed and noted that she did not read the threads before contributing because she did not want to know whether she was duplicating ideas that had already been offered. For similar reasons, students suggested the need to eliminate any criterion that specified on what days of the week they should participate. Once they planned their contribution they returned to the discussion board only to find that someone else had offered their idea. Also, several preservice teachers noted that they were printing out discussion contributions and sometimes even entire threads of discussion and reading them. So, while their actual contributions might come on a single day of the week, they were spending time considering the discussion across numerous days of the week. V. The Third Semester CMC Assignment The third-semester version of the assignment included several changes from the previous semester. Notably, these changes were based on feedback from the preservice teachers, which was reported in the previous section of this paper. In this section, I describe the milieu that contributed to the development of the third-semester assignment guidelines, the changes that were implemented, and evaluation. A. Factors Contributing to the Third-Semester Design The feedback that I had solicited from the preservice teachers did contribute to the prototyping of the third-semester design, but other factors contributed, as well. Notably, the format of the weekly seminars once again changed during the third semester of the partnership. It was determined that certain content areas—educational psychology being one such area—would not be given any formal emphasis during seminars. Yet, I was still accountable for assessing the preservice teachers and giving an Educational Psychology grade to each of them at semester’s end. Because of this dilemma between, on the one hand, needing to assess the preservice teachers and, on the other hand, not having formal seminar time to assess them, continuing to formally use CMC seemed appropriate. B. Characteristics of the Third-Semester CMC Guidelines The assignment was still based on the idea of the preservice teachers sharing real problems that they were experiencing and cycles of theoretically framing and solving those problems. Three changes to the assignment guidelines were made in an effort to overcome some of the weaknesses evident in the second-semester version. The first was an administrative change. The second was a change in the types of problems that the preservice teachers should Knowlton, D.S. 83 share. The third change concerned guidelines governing week two and week three discussion contributions. Administrative Change. During the second-semester version of the assignment, I had spent large amounts of time grading the preservice teachers’ contributions, as opposed to assessing and providing participants with the types of authentic feedback that could improve their problem-solving and analysis skills. To shift my own role within the discussion from a grader—which is often viewed as punitive—to a facilitator—which offers the opportunity to be more constructive—I developed a self-report form. At the end of each cycle of discussion, the preservice teachers completed and submitted this self-report, which allowed them to report factual information about their participation. For example, they could list the subject line of the threads in which they participated and cite the various resources that they used in theoretically framing a problem to which they responded. When the preservice teachers submitted their report, I had a list of threads in which I could find their contributions. This made the process of “grading” less time consuming. Changes to the Types of Problems Offered for Discussion. The third-semester version of the assignment guidelines required that all problems contributed to week one of the discussion cycle must be “instructional problems”—as opposed to the type of behavior and discipline problems that dominated the second semester. Specifying that the problems should be “instructional” in nature was designed to broaden the preservice teachers’ thinking regarding what constitutes a classroom problem that was worthy of analysis. Changes to Guidelines Governing Week Two and Week Three Contributions. I reduced the number of required contributions during week two of each cycle from three to two. The preservice teachers had advocated for the need to lower the number of required discussion contributions. They argued that lowering the quantity of required contributions would allow them to be more thorough in their analysis of the problems contributed to the discussion. While skeptical of such claims, I implemented this change in the hope that my skepticism would be unfounded. Also, I specified that week three contributions had to be replies to week two contributions, not replies to the original problem discussed during week one of each cycle. I hoped that this criterion would improve interaction among the preservice teachers within the computer-mediated discussion and promote a deeper analysis of the issues embedded within the problems, not just continued (and often redundant) “solutions” to the original problem. Because of this more specific purpose of week three contributions, I developed a list of possible strategies that the preservice teachers might use as they contributed to the discussion during week three. (See Table 2.) C. Evaluation of the Third-Semester Prototype Evaluation consisted mainly of the preservice teachers completing an open-ended survey about their views of using CMC. Space limitations prevent a full explication of the survey results. I focus here, though, on feedback that directly related to changes made in prototyping the final version of the CMC assignment. Changes Governing the Types of Problems Shared During Week One. Several preservice teachers noted that describing an instructional problem was more difficult than describing problems with student behavior or interpersonal conflicts. Many of them acknowledged, though, that being asked to describe instructional problems forced them to look at their own curriculum development and instructional practices in a more detailed way. As one preservice teacher noted, Knowlton, D.S. 84 Table 2: Strategies for replying to week three contributions As you write contributions to the discussing during week #3, you should work to add a deeper level of analysis to the discussion, not to simply repeat what has already been offered. If you are stuck for ways to contribute during week #3, consider the following possible strategies. • Pick two replies to the same problem and discuss why you think one would work better than the other. • Pick a reply to a problem and discuss the strengths and weaknesses of the proposed solution • Pick a theory that someone mentioned as a help to understanding week #2 and apply that theory differently (or more thoroughly). • Discuss your experiences with how a solution has/has not worked in the classroom. • Write a summary of responses to your own problem and describe what the biggest things that you are taking away from your problem are. “I was surprised that it was more difficult to relate an instructional problem to ed[ucational] psych[ology]. The behavior problems stood out more and the connections were more obvious. Because we had to share instructional problems, I think that I learned how interwoven ed[ucational] psych[ology] and teaching truly are.” Changes to Week Two and Three Contributions. The preservice teachers on average tended to agree that the changes to week two and three contributions were positive. One preservice teacher noted that the changes to the guidelines allowed her to “actually relate different theories and sources of information to the problems [that] others [were] experiencing.” Her point was that by being required to offer fewer contributions she could consider those contributions more carefully. Other students seemed to indicate that the suggestions for week three responses were useful. Many students noted that from these possible responses they realized that they could share their own experiences to a problem. One preservice teacher phrased it this way: “The most helpful responses were not the ones that said, ‘On page 276 of the text book, it states.…’ Instead, responses that described what [others] were doing in their classrooms to help with similar problems were . . . much more helpful.” From this and several similar comments, I infer that the suggestions for week three responses (as shown in Table 2) were useful to students in guiding them toward offering more salient contributions to the CMC discussion. VI. Implications of this Prototyping Approach In this paper, I have offered a case example of rapid prototyping as a design approach for developing a CMC assignment for the higher education classroom. The details of such an example should provide faculty members with new perspectives about the iterative nature of development processes. Specifically, several implications of this case cut across many higher education scenarios and are worthy of comment. Knowlton, D.S. 85 A. Instructional Strategies Influence Learning Noticeably absent from the case example is a discussion of the importance of customizing WebCT to improve the educational utility of the discussion. Instead of focusing on the prototyping of improved media, this case focuses on the development of instructional strategies. Such a focus is fully appropriate, as it is consistent with a view supported in the literature. Namely, instructional strategies, not computers, are the cause of learning (e.g., Clark, 1983, 1994a, 1994b). Admittedly, such a perspective is not without detractors (e.g., Kozma, 1991), but even these detractors agree that there is no credible evidence to suggest that computers influence learning. Professors who are using rapid prototyping to design media-based assignments would do well to consider their own philosophy among media, instructional strategies, and positive learning outcomes. If, in fact, computers do not influence learning, then prototyping should focus on strategy development more so than on media development. B. Consideration of Design Rigor For readers of this article who come from a traditional empirical background, this case example may have proved a frustrating read. No method of data collection and analysis was offered and applied, and no discussion of “significant” results was provided. But Edelson (2002) distinguishes between traditional research and design research. With this distinction comes a distinction in approach. For the professor who is interested in achieving a level of understanding to justify change within a course or assignment, empirical rigor is not needed, and may even be misleading. Instead, pedagogical rigor can provide insights sufficient to adjust assignments so that they promote a stronger opportunity for learning among students. The point is that faculty members across disciplines should gravitate toward design processes that allow for functional revision of assignments, even if such gravitation limits one’s ability to publish more scientific claims that are indicative of traditional research. C. Nature of Systematic Design This third implication builds largely from the second. To suggest that empirical rigor indicative of the positivist research paradigm is unnecessary is not to suggest that design is haphazard and non-systematic. The case example noted here serves as a worthy model for professors across disciplines because it illustrates the relationship between the prototyping process and a dependence on inputs and outputs, which is one characteristic of “systematic” design. Consideration was given, for example, to the macrocontext of the field-based program. Consideration was also given to the changing needs of the preservice teachers. As professors adopt rapid prototyping procedures, they, too, should consider the role of context as a factor that influences and informs their design. As a learning context evolves, design practices must become increasingly iterative and flexible. Furthermore, the evaluation of design is one unique stage of the design process that is particularly dependent on inputs and outputs. Professors who are prototyping assignments across semesters or even within a single semester should plan for evaluating the quality of their own designs. As can be noted within the case described in this paper, the professor’s judgment was involved in evaluating the assignment, but the prototyping of the assignments did not stand on the professor’s judgment alone. Student input was a part of the evaluation process and the assignment was prototyped—at least to some extent—based on that student input. Within the Knowlton, D.S. 86 case reported in this paper, perhaps I had an obvious advantage in that my students (i.e., the preservice teachers) remained the same across each semester of implementation. This allowed me to develop a rapport with them, and they perhaps felt more invested in assisting with the prototyping of the strategies, since they knew that they would be engaged in CMC discussion in future semesters. Professors who do not have such an advantage might have to go to greater lengths to account for student input as they are prototyping assignments. The use of additional formal surveys, focus groups, or other opportunities for student input may be useful to this end. References Bauer, John F., and Anderson, Rebecca S. (2000). Evaluating students’ written performance in the online classroom. In Renee E. Weiss, Dave S. Knowlton, and Bruce W. Speck (Eds.), Principles of effective teaching in the online classroom (pp.65-72). San Francisco: Jossey-Bass. Beckett Julia, and Grant, Nancy K. (2003). Guiding students toward solutions in field experiences. In Dave S. Knowlton & David C. Sharp (Eds.), Problem-Based Learning for the Information Age, (pp. 67-72). San Francisco: Jossey-Bass. Clark, Richard E. (1983). “Reconsidering Research on Learning from Media.” Review of Educational Research, Vol. 53, No. 4, pp. 445-459. Clark, Richard E. (1994a). “Media Will Never Influence Learning.” Educational Technology Research and Development, Vol. 42, No. 2, pp. 21-29. Clark, Richard E. (1994b). “Media and Method.” Educational Technology Research and Development, Vol. 42, No. 3, pp. 7-10. Collins, Allan, Joseph, Diana, and Bielaczyc, Katerine (2004). “Design Research: Theoretical and Methodological Issues.” The Journal of the Learning Sciences, Vol. 13, No. 1, pp. 15-42. Edelson, Daniel C. (2002). “Design Research: What We Learn When We Engage in Design.” The Journal of the Learning Sciences, Vol. 11, No. 1, pp. 105-121. Eggen, Paul, and Kauchak, Donald (1997). Educational Psychology, 5th Ed. Columbus, OH: Prentice Hall. Jones, Toni S. and Richey, Rita C. (2000). “Rapid Prototyping Methodology in Action: A Developmental Study.” Educational Technology Research and Development, Vol. 48, No. 2, pp. 63-80. Knowlton, Dave S. (2001). “Determining Educational Viability in Online Discussions: A Student-Centered Approach.” Academic Exchange Quarterly, Vol. 5, No. 4, pp. 162-168. Knowlton, Dave S. (2002). “Promoting Liberal Arts Thinking through Online Discussion: A Practical Application and Its Theoretical Basis.” Educational Technology & Society, Vol. 5, No. 3, pp. 189-194. Knowlton, D.S. 87 Knowlton, Dave S., Knowlton, Heather M., and Davis, Camela. (2000). “The Whys and Hows of Online Discussions.” Syllabus: New Directions in Educational Technology, Vol. 13, No. 10, pp. 54-58. Kozma, Robert B. (1991). “Learning with Media.” Review of Educational Research, Vol. 61, No. 2, pp. 179-211. Lohr, Linda, Javeri, Manisha, Mahoney, Chris, Gall, Jim, Li, Kathy, and Strongin, Dawn (2003). “Using Rapid Application Development to Improve Usability of a Preservice Teacher Technology Course.” Educational Technology Research and Development, Vol. 51, No. 2, pp. 41-55. Reiser, Robert A. (2001). “A History of Instructional Design and Technology: Part 2: A History of Instructional Design.” Educational Technology Research and Development, Vol. 49, No. 2, pp. 57- 67. Scanlon, Patricia A., and Ford, Michael P. (1998). Grading Student Performance in Real-World Settings. In Rebecca S. Anderson and Bruce W. Speck (Eds.). Changing the way we grade student performance: Classroom assessment and the new learning paradigm, (pp. 97-105). San Francisco: Jossey-Bass. Weber, Alan (1996). “Professional Development Schools and University Laboratory Schools: Is There a Difference?” The Professional Educator, Vol. 18, No. 2, pp. 59-65. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006, pp. 88 – 109. Developing the Scholarship of Teaching Through Transformative Learning Carolin Kreber1 Abstract: Following a cognitive-developmental perspective, the Scholarship of Teaching & Learning is understood as a process of knowledge construction whereby knowledge claims are validated through reflection on teaching experience and educational theory. These reflective processes can be documented and peer reviewed. Teaching portfolios allow for the documentation of indicators of reflection. Indicators can be developed for each of three domains of teaching knowledge: (1) what we consider to be meaningful goals and purposes of higher education; (2) what we know about student learning and development in relation to these goals; and (3) what we know about the teaching and instructional design processes needed to bring about academic learning and development. Keywords: Learning about teaching, transformative learning, reflection, professionalism and citizenship, scholarship, documentation and peer review I. Introduction: The scholarship of teaching as professionalism guided by citizenship “We develop a scholarship of teaching when our work as teachers becomes public, peer- reviewed and critiqued. And exchanged with members of our professional communities so they, in turn, can build on our work” (Shulman, 2000). Far from having remained “an amorphous term, equated more with commitment to teaching than with any concrete, substantive sense of definition or consensus as to how this scholarship can be recognized” (Menges & Weimer, 1996, p.xii), the scholarship of teaching and learning has gained much clearer contours and recognition in recent years (Kreber, 2003). Often linked to the notion of professionalism in university teaching, the scholarship of teaching and learning is progressively associated with a form of knowledge about teaching and student learning that can be rationally verified through disciplined inquiry. “Professional knowledge” thus construed is knowledge oriented towards “best practices”. While the question of what constitutes “best practices”, ultimately, is a philosophical one, there remains little doubt that we can observe a trend in the educational policy arena to equate the idea of “best practices” increasingly with notions of effectiveness and efficiency. Applied to the scholarship of teaching and learning, professional practices (or “best practices”), then, are identified by exploring the question “which teaching innovations produce the best results (i.e., more learning, better/deeper learning, or a closer fit of learning outcomes with those required by the job market, etc)?”. No one would dispute that this is a significant question to delve into; however, it is just one question that the scholarship of teaching is (or should be) concerned with. A second way of exploring university teaching in a scholarly (or if you will professional) way is to turn to its moral and civic purposes. Thus construed, the scholarship of teaching (and 1 Director of The Centre for Teaching, Learning and Assessment, Moray House School of Education, Paterson’s Land, University of Edinburgh, UK. Questions and comments may be emailed to carolin.kreber@ed.ac.uk. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 89 learning), or professionalism in teaching, is more appropriately associated with the notion of citizenship rather than with “effectiveness or efficiency”. While understanding how students learn and finding ways to optimize their learning, preferably through replicable and publishable forms of scientific inquiry, is clearly important, this alone cannot be the essence of the scholarship of , or professionalism in, teaching. “Best (teaching) practices”, therefore, are no more than the means by which to bring about desired educational outcomes. And so a question to be addressed early on in any deliberations on university teaching ought to be “what are the ends that the contemporary university serves through its teaching practices and curriculum?” and, more importantly, “are these the same that it should serve”? Are we concerned with training and socializing researchers into our discipline, preparing students for specific jobs or for lifelong learning, facilitating their personal development, promoting their successful participation in a democratic society, or perhaps none, or all, of these? Analyses of these latter questions are just as essential aspects of practicing the scholarship of teaching as are explorations of how well certain teaching methods work and how, or how well, students learn (Hutchings & Shulman, 1999). The scholarship of teaching and learning, or professionalism in teaching, therefore, needs to be conceptualized broadly and integrate the notion of professionalism with the notion of citizenship (see also Walker, 2001). By asking (1) what do we consider to be meaningful goals and purposes of higher education, (2) what do we know about student learning and development in relation to these goals, and (3) how can we promote such learning and development (Kreber & Cranton, 2000; Kreber, 1999), the scholarship of teaching and learning could lead to changes that go beyond the development and implementation of instructional innovations but are expressed also in the larger curriculum and co-curriculum (Kreber, 2005a) I am stating what is obvious to everyone, of course. Certainly goals are important and no one would dispute this. In recent years, many scholars have highlighted the university’s role in promoting moral and civic education (e.g., Colby, Ehrlich, Beaumont, & Stephens, 2003; Lempert, 1996; Piper, 2002; Orr, 1993; Rhoads, 2000) and numerous higher education policy documents directly speak to the importance of these goals (e.g., DfES, 2003; World Conference of Higher Education, 1998; National Panel Report, 2002). More over, the focus for the 2003- 2004 cohort of Higher Education Carnegie Scholars organized by the Carnegie Academy for the Scholarship of Teaching and Learning, was on Liberal Learning. The Carnegie Foundation also recently initiated the “Political Engagement Project” to address the problem of young people increasingly disengaging from politics. This initiative builds on the earlier work by Colby et al. (2003) on moral and civic responsibility. Clearly, all these projects demonstrate careful consideration of the goals and outcomes of higher education and a concern with student learning that goes beyond the development of discipline-experts, or training for specific jobs. However, the links between these moral and civic goals and the scholarship of teaching and learning, though possibly assumed by many, has not always been made explicit. Even though many have discussed the importance of educational goals and purposes over the years, until recently these discussions occurred largely outside rather than within the discourse on the scholarship of teaching and learning (by which I mean a discourse found in SofTL specific journals and conferences). I suggest that it is perhaps for this reason that many faculty and administrators associate the scholarship of teaching and learning still primarily with the notion of “best (teaching) practices” rather than a broader notion of professionalism that would integrate the idea of “citizenship”. Though the latter is possibly taken for granted by some, it still occupies Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 90 somewhat of a secondary role in conceptions of what the scholarship of teaching and learning is, should, or could be. In this article, then, I suggest that the scholarship of teaching and learning involve (1) careful consideration of educational goals and purposes suitable for addressing the various political, social, cultural, environmental and economic challenges of our times, (2) understanding how students learn and develop toward these and other academic goals, and (3) identifying ways to best facilitate this learning and developmental process. I further contend that the notion of the scholarship of teaching and learning implies that we approach our teaching practice with the same sense of skepticism that guides our research. As researchers, we habitually provide arguments or reasons for our assertions. Depending on our discipline or subject area, we engage in the process of hypothesis testing, interpretation or critical analysis routinely. Moreover, we recognize that it is important to share with colleagues the evidence we generated for our point of view and invite them to follow, and possibly critique, our lines of argumentation. It has been proposed that we engage in similar processes with respect to our teaching; however, traditional ways of sharing such as conference papers and journal publications are but two of several possibilities. Later in this paper I will discuss how and why teaching portfolios are particularly suitable for this purpose. In the remainder of this article I will build on these observations and discuss how the scholarship of teaching and learning may be developed through transformative learning (Mezirow, 1991), a process by which faculty construct knowledge about teaching and learning through reflection. In line with the earlier arguments, I suggest that faculty construct knowledge in three different domains. The first domain of knowledge relates to what we consider to be meaningful goals and purposes of higher education (Curricular Knowledge). The second refers to what we know about student learning and development in relation to these goals (Pedagogical Knowledge, or perhaps more appropriately referred to as Psychological knowledge). The third pertains to what we know about the teaching and instructional design processes needed to bring about student learning and development (Instructional Knowledge). Questions that will be examined in this article include: • How is reflection on teaching and learning valuable? • What role do experience and theory play in reflection on teaching? • What is transformative learning? • Are there different kinds or levels of reflection, and if so, are all levels of reflection equally conducive to fostering change and development in higher education teachers? • How is reflection, and transformative learning, linked to the scholarship of teaching? • How can transformative learning on teaching and student learning be demonstrated and reviewed? II. How is reflection on teaching and learning valuable? For more than two decades researchers have explored the role of reflection in teacher training (e.g., Hatton & Smith, 1995; Calderhead, 1989; Zeichner et al., 1987) and more recently, it has become a buzzword also in higher education. Time and again faculty are reminded of the importance of reflecting on their teaching (e.g., Brookfield, 1991, 1995; Cranton, 1998; Ramsden, 1992; Schön, 1995), though the process of reflection itself remains poorly understood Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 91 (e.g., Moon, 2000). Reflection also has been identified as a key process in the scholarship of teaching and learning. Andresen (2000), for example, argued that the scholarship of teaching, should be inquiry- driven, involve critical reflectivity, and scrutiny by peers. Two other studies, one with “experts” in the scholarship of teaching (Kreber, 2002b) and one with “regular faculty” (Kreber, 2003) showed that both “experts” and regular academic staff consider the scholarship of teaching to be inquiry driven and to involve critical reflection. Trigwell et al. (2000) identified five qualitatively different conceptions of the scholarship of teaching in a study with faculty at an Australian university following the research tradition of phenomenography (Marton 1981). These five conceptions were shown to differ in terms of four dimensions, one of them being the focus that reflection on teaching can take. Finally, Glassick et al (1997) proposed reflective critique as one of six criteria or standards by which to judge the scholarship of teaching. Clearly, reflection is recognized as an important aspect of the scholarship of teaching. However, just what precisely it is that we hope reflection will accomplish is not always made clear. At the same time there perseveres a deeply-held belief that through reflection we can enhance our teaching practice, and by extension, the learning experiences of our students. Such conclusions presuppose that reflection will lead to valid and valuable forms of knowing. For if the outcomes of reflection on teaching were not assumed to be valid, how would such reflection be meaningful? It is this idea of checking whether what we think actually makes sense, or is “valid”, given the context we find ourselves in, that is at the core of Mezirow’s (1991) theory of transformative learning. In emphasizing the importance of validity testing, Mezirow is inspired by the work of German sociologist Jürgen Habermas. Borrowing heavily from Habermas’s (1971) theory of knowledge-constitutive interests, Mezirow distinguishes three different forms of learning: instrumental, communicative and emancipatory. It is through these three forms of learning that we can test the validity of our assumptions. A.Three different forms of learning When engaged in instrumental learning we verify a belief or assertion by subjecting it to the empirical-analytical method; that is by posing it as a hypothesis that then can be tested by gathering data that will either support or contradict it. Communicative learning, on the other hand, relies on the notion that through communication with others we can reach a common understanding on what is true. An assertion or belief is valid, therefore, if agreement on it can be reached within a community. Such a consensus then is based on what the community at some point has accepted as the norm. While communicative learning is very valuable if the goal is to reach greater understanding within a framework of given norms, this form of learning does not concern itself with the question of how these norms have come about. Put differently, through communicative learning we do not ask “why did we ever conclude that things should be this way?” or “Why does it matter that we do them this way?” In response to the limits of communicative learning, Mezirow (1991), leaning on Habermas (1971), suggests that important aspects of learning do not occur on the basis of subjective understanding and consensus within a given social context. Instead the most significant forms of learning involve a critical analysis of the processes and conditions by which certain norms we have come to take for granted have evolved and how “consensus” was reached. This is the nature of emancipatory learning. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 92 Whether or not the assumptions or conceptions we hold about university teaching are valid, therefore, can be determined through instrumental, communicative, or emancipatory learning or any combination of these. III. What role do experience and theory play in reflection on teaching? When we think of reflection on teaching, we typically have in mind teachers reflecting on their personal teaching experiences rather than on research findings (see also Hiebert, Gallimore & Stigler, 2002; Huberman, 1985). This notion is supported by an extensive body of literature which argues that instructors who reflect on their teaching experiences acquire knowledge that is useful to them in the contexts in which they teach (e.g., Cochran-Smith & Lytle, 1990; Munby & Russell, 1994; Schön, 1983). Moreover, these scholars suggest that the teachers’ personal knowledge, constructed on the basis of teaching experience, is more valuable than theoretical or research-based knowledge on teaching. Theoretical knowledge about teaching, such as that found in books and academic journals, is, according to these scholars, more or less irrelevant or worthless as it cannot be directly applied to practice. An intriguing question, however, is whether the reflective process might also be directed to theoretical knowledge about teaching, and if so, under what circumstances would theoretical knowledge be of value to teachers? Norris (2001) suggests that the value of educational theory for the practice of teaching depends on how teachers engage with theory. Educational theories, he argues, surely will not seem particularly useful to teachers if they are wrongly expected to serve as situational or context-specific problem-solving strategies. This cannot be the purpose of any theory. Instead, teachers need to understand the value of theories as “general models” which they need to adapt to their specific context. Whether and, if so, how such research-based knowledge applies to a teacher’s given situation is a question that only those who know the particulars of the situation can answer. “When the situation is the classroom, teachers know the most about them” (Norris, 2001). Hiebert et al. (2002) also emphasize the importance of local hypotheses that teachers develop and test across specific contexts thereby working in collaboration with researchers “to digest and transform their general findings into professional knowledge for teaching “ (p.13). It follows that while reflection is certainly associated with experience (see also Boud, Keoch & Walker, 1985; Dewey, 1991; Kelly, 1955; Kolb, 1984; Mezirow, 1991; Moon, 2000) it also plays a significant role in determining the usefulness of theoretical or research-based knowledge. Jarvis (1999) summed it up most succinctly when he argued that theories serve as information that practitioners need to transform into situation- specific knowledge as they try them out in practice. In doing so they create valid knowledge. The view that there are at least two equally important sources of reflection on teaching – educational theory and teaching experience--, has been repeatedly articulated also in the higher education teaching and learning literature (e.g., Kreber, 2002; Kreber & Cranton, 2000; Menges & Weimer, 1996; Paulsen, 2001; Rando & Menges, 1991). As Rando and Menges (1991) suggested more than a decade ago: “articulating a rationale for one’s instructional world…requires reflection about personal theories, knowledge of formal theories, and blending of the personal and formal” (pp. 13-14). While it is indeed important to consider both experience and theory in the discussion of reflection on teaching, doing so does not in and of itself address the question of how reflection enhances the practice of teaching. As we have just seen, Jarvis proposed that theories are validated through practical experience but the idea of validation would benefit from a more thorough analysis. As discussed above, Mezirow (1991) suggests that we come to know things as being either “true” or “false” through instrumental, Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 93 communicative, and/or emancipatory learning. These forms of learning are linked to different kinds of reflection. We will turn to these different forms of learning and kinds of reflection next. IV. What is transformative learning? Mezirow’s theory of transformative learning, to a large extent, is informed by the cognitive-developmental tradition (e.g., Kelly, 1955; Piaget, 1964). This particular strand of psychology assumes that individuals develop intellectually as they encounter events that cannot be interpreted through their existing mental frames of reference. Intellectual development occurs as frames of reference get revised as a result of reflection. A frame of reference, often called a conceptual structure, is interpreted as an interrelated set of assumptions, constructs, or conceptions, individuals actively form through experience. As specific assumptions are questioned and possibly revised (or transformed) in light of contradicting evidence, this can lead to a more substantial change in frame of reference (or “conceptual structure”). Since individual assumptions are understood to be hierarchically organized, it would depend on their position or relative importance within the larger frame of reference, whether a revised assumption leads to a transformation in the frame of reference itself. To illustrate this point by means of an example, think of a new faculty member whose assumption that all students would prepare the readings assigned for class is challenged early in the semester when she notices that this holds true for only about 50 % of students. Clearly, she now realizes that undergraduate students are not equally ready to assume responsibility and control over their learning. Whether or not the transformation of this one assumption (or conception) will actually promote a more substantial transformation in frame of reference (or conceptual structure) will depend on the importance she attributes to this new knowledge. It would be possible, for instance, that she begins to question related assumptions and reflects on how she could better facilitate the process of self-regulated learning for different students, and whether, and if so, why, it matters that students learn to take on more responsibility for their learning. Since, as was noted, the assumptions (or conceptions) we hold about teaching and learning are in some ways interrelated, a transformation of one assumption may promote reflection on other assumptions. Not in all cases, however, will reflection lead to a drastic change in frame of reference for, through reflection, we may also find our assumptions to be confirmed or validated. Let us now look at the different forms “reflection” (as conceived by Mezirow) can take. V. Are there different kinds or levels of reflection, and if so, are all levels of reflection equally conducive to fostering change and development in higher education teachers? In distinguishing between different kinds of reflection, Mezirow put clearer parameters on the rather vague term “reflection” and, hence, made a significant contribution to our understanding of the reflective process and teacher thinking. The three kinds of reflection Mezirow identified are content, process, and premise reflection. He describes the differences between the three forms of reflection as such: The critique of premises or presuppositions pertains to problem posing as distinct from problem solving. Problem posing involves making a taken-for-granted situation problematic, raising questions regarding its validity. … the term “critical reflection” often has been used as a synonym for reflection on premises as distinct from reflection on Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 94 assumptions pertaining to the content or process of problem solving (Mezirow, 1991, p.105). It follows that content, process and premise reflection are very different in terms of their nature and what they can achieve; indeed, one could say that they represent different levels of reflection. The term content reflection is at times confusing to people, particularly when discussed in the context of teaching and learning. Contrary to our intuitive understanding, the term “content reflection” does not refer to reflection on the content of the courses we teach. What Mezirow means by content reflection is having a clear sense of, and providing a description of, the content of the problem that we need to solve. In short, content reflection asks “What’s really the problem here and what do I need to do?”. In content reflection, we do not question the presuppositions underlying our argument but simply use our present knowledge, that is the assumptions or conceptions we presently hold, to describe a problem and how it is habitually solved by us. According to Mezirow, content reflection, is a process in which we “are not attending to the grounds or justification for our beliefs but are simply using our beliefs to make an interpretation” (Mezirow, 1991, p.107). To be clear, the question of whether our knowledge is valid is not one posed by content reflection. All we ask through content reflection is “what do I presently know about how to solve this problem?” Process reflection, on the other hand, is focused on the effectiveness of the problem- solving strategy itself. Here we ask “how do I know that I am effective (or was conscientious) with what I do?”. Finally, in premise reflection, we call into question the presuppositions on which our present knowledge is based and ask “why is it that I choose to attend to this problem— is there an alternative?” How these forms of reflection are linked to the three forms of learning discussed earlier-- instrumental, communicative and emancipatory-- is illustrated next. It has become evident that content reflection does not address the question of validity of the outcomes of reflection. Through process and premise reflection, however, we test the validity of our assumptions or conceptions. In process reflection we find out whether what we do works by seeking some form of evidence for its “effectiveness” (which in some case is better interpreted as “meaningfulness” or “conscientiousness”). This evidence might be found through published research we read about, research we conduct ourselves, or through experience such as talking to others. Reflection then can be informed by the two sources of knowledge construction discussed earlier: personal teaching experience and educational theory. Process reflection then occurs through either instrumental or communicative forms of learning, or both. In case of instrumental learning we might validate our knowledge by posing it as a hypothesis that we then test (for example, “students achieve better test scores if I give them the opportunity to choose between two assignments”). In case of communicative learning, we might validate our knowledge as we discuss what we assume to be true with a community of peers to achieve consensus (for example, as we discuss the meaningfulness and relevance of certain goals or values which guide our curriculum planning). Obviously, it is only when we are engaged in premise reflection, that is the questioning of presuppositions of what we believed to be true, that our learning would become also emancipatory. We may question, for example, why we ever decided on certain goals and values and examine the processes and conditions by which these came about. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 95 If our goal is to enhance university teaching, it is vital that we encourage also process and premise reflection rather than just content reflection on teaching. Let us now explore what it might look like when people engage in content, process or premise reflection on teaching. V. How is reflection, and transformative learning, linked to the scholarship of teaching? Kreber and Cranton (1997, 2000) suggest that the scholarship of teaching and learning involves learning about three equally important domains of teaching knowledge: (1) knowledge about the goals and purposes of our teaching (labelled curricular knowledge), (2) knowledge about how students learn (back then was labelled pedagogical knowledge but should perhaps have been labelled psychological knowledge), and (3) knowledge about instructional design and the instructional process (labelled instructional knowledge). This taxonomy is not unrelated to other models describing the knowledge base of teaching (e.g., Rice, 1991; Shulman, 1987). The main difference is that the SofT model is not limited to identifying knowledge domains but is concerned with the construction of knowledge, through reflection, in each domain. Another important aspect of the model is that it stresses the critical examination of goals and purposes of higher education as an integral part of the Scholarship of Teaching and Learning. Mezirow’s three levels of reflection (on content, process and premise) serve to explain how faculty might construct knowledge in the domains of curriculum (what are the goals and purposes of our teaching?), pedagogy (how do students learn and develop toward these goals?), and instruction (what can be done to optimize this learning or developmental process?). It follows that individuals may actually be involved in as many as nine different kinds of reflection (namely content, process and premise reflection in the three domains of instructional, pedagogical and curricular knowledge) with each of the nine kinds of reflection generating a slightly different form of teaching knowledge. In summary, as higher education teachers are involved in any of these nine forms of reflection, they draw on their teaching experience or educational research, or both. Clearly then, reflection, whether informed by experience or theory, leads to knowing, indeed, as many argue, is a process of active knowledge construction. The knowledge higher education teachers construct through these forms of reflection can be tested for its validity through instrumental, communicative or emancipatory learning processes. When our assumptions become validated as a result of process or premise reflection, we can present some sound arguments by which to justify our practice. To borrow Dewey’s (1933) words, through reflection we carefully considered “any belief or supposed form of knowledge in light of the grounds that support it and the further conclusions to which it tends” (p.9). When individual assumptions were not validated as a result of reflection, transformative learning (a revision of assumptions or conceptions) might occur as a result. Premise or critical reflection on a single assumption may or may not lead to a transformation of one’s larger frame of reference (or conceptual structure) on teaching2. Table 1 summarizes what has been argued and provides some concrete examples of possible responses to the reflective questions posed by the scholarship of teaching (SofT) model. Tables 2 to 4 illustrate the process of content, process 2 It might be useful to think of the conceptual structure as something similar to Prosser and Trigwell’s (1999) “conceptions of teaching” (see also Martin & Balla, 1991; Ramsden, 1992; Kember & Gow, 1994), which can be predominantly teacher-focussed or increasingly more student-focussed and oriented towards promoting students’ conceptual change in the understanding of subject matter. For instructors to change their conceptions, transforming one single assumption may or may not trigger a more drastic change or transformation in “conceptions of teaching”. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 96 and premise reflection for the domains of curricular knowledge, pedagogical (or rather psychological) knowledge and instructional knowledge respectively. Clearly, the goals we identify as a result of reflection within the domain of curricular knowledge influence the reflective processes in the other two domains. The three goals discussed in the tables --self- management, autonomy and social responsibility-- are examples only (though they are, of course, consistent with the understanding of the scholarship of teaching and learning as a “professionalism” that is inclusive of the notion of “citizenship”). Obviously there are other important goals of higher education including those that are more subject, discipline or program specific. An essential aspect of the SofT model is its emphasis on justifying educational goals through process and premise reflection. Following this model, faculty can provide evidence of engagement in the scholarship of teaching. This involves demonstrating that we made efforts to validate our knowledge of teaching in learning in the three knowledge domains, and acted on the results of our reflection. Given appropriate criteria, it could also be assessed through a process of peer review (Kreber, 1998; 2001). How the various reflective processes could be demonstrated in a teaching portfolio I will discuss below. VII. How can transformative learning on teaching and student learning be demonstrated and reviewed? The idea of the teaching portfolio originated in Canada in the early 1970s (Knapper, McFarlane, & Scanlon, 1972) and later resulted in a publication sponsored by the Canadian Association of University Teachers (CAUT), entitled "The teaching dossier: A guide to its preparation and use" (Shore and others, 1980, 1986). Teaching portfolios are meant to have both formative and summative evaluative purposes (Edgerton, Hutchings, & Quinlan, 1991; Knapper, 1995; Smith, 1995). This is to say that they are meant to promote teacher growth and provide a basis for judging teaching effectiveness. Typically, teaching portfolios include various documents among them a philosophy statement, outlines of courses taught, unsolicited comments from students, written feedback from colleagues, examples of course work completed by students, summary of teaching evaluation from students, and so forth. Evidently, sources such as these, particularly if compiled together, provide a broader and more objective picture of teaching effectiveness than, for example, student ratings of instruction alone. Clearly, teaching portfolios can be very useful in demonstrating teaching effectiveness to an evaluation committee. However, it is less obvious how teaching portfolios thus construed serve their formative purpose. Most teaching philosophy statements that I have read over the years provide “thick” descriptions (e.g., Lincoln & Guba, 1985) of present practices but demonstrate little evidence of critical reflection on the underlying assumptions themselves. These descriptions are good examples of what is meant by “content reflection”. Surely, this does not necessarily mean that faculty are not critically reflective; it is equally possible that many simply do not know how to demonstrate their reflection. My purpose in this essay is to show how teaching portfolios could be used to document engagement in the various reflective processes associated with the SofT model. The portfolio then could be a means not only for stating the assumptions we hold about instructional design issue, student learning and development and goals and purposes but also for documenting the processes by which assumptions were constructed. The basic idea behind such a portfolio is that both authors and reviewers can form judgements regarding the validity of the outcomes of reflection by exploring and assessing the extent to which stated assumptions are the result of Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 97 Table 1 The Scholarship of Teaching (SofT) Model (adapted by Kreber & Cranton, 2000): Content, process and premise reflection in the three knowledge domains (Examples of possible responses) Curricular knowledge Pedogogical/Psycholog ical knowledge Instructional knowledge Validity testing involved? Form of learning Sources of knowledge used in reflection Content reflection “What are the goals and purposes of my teaching?” “What do I know about how students learn and develop?” “What instructional strategies should I use?” The problem is described “The problem I need to solve here is clarifying my goals.” “The problem I need to solve here is how to promote moral development and social responsibility.” “The problem I need to solve here is (for example) how to provide students with real-life problems.” I make explicit what I already “know” (what I believe to be true) no challenging of assumptions is taking place at this point. Largely experience-based but could also be research-based Possible habitual response “My main goal is to promote in students a greater sense of moral development and social responsibility.” “I can do this by providing them with opportunities to solve real-life ill-structured problems.” “I can do this by incorporating a service learning component.” NO Process reflection “How conscientious have I been in identifying this goal?” “How effective am I in learning how to promote moral development and social responsibility?” “How effective has my use of service learning been in providing students with opportunity to solve real-life problems?” YES Instrumental and/or communicative Experience-based and/or research- based Premise reflection “Why does my goal of promoting social responsibility matter – what possible alternatives are there?” “Why does it matter that I offer opportunities to solve real-life and ill- structured problems – what possible alternatives are there?” “Why does it matter that I use this approach (here service learning). Is there an alternative? YES Emancipatory (possibly preceded by instrumental and/or communicative learning) Experience-based and/or research- based Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 98 Table 2 Examples of possible responses to content, process and premise reflection questions in the domain of curricular knowledge. First goal Second goal Third goal Content reflection “What are my goals” Process reflection “How effective or conscientious have I been in identifying these goals” or “How did these goals come about”? Premise reflection “Why does it matter that I promote these goals?” “My goal is to promote self- management.” “The need for students to develop the skills, abilities and attitudes needed for continuous learning has been repeatedly emphasized in the educational literature.” “While there are other important goals, students need to acquire the capacity to engage in continuous adaptive learning because such learning has become a reality in our rapidly changing world. Without this capacity individuals will be seriously limited in their opportunity to make changes necessary to improve their lives.” “My goal is to promote personal autonomy” “Philosophers, educators and social critics have identified the cultivation of autonomous individuals as an important purpose of higher education.” “While there are other important goals, students need to be able to distinguish mere habit and convention from what they can defend by argument in order to solve the most pressing problems of our times. Without this ability individuals easily become puppets and automatons of hegemonic forces that take control of their lives.” “My goal is to promote social responsibility” “People arguing that higher education should bring about autonomous thinkers also emphasize the need for responsible citizens. Carnegie identified encouraging civic engagement as an important educational purpose.” “While there are other important goals, students need to develop a sense of responsibility towards the community and the environment because our most pressing problems can be solved only by recognizing that we are in this together. Without this sense of stewardship and citizenship we open the door to conflict as marginalized groups and social and environmental issues easily get ignored.” Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 99 Table 3 General questions asked in content, process and premise reflection in the domain of pedagogical (or “psychological”) knowledge. Self-management Autonomy Social Responsibility Content reflection Process reflection Premise reflection “What do I know about how students develop a sense of self-management?” “How do I know that what I believe about how to promote self-management (for example, promote self- regulated learning, deep level approaches, self-evaluation, information finding, etc), is actually true?” and “How effective have I been in identifying, or learning, about how students develop self- management?” “Why does it matter that I focus on these constructs, ideas or theories to help students develop self- management – what possible alternatives are there?” “What do I know about how students develop a sense of autonomy?” “How do I know that what I believe about how to promote autonomy (for example, promoting intellectual development, critical thinking, exercising freedom of choice, etc) is actually true?” “Why does it matter that I focus on these constructs, ideas or theories to help students develop autonomy -- what possible alternatives are there?” “What do I know about how students develop a sense of social responsibility?” “How do I know that what I believe about how to promote social responsibility (for example, promoting moral development, social commitments, cultural and environmental sensitivity, etc) is actually true?” “Why does it matter that I focus on these constructs, ideas or theories to help students develop social responsibility -- what possible alternatives are there?” Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 100 Table 4 General questions asked in content, process and premise reflection in the domain of instructional knowledge. Self-management For example, linked to Self-regulated learning Deep level learning Self-evaluation Finding information Autonomy For example, linked to Critical thinking Deep level learning Intellectual development Exercising freedom of choice Social Responsibility For example, linked to Moral development Cultural and environmental sensitivity Content reflection Process reflection Premise reflection “What do I know about the strategies needed to help students develop a sense of self-management?” “How do I know that these strategies are effective?” “Why does it matter that I use these strategies -- what possible alternatives are there?” “What do I know about the strategies needed to help students develop a sense of autonomy?” “How do I know that these strategies are effective?” Why does it matter that I use these strategies -- what possible alternatives are there?” “What do I know about the strategies needed to help students develop a sense of social responsibility?” “How do I know that these strategies are effective?” Why does it matter that I use these strategies -- what possible alternatives are there?” Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 101 engagement in process and premise reflection, or instrumental, communicative and emancipatory learning. In order for us to demonstrate our reflection on the various aspects of the scholarship of teaching model, it is not enough to tell others that we reflected. Demonstrating reflection more convincingly, would involve providing indicators of engagement in the reflective processes underlying the scholarship of teaching and learning (here the SofT model). Indicators of engagement in the scholarship of teaching are concrete actions we took from which engagement in the various reflective learning processes can be inferred. While Kreber and Cranton (2000) proposed a list of indicators based on a conceptual analysis, a recent study with thirty-six science faculty (Kreber, 2005) identified indicators empirically. The list below draws on both records but makes some additional suggestions. The items in the list are concrete actions faculty can take and also describe and document in the portfolio. 1. Describing the instructional strategies one uses (content reflection/instructional knowledge--experience-based); 2. Asking for peer review of course outline (process reflection/instructional knowledge--experience-based); 3. Collecting data on students' perceptions of methods and materials (process reflection/instructional knowledge—experience-based); 4. Experimenting with alternative teaching approaches and checking out results (premise reflection/instructional knowledge—experience-based); 5. Comparing different instructional strategies for their suitability in a given context (premise reflection/instructional knowledge—experience-based) 6. Paying attention to end of term teaching evaluations (process reflection/instructional knowledge—experience-based) 7. Writing critiques on “how-to teaching books” (premise reflection/instructional knowledge—research-based); 8. Administering learning styles or other inventories to students (process reflection/pedagogical knowledge—research-based/experience-based); 9. Writing an article on how to facilitate learning in the discipline and submit it to a scholarly journal (content/process reflection/pedagogical knowledge—research- based); 10. Gathering feedback from students on their learning of discipline-specific concepts (process reflection/pedagogical knowledge—experience-based); 11. Comparing research-based insights gained from courses on teaching and learning to one’s knowledge of how students learn (process reflection/pedagogical knowledge—research-based) 12. Listening to others, observing how others learn, and discussing or writing about it (process reflection/pedagogical knowledge—experience/research-based) 13. Reading articles or books on learning and developmental theory (content reflection/pedagogical knowledge—research-based); 14. Observing others teach and observing the reactions of their learners (process reflection/pedagogical knowledge—experience-based) 15. Conducting an action research project on student learning (process reflection/pedagogical knowledge—research-based); Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 102 16. Presenting findings from classroom teaching experiments at teaching-related sessions at conferences (process reflection/instructional knowledge—research- based); 17. Showing how goals of one’s teaching relate to what students need to live successful lives (Process reflection/curricular knowledge—experience-based) 18. Consulting with an educational development specialist (process reflection/pedagogical knowledge—research-based); 19. Comparing classroom experience to formal research results on student learning (process reflection/pedagogical knowledge—research-based); 20. Explaining how and why goals have changed over time (Premise reflection/curricular knowledge—experience-based); 21. Consulting with employers to see what goals they have in mind (Premise reflection/curricular knowledge—experience-based). 22. Participating in a curriculum review committee (Premise reflection/curricular knowledge—experience-based). 23. Participating in philosophical discussions on student learning, for example through a listserv or with colleagues (premise reflection/pedagogical knowledge—experience-based) 24. Reading books on the goals of higher education and comparing goals to those underlying the programs offered in the department (process reflection/curricular knowledge—research-based) 25. Writing articles that compare the usefulness of textbooks in one’s field and compare outcomes of analysis to own text and course content (Process reflection/curricular knowledge—research-based) Prior to discussing this list it might be useful to make one further observation. It has been suggested that there is a difference between scholarly teaching and the scholarship of teaching (e.g., Richlin, 2001) and some feel that this distinction is one of critical importance. To the extent that we adopt this view, we will conclude that most of the above indicators reflect scholarly teaching but not the scholarship of teaching. However, does interpreting the scholarship of teaching exclusively as “publication of research on teaching and learning” (and considering any practices that do not meet this criterion as scholarly teaching) not unnecessarily reduce the scholarship of teaching to the scholarship of discovery on teaching3? The latter, I propose, is just one way by which to engage in and demonstrate the scholarship of teaching. If the scholarship of teaching is aimed at enhancing the quality (and recognition) of teaching and supporting student learning, is informed by knowledge of the field, is inquiry-driven, involves critical reflectivity and scrutiny by peers, as many have argued over the years (e.g., Andresen, 2000), does it then not follow that the above indicators are indeed indicators of the scholarship of teaching, particularly if they themselves are shared with peers? Essentially, the idea of using indicators is that they allow us to make inferences about the kind of learning or reflection the faculty member has engaged in. To the extent that the indicators suggest engagement in process and premise reflection (through instrumental, 3 I assume that advocates of this view would suggest that the difference between what educational researchers do and what scholars of teaching do, is that for scholars of teaching the research problem originates in their own teaching practice, whereas for educational researchers the problem originates on the basis of theory. This distinction, however, would not hold true for all educational researchers either. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 103 communicative or emancipatory learning), we can infer that assumptions about teaching and learning were tested for their validity. To be clear, not every indicator on this (or any other) list of suggestions needs to be addressed. The idea is that one would want to see some evidence of process and premise reflection in the domains of curricular, pedagogical and instructional knowledge. Such evidence might be produced through traditional forms of inquiry and sharing such as studies of how students learn that are then published, but many of the above indicators do not involve publication. Without a doubt, the scholarship of teaching can be demonstrated in many different ways (see for example, Theall & Centra, 2001). Other indicators than the ones suggested here are clearly possible and the development of further indicators by those who practice the scholarship of teaching is both necessary and encouraged. For an outsider it is not always easy to decide whether an indicator (i.e., a concrete action that a faculty member may take) gives evidence of content, process or premise reflection on the part of a faculty member. A higher degree of trustworthiness in the interpretation of indicators might be possible only by talking to the faculty member him or herself. When teaching portfolios are used as described in this essay, namely with the intent to record and demonstrate reflection of different kinds, their greatest value may lie in the formative purposes they serve. Conversations based on the portfolio between educational developer and faculty member would hold great promise for further reflection and continued growth in teaching. As for summative purposes, the portfolio holds potential as well. As was noted, it is neither likely nor necessary that reviewers of portfolios will be able to decide for each recorded instance whether reflection was focussed on content, process or premise. Nonetheless, reviewers can still arrive at an overall impression of whether the faculty member went beyond content reflection on goals, his or her understanding of student learning and development, and how to promote academic learning and development. Results from a recent study suggest that faculty engage primarily in content reflection on teaching, followed by process reflection and that premise reflection across all three knowledge domains is not as common (Kreber, 2005b). VIII. Concluding comments I argued in this essay that the scholarship of teaching and learning needs to be informed by a broader conceptualizations of professionalism, one that is not limited to “best teaching practices” but is inclusive of the notion of citizenship (contributing to the university’s moral and civic purpose). The Scholarship of Teaching (SofT) model (Kreber & Cranton, 2000) suggests that faculty develop in the scholarship of teaching and learning as they actively construct and validate their knowledge through reflection in three domains of teaching knowledge: (1) knowledge about the goals and purposes of university teaching (curricular knowledge), (2) knowledge about student learning and development toward those goals (pedagogical knowledge, or rather psychological knowledge), and (3) knowledge about how to optimize this learning and developmental process (instructional knowledge). Reflection is informed by knowledge gained through personal experience and/or through formal inquiry faculty conduct themselves or read about. Content, process and premise reflection are three qualitatively different kinds of reflection. Only in process and premise reflection are assumptions or conceptions questioned for their validity. In content reflection we merely make our present assumptions explicit (i.e., we state what we believe to be true); however, content reflection is not irrelevant as identifying Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 104 assumptions is a critical first step in reflection. Both process and premise reflection go beyond this and involve “validity testing”. Process reflection occurs through instrumental and/or communicative forms of learning and focuses on the problem-solving strategy (“how do I know that what I’m doing makes sense?”). Premise reflection can lead to emancipatory learning and focuses on the presuppositions underlying our practices and how they came to be taken for granted (“why does it matter that I/we focus on this problem?”). Reflection on assumptions (or conceptions) we hold does not always lead to a transformation of assumptions (or conceptions) to be meaningful (Kelly, 1955; Mezirow, 1991), as through reflection we may find justification for our practices. Only if, through reflection, assumptions are not validated might transformative learning take place. However, even then it will depend on a combination of factors -- including personal (for example, willingness to change), social (for example, support by others for change) and contextual ones (for example, external constraints to change) -- whether reflection will lead to a transformation of assumptions and, ideally, changes in practice. As well, as was stated earlier, whether or not one transformed assumption leads to a transformation in the larger frame of reference (or conceptual structure) on teaching depends on the importance we attribute to our new insight or knowledge. The scholarship of teaching and learning thus construed involves both learning and knowing about teaching. Teaching portfolios offer the opportunity to document or publicize our engagement in the scholarship of teaching and learning and to share or exchange the insights we gained through the various reflective processes with the larger academic teaching community so that they, in turn, can review and critique our practices. This exchange or sharing of indicators of reflection with members of the academic teaching communities may encourage others to build on our work. It might make most sense to start this sharing within our own departmental contexts where our insights can immediately be built upon to improve practice. In an ideal scenario there would be teaching environments in place in all departments across the university that encourage faculty to support each in other in the process of content, process and premise reflection on educational goals and purposes, learning and student development and instruction design. But small groups of faculty who start a weekly or monthly discussion group can make a difference. And if this group decides to go together to conferences on teaching and learning to share their own work more widely or to learn from that of colleagues, they have even greater insights to share with their own departmental colleagues when they return. I should not conclude without noting that the three forms of reflection described here (on content, process and premise) are also useful in the planning of educational development program initiatives. Questions program planners could ask themselves include: ● To what extent are participants in our program encouraged to engage in content, process and premise reflection on the goals and purposes of the courses they teach specifically and on the goals and purposes of a university education more generally? ● To what extent are they encouraged to engage in content, process and premise reflection on learning and student development? ● To what extent are they encouraged to engage in content, process and premise reflection on instructional design (including teaching and assessment methods)? ● To what extent are they encouraged to reflect on their personal experience and on education theory? ● To what extent are they encouraged to contribute to educational theory? ● How do we evaluate these kinds of learning? Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 105 I suggested that faculty can record the indicators of their reflections (i.e., the concrete activities they engaged in that made them reflect) in the form of a teaching portfolio for formative and summative evaluation purposes. However, the list of indicators introduced earlier also serve a second purpose: these indicators are useful also for planning educational development assignments or activities for faculty and GTAs as they are concrete examples of activities that can be planned that would involve course participants directly in the desired forms of reflection. References Andresen, L.W. (2000). A usable, trans-disciplinary conception of scholarship. Higher Education Research and Development, 19(2), 137-153. Boud, D., Keoch, R., & Walker, D. (1985). Reflection: turning experience into learning. London: Croom Helm. Brookfield, S. (1991). The Skillful Teacher. San Francisco: Jossey-Bass. Brookfield, S. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass. Calderhead, J. (1989). Reflective teaching and teacher education. Teaching and Teacher Education, 5(1), 43-51. Cochran-Smith M. & Lytle, S.L. (1990). Research on teaching and teacher research: the issues that divide. Educational Researcher, 19(2), 2-11. Colby, A., Ehrlich, T., Beaumont, E. & Stephens, J. (2003). Educating citizens. Preparing America’s undergraduates for lives of moral and civic responsibility. The Carnegie Foundation for the Advancement of Teaching. San Francisco: Jossey-Bass. Cranton, P.A. (1998). No one Way. Toronto: Wall and Emerson. Dewey, J. (1933). How we think. New York: Heath. DfES (2003). The future of higher education (white paper). Norwich, UK. Edgerton, R., Hutchings, P., & Quinlan, K. (1991). The teaching portfolio: Capturing the scholarship of teaching. Washington, DC: American Association for Higher Education. Glassick, C.E., Huber, M.T. & Maeroff, G.I. (1997). Scholarship Assessed. Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass. Habermas, J. (1971). Knowledge and Human Interests. Boston: Beacon Press. Hatton, N., & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, 11, 33-49. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 106 Hiebert, J., Gallimore, R., & Stigler, J. (2002). A knowledge base for the teaching profession: Whaat it would like it and how we can get there, Educational Researcher, 31(5), 3-16. Huberman, M. (1985). What knowledge is of most worth to teachers? A knowledge-use persepctive. Teaching and Teacher Education, 1, 251-162. Hutchings, P., & Shulman, L. S. (1999). http://www.carnegiefoundation.org/eLibrary/docs/sotl1999.htm (also published in Change Magazine, September/October 1999. Volume 31, Number 5, pages 10-15) Jarvis, P. (1999). The Practitioner Researcher: developing theory from practice. San Francisco: Jossey-Bass. Kelly, G.A. (1955). The Psychology of Personal Constructs, Vol. 1. New York, NY: Norton and Company. Knapper, C.K., McFarlane, B., & Scanlon, J. (1972). Student evaluation: An aspect of teaching effectiveness. CAUT Bulletin, 21(2), 26-34. Knapper, C.K. (1995). The origins of teaching portfolios. Journal of Excellence in College Teaching, 6(1), 45-56. Kolb, D. (1984). Experiential Learning. Englewood Cliffs, NJ: Prentice Hall. Kreber, C. (2005a). Charting a Critical Course on the Scholarship of University Teaching Movement. Studies in Higher Education. Volume 30, Number 4, 389-407. Kreber, C. (2005b). Reflection on teaching and the scholarship of teaching. Higher Education. Volume 50 (2), 323-359. Kreber, C. (2003). The scholarship of teaching: Conceptualizations of experts and regular academic staff. Higher Education,46 (1): 93-121. Kreber, C. (2002a). Teaching excellence, teaching expertise, and the scholarship of teaching. Innovative Higher Education, 27(1), 5-23. Kreber, C. (2002b). Controversy and consensus on the scholarship of teaching: A Delphi Study. Studies in Higher Education, 27 (2), 151-167. Kreber, C. (2001). Designing teaching portfolios based on a formal model of the scholarship of teaching. To Improve the Academy. The Professional and Organizational Development Network in Higher Education. Bolton, MA: Anker Publishing Com, Vol 19, pp. 285-305. Kreber, C. (1999). A course-based approach to the development of teaching-scholarship: a case study. Teaching in Higher Education, 4(3),309-325. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 107 Kreber, C. (1998). Perfection through reflection: The teaching portfolio to evaluate and professionalize university teaching (in German). In, Handreichungen zur Hochschuldidaktik (section 18, pp.1-24), Handbuch Hochschullehre. Bonn, Germany: Raabe Verlag. Kreber, C. & Cranton, P. A. (2000). Exploring the scholarship of teaching. Journal of Higher Education, 71(4), 476-496. Kreber, C. & Cranton, P.A. (1997). Teaching as scholarship: a model for instructional development. Issues and Inquiry in College Learning and Teaching, 19(2), 4-13. Kember, D., & Gow, L. (1994). Orientations to teaching and their effect on the quality of student learning. Journal of Higher Education, 65, 59-74. Lempert, D.H. (1996). Escape from the ivory tower. Student adventures in democratic experiential education. San Francisco: Jossey-Bass. Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage. Martin, E., & Balla, M. (1991). Conceptions of teaching and implications for learning, Research and Development in Higher Education, 15, 148-55. Marton, F. (1981). Phenomenography-describing conceptions of the world around us. International Science, 10, 177-200. Menges, R. J. & Weimer, M. (1996). Teaching on Solid Ground: using scholarship to improve practice. San Francisco: Jossey-Bass. Mezirow, J. (1991). Transformative Dimensions of Adult Learning . San Francisco: Jossey-Bass. Munby, H. & Russell, T. (1994). The authority of experience in learning to teach: messages from a physics methods class. Journal of Teacher Education, 45(2), 86-95. Moon, J.A. (2000). Reflection in Learning and Professional Development. London: Kogan Page. National Panel Report (2002). Greater Expectations: A new vision for learning as a nation goes to college. Washington, DC: Association of American Colleges and Universities. Norris, S. P. (2001). The pale of consideration when seeking sources of teaching expertise. The American Journal of Education, 108 (3), 167-196. Orr, D.W. (1990). The liberal arts, the campus, and the biosphere. Harvard Educational Review, 60(2), 205-217. Paulsen, M. B. (2001). The relation between research and the scholarship of teaching, in Kreber, C. (Ed.) Revisiting Scholarship: Perspectives on the Scholarship of Teaching. New Direction for Teaching and Learning, No. 86. San Francisco: Jossey-Bass. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 108 Piaget, J. (1964). The child's conception of the world. London : Routledge & Kegan Paul. Piper, M. (2003). Special feature: Building a civil society. A new role for the human sciences. 2002 Killam Annual Lecture. The Canadian Journal of Higher Education, 33(1), 113-130. Prosser, M., & Trigwell, K. (1999). Understanding learning and teaching. The experience in higher education. Society for Research into Higher Education: Open University Press. Ramsden, P. (1992). Learning to Teach in Higher Education. London: Routledge. Rando W.C. & Menges, R.J. (1991). How practice is shaped by personal theories, in: Menges, R.J. & Svinicki, M.D. (Eds) College Teaching: from theory to practice. San Francisco: Jossey- Bass. Rhoads, R.A. (2000). Democratic citizenship and service learning: Advancing the caring self. In, M. Baxter Magolda (ed), Teaching to promote intellectual and personal maturity: Incorporating students’ worldviews and identities into the learning process (pp. 27-44). New Directions for Teaching and Learning, No. 82. San Francisco: Jossey-Bass. Rice, R.E. (1991). The new American scholar: scholarship and the purposes of the university, Metropolitan Universities, 1, 7-18. Richlin, L. (2001). ‘Scholarly teaching and the scholarship of teaching’, in Kreber, C. (ed), Revisiting Scholarship: Perspectives on the Scholarship of Teaching, New Directions for Teaching and Learning, no 86. San Francisco, CA: Jossey-Bass. Schön, D. (1983). The Reflective Practitioner. San Francisco: Jossey-Bass. Schön D (1995, November, December). The new scholarship requires a new epistemology. Change, 27-34. Shore, B. M., Foster, S. F., Knapper, C. K, Nadeau, G. G., Neill, N., & Sim, V. (1980, 1986) The teaching dossier. (Revised edition). Montreal: Canadian Association of University Teachers. Shulman, L.S. (1987). Knowledge and teaching, Harvard Educational Review, 57, 1-22. Smith. R.A. (1995). Creating a culture of teaching through the teaching portfolio. Journal of Excellence in College Teaching, 6(1), 75-100. Theall, M.,& Centra, J.A. (2001). Assessing the scholarship of teaching: Valid decisions from valid evidence. In C. Kreber (Ed.), Revisiting scholarship: Perspectives on the scholarship of teaching. New Directions for Teaching and Learning, 86. San Francisco: Jossey-Bass. Trigwell, K., Martin, E., Benjamin, J, & Prosser, M. (2000). Scholarship of teaching: a model. Higher Education Research and Development, 19(2), 155-168. Kreber, C. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August, 2006. 109 Walker, M. (ed) (2001). Reconstructing professionalism in university teaching. Society for Research into Higher Education: Open University Press. World Conference on Higher Education (1998). Higher education in the twenty-first century. Vision and action. Volume I, Final Report. Paris: UNESCO Publishing. Zeichner, K.M., Tabachnik, B.R., & Denmore, K. (1987). Individual, institutional, and cultural influences on the development of teachers’ craft knowledge, in J. Calderhead (Ed) Exploring Teacher Thinking. London, UK: Cassell. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006, pp 110 - 117. Bridging the Theory-practice Divide: A Creative Approach to Effective Teacher Preparation Jacqueline A. Hughes, Ph.D.1 Abstract: Teacher educators need to remain current regarding the challenges that prospective teachers are going to face in their classrooms. One way to maintain this currency is for teacher educators periodically to spend some time in the K-12 classroom testing the theories they teach. This paper will discuss the benefits both teacher educators and prospective teachers will derive from engaging in such an activity. I. Introduction Preparing prospective teachers for the realities of today’s classrooms is a complex and challenging undertaking for teacher educators. This complexity and challenge is a result of the changing nature of the classroom. Schools today face an increasing number of language learners, the mainstreaming of special population students, and, working with a standards driven curriculum, all of which present new challenges for the teacher as they attempt to meet their students educational needs. As a result of this “new classroom environment” and the educational needs they present teacher educators must now seek different approaches to prepare prospective teachers to meet these needs because the traditional (e.g. coursework independent of fieldwork) approaches to teacher preparation are no longer effective in equipping teachers to address these issues. It has been my observation that some teacher educators are so far removed from the K-12 environment that WHAT they teach sometimes does not reflect the realities their students face. Additionally, there is the belief that “learning to teach is a two-step process of knowledge acquisition and application or transfer” (Feiman-Nemser & Remillard, 1996, p. 79). The latter view infers a mutually exclusive relationship between the teacher educator, the prospective teacher and the classroom. In this approach to teacher preparation, the teacher educator provides the knowledge and the prospective teacher applies it. However, the teaching of theories or knowledge to prospective teachers and expecting that they will effectively apply them is an inadequate approach (Stuart & Thurlow, 2000;Wideen, Mayer-Smith & Moon, 1998; Adams, Shea, Liston & Deever, 1998) to teacher preparation. The assumption lying herein is that prospective teachers not only acquired the knowledge and theories in their program but the wherewithal to apply it in their classrooms. This, of course, might be true if the process of learning to teach were linear rather than dynamic; free of extraneous influences and circumstance rather than a complex mélange of variables This thought is best captured by Britzman (1991): “learning to teach is not a mere matter of applying decontextualized skills or of mirroring predetermined images: it is a time when one’s past, present and future are set in dynamic tension. Learning to teach- like teaching itself- is always the process of becoming: a time of formation and transformation, of scrutiny into 1 Assistant Professor, School of Education, California State University, Bakersfield, (661) 665-6807, jhughes4@csub.edu Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 111 what one is doing and who one can become…Learning to teach is a social process of negotiation rather than an individual problem of behavior.” Since preparation of teacher candidates is, at best, a complicated process, teacher educators must consider adopting new practices: As such, for teacher educators to better prepare prospective teachers three things must occur: examination of their teaching practices and “the process of learning to teach” (Szabo, Scott & Yellin, 2002, p.1); utilization of field work to aid prospective teachers in their process of meaningful reflection and construction of practical knowledge (Perry & Power, 2004); and, finally, inculcating prospective teachers’ understanding of the relationship between theory and practice (Szabo, Scott & Yellin, 2002). It is the consistent interconnection and persistent engagement in the above facets of learning-to-teach that show promise of more effectively preparing teachers. In this paper, I will discuss a field-based approach I implemented to examine my own teaching practices in linking theory with practice and regaining currency in the real world of public school. I approached this project with the tentative optimism that my experiences would assist me in preparing prospective teachers for their “multiple roles and [the] contextual complexities of life in schools” (Knowles & Cole, 1996, p. 648). II. A Theoretical Framework The teaching of theory must be (or should be) inextricably linked to its application (Brunner, 1997). When students are exposed to theoretical concepts for the first time, they must be introduced to these ideas in a manner to which they can best relate (Brunner, 1997). Brunner (1977) refers to this “as grasping the structure of a subject.” He further argues that “teaching specific topics or skills without making clear their context in the broader fundamental structure of a field of knowledge is uneconomical in several deep senses” (pg. 31), in that: “such teaching makes it exceedingly difficult for a student to generalize from what he has learned to what he will encounter later…The best way to create interest in a subject is to render it worth knowing, which means to make the knowledge gained usable in one’s thinking beyond the situation in which the learning have occurred. Third, knowledge one has acquired without sufficient structure to tie it together is knowledge that is likely to be forgotten. An unconnected set of facts has a pitiably short half-life in memory.” Similarly, theories cannot be taught in a vacuum; prospective teachers must understand the relationship between the ideas they are taught and the applications they will encounter. One way to develop this skill is to arrange for this connection to be made in the context of their “lived” realities. Such learning-in-context will provide prospective teachers with the opportunity of questioning what they do and think (Brookfield, 1995). It is during this process of inquiry, thinking about their practice, that teacher transformation occurs. Another valuable theoretical approach which informs this process is the constructivist approach to learning, which derives its name and its power from the belief that knowledge is best constructed when the learner actively interacts with the environment and, hence, constructs meaning from that experience (Hausfather, 2001). Similarly, Hall-Quest asserts in the editorial foreword to Dewey’s Experience and Education, that “sound educational experience involves, above all, continuity and interaction between the learner and what is learned” (Dewey, 1938, Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 112 p.10). The end result of this nexus is the teacher’s ability to transfer teacher-knowledge to effective practice. In sum, the theories that drive this project are those of constructivism, the theory of integration, and the rigorous application of critical reflective thinking. III. The Role of Fieldwork in Teacher Preparation Field experiences are significant means through which to develop prospective teachers’ understanding of the why, what, and how of teaching and learning. However, simple placement of student teachers in the field does not automatically result in a valuable experience for the teacher candidate (Zeichner, 1990). After all, not “all experiences are genuinely or equally educative” (Dewey, 1938, p. 25). Dewey (1938) asserts that “it is not enough to insist upon the necessity of experience, nor even of activity in experience [emphasis mine]. Everything depends upon the quality of the experience which is had” (Dewey, 1938, pg. 27). Although, there is little doubt among teacher educators about the role of fieldwork in preparing better teachers, “there is persistent concern that such experiences do not reach their full potential value” (Bowman & McCormick, 2000, p.256). Several circumstances may account for this: traditional structures of student teaching (Zeichner, 2002), which are often developed out of “convenience or tradition” (Guyton & McIntyre, 1990, p 517) rather than innovative practices; limited resources to carry out field work (Goodlad, 1990; Darling Hammond, 1999); the individualized nature of fieldwork (Goodlad, 1994); the quality of the field placement (Laboskey & Richert, 2002); and a traditional approach to university supervision (Bowman & McCormick, 2000). Empirical evidence and current thinking suggest that many time-honored and time-worn field experience practices, such as those referenced earlier, need to be either refurbished or abandoned all together. One viable solution to teacher-educator lack of currency lies in the periodic return of teacher-educators to the public school environment with the intent of gaining practical experience. A return to the living laboratory of the K-12 classroom will allow teacher educators to test the theories and concepts they teach as well as to examine their own teaching practices while making pertinent and necessary revisions and adjustments in their practice. IV. Approach From several years of informal conversations with teacher candidates during office hours, class discussions, brief encounters with them in the hallways, and reading their observation journals, I have concluded that for teacher preparation to be effective it must take place in the context in which it occurs, the school environment. Over the course of several semesters, I utilized various approaches to field work, such as focused observation activities with required critical reflective inquiry of their observation and case study development and analysis of issues of interest to the prospective teacher. Although these exploratory attempts at “teaching in context” yielded some satisfying results, I became increasingly restless with my methodology. Searching for something new and innovative, I assumed the role of a 9th grade Algebra-I teacher, sharing teaching responsibilities with the teacher of record. This pilot project was conducted at a high school located in a predominantly Hispanic community in Southern California. The high school population consisted of approximately 95% Hispanic with Asians, Caucasians, and African Americans constituting the remaining 5%. I selected this school because of its cultural location (a high percentage of minority students, Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 113 Mexican and South-American, of low socio-economic status) as well as the current commitment between the university and the school district.2 Four students fully participated in this pilot project. Because the majority of my students were themselves classroom teachers (on emergency permit) or holding jobs outside of the field of education, the remainder of my college class members participated in a somewhat more limited way. The project full-participants observed my classroom experience for a minimum of two class periods twice weekly. In addition, a thirty to forty minute critical and reflective debriefing was conducted immediately after the observation. If time constraints prevented this from happening, the requisite discussion session took place in my university office later that same day. During these discussion sessions, participants were encouraged to speak freely about my teaching strategies and my interaction with the algebra students, as well as the general classroom atmosphere; they provided insights into the events of the period, explored various perspectives on the relationship between classroom practice and education theory, highlighted various concepts already covered in class discussions at the university, interpreted and analyzed particular incidents and offered suggestions for solutions; further, they discussed with me the reasons I had handled an issue or situation in the way that I had. In addition to these discussions, all participants maintained an observation journal in which they reflected on the various concerns and issues they encountered, some entries of which are noted below: It is important to note here that the 9th graders received a combination of before and/or after school tutoring; tutoring from the student teachers; exam review activities; and, homework to practice Algebraic concepts. A. Student #1 First reflective entry: How do I adjust Piaget’s cognitive theories to meet the different learning stylesin my classroom? It does not seem as if it applies. Especially, when I have to interpret the curriculum standards in order to teach it to my students who have such grave disparities in their learning levels. Second reflective entry: I also find that learning the concept takes so much time and I don’t have the luxury to study it in a way that would help me put it into practice due to the day- to-day demands of my classroom environment. I know that the things that I learn in this course are important and informs my teaching in some way. But, I don’t see it, yet. I know that students are at different stages cognitively, intellectually, socially, and so forth. That’s evident! I see it demonstrated in my students everyday. So I don’t need a theorist to tell me this. My problem is how these seemingly relevant concepts, theories, help me motive, J--e. How does it help me teach my academically diverse students complex curriculum standards while being cognizant of their overall development – social, emotional, behavioral? Third reflective entry: I find myself not wanting to learn these concepts because I don’t know yet howto readily apply it to my class. I hear my peers speak of how they have applied cooperative learning in their classroom and how successful it was. But, I am afraid I don’t share those experiences. Dr. Hughes – struggled to illustrate this concept in class last night and there were no videos except from her high school class regarding this issue. So I was really stuck. 2 Human subjects clearance was not needed for this research. Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 114 B. Student #2 Reflective entry: I think I got it! The theory of motivation. I so want my students to want to learn. Then it occurred to me that (as I watched Dr. Hughes struggle with how best to motivate A----a and some of the other low motivated students) that she missed something important – what interest them). She talked about it in class but she didn’t do it. I don’t know why! I will ask her later. Reflective entry – a week later: This week Dr. Hughes planned to review for the upcoming exam. S----n and I have worked hard all weekend putting together a Jeopardy Algebra game. We weren’t sure if it would work but Dr. Hughes let us try anyway. It worked well! I was so relieved. The students were excited and very much into the game. I thought it was because the method was fun, innovative and so forth. I was worried about no real reward (e.g. no homework, etc.) except for the points they received for the correct answers in each category. Much to our surprise (Dr. Hughes, too). The students were really into the activity. They didn’t seem to mind that all they would have at the end of the period was just points for first, second, or third place. I am not sure if the students learned much in preparation for the exam. But, they indicated that they liked the game, that it was fun. Dr. Hughes informed us the next week that a majority of the students had passed the exam… We had hope that the game would motivate the students to go home and study as a result of the positive feelings of giving the right answer and being praised by their team-mates and teacher. These sessions (as well as several days of my teaching) were videotaped; these tapes were then used in my university class as a teaching tool to enhance the curriculum and to demonstrate best or worst practices. This procedure provided a rich foundation for discussion by both full and limited participants. The limited participants benefited from watching the video of my teaching as well as listening to the discussions between me and their student colleagues. The retrospective discussions allowed all students the opportunity to form sound and valid arguments, to make explicit their practical knowledge, and to apply this new knowledge to current educational issues. Throughout the entire experience, I provided extensive feedback to guide interpretations and encourage critical reflective inquiry. It is important to note here that the 9th graders received a combination of before and/or after school tutoring; tutoring from the student teachers; exam review activities; and, homework to practice Algebraic concepts. V. Reflections I found from my experience that when both the teacher-educator and prospective teachers are actively involved in both the college class and the K-12 environment, the ability for everyone to teach and learn simultaneously was enhanced. Furthermore, this process made it possible for me and my prospective teachers to identify and examine the convergence of theory and practice. The goal of exposing students to the process of teaching and learning through such organized field experience was met. I fervently believe that a process similar to that which I have described has the potential to augment the quality and merits of the field observation requirement. As such it contributes immeasurably to the overall ability of the students to teach effectively in the K-12 environment. A summative entry from my own journal follows: Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 115 What I learned as a Teacher Educator. As an educator I learned that it wasn’t just about teaching the theories of motivation, Piaget’s theories of cognitive development, or Vygostky’s Zone of Proximal development in my Educational Psychology course. Rather it was about teaching motivation as it came to life in my Algebra class instead of in abstract form. I was challenged with how best to motivate my 9th grade students. I tried no homework if they completed their worksheet, etc. But, I was faced with the fact that most of my students did not understand the concepts. So with the help of the collaborating teacher we divided the class into several smaller groups and assigned them work that met their skill and comprehension level. This approach worked very well. The next semester I used the experience to exemplify how Piaget’s theory of Cognitive Development might inform high school teaching. My college students remarked how very beneficial this was. Of course, I did not do away with teaching the stages of the theory of Piaget’s Cognitive Development, etc. What this experience did for me was to transform how I teach and how I constructed my course curriculum. Theoretical concepts were paired with some real life classroom experience I had encountered. My text selection was based on how well the authors presented complex theoretical concepts (that is their approach to illustrating these complex theories in the hope of increasing comprehension, and possible later transfer of this understanding). As a result, I looked for the level of language used to introduce complex concepts, case studies, activities that would facilitate understanding and skill acquisition. VI. Suggestions for Change The serious nature of the process of preparing students to meet the demands of their profession requires a critical examination of the professional requirements. One such requirement is the familiarity with the changing nature of the profession. Meeting this requirement will likely provide insights on how to interpret and integrate professional standards and expectations into program course work and requirements. Educators can meet this requirement in one of two ways. The first is, a return to the environment of professional practice for a semester or a year-long reintroduction to the field. The educator may serve as a social worker, counselor, or teacher performing duties similar to what their current students would do once in the field. The second approach to gaining familiarity with the current practices in one’s field is to allow former students to return to the college classroom to share their lived experiences and current practices as it relates to theory. The above approaches can be combined or employed separately. I prefer the approach wherein the educator returns to the field. Implementation of one or both of the above approaches or a variation will no doubt demonstrate how serious we are in bridging the divide between what our students are learning in their college classrooms and what they do in their jobs. As a result of my experiences, I am now even more convinced that faculty currency is critical to the development of well-grounded students who can effectively respond to the demands and challenges of their profession. Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 116 References Adams, N., Shea, C., Liston, D., & Deever, B. (1998). Learning to teach: A critical approach to field experiences. New Jersey: Lawrence Erlbaum. Baker, B. (1997). “Anthropology and Teacher Preparation: Some possibilities and precautions.” Queensland Journal of Educational Research, 13(2), 41-58. Bullough, R. V., jr. Young, J., Erickson, L., Birrell, J.R., Clark, D.C.’ Egan, M.W., Berrie, C. F., Hales, V. & Smith, G. C., (2002). “Rethinking field experiences partnership teaching versus single-placement teaching.” Journal of Teacher Education, 53(1), 68-80. Bowman, C., & McCormick, S. (2000). “Comparison of peer coaching versus traditional supervision effects.” Journal of Educational Research, 93(4), 256-261. Britzman, D. (1991). Practice makes practice. Albany, NY: State University of New York Press. Brookfield, S. D. (1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass. Brunner, J. (1977). The process of education. Massachusetts: Harvard University Press Darling-Hammond, L. (1999). “Educating teachers for the next century: Rethinking practice and policy.” In G. Griffin (Ed.), The Education of Teachers. Chicago: University of Chicago Press. Dewey, J. (1933). How we Think. New York: Houghton Mifflin. Dewey, J. (1938). Experience and Education. New York: Macmillan. Feiman-Nemser, S. & Remillard, J. (1996). “Perspectives on learning to teach.” In F. Murray (ed.), The Teacher Educator’s Handbook (pp.63-91). San Francisco: Jossey-Bass. Goodlad, J. (1990). Teachers for our nation’s schools. San Francisco: Jossey-Bass. Goodlad, J. (1994). Educational renewal better teachers, better schools. San Francisco: Jossey- Bass. Guyton, E., & McIntyre, D. J. (1990). “Student teaching and school experiences.” In W.R. Houston (Ed.), Handbook of research on teacher education. New York: Macmillan. Hausfather, S. (2001). Where’s the Content? “The Role of Content in Constructivist Teacher Education.” Educational Horizons, 80(1), 15-19. Knowles, J. & Cole, A. (1996). “Developing practice through field experiences. In F. Murray (ed.), The teacher educator’s handbook (pp. 648-688) San Francisco: Jossey-Bass. Laboskey, V.K., & Richert, A. E. (2002).” Identifying good student teaching placements: A programmatic perspective.” Teacher Education Quarterly, 29(2), 7-34. Hughes, J.A. Journal of Scholarship and Teaching, Vol. 6, No. 1, August 2006. 117 Perry, C. M. & Power, B. M. (2004). “Finding the Truths in Teacher Preparation Field Experiences.” Teacher Education Quarterly, 31(2), 125 –136. Stuart, C., & Thurlow, D. (2000). “Making it their own: Preservice teachers’ experiences, beliefs and classroom practices.” Journal of Teaching Education, 51(2), 113-121. Wideen, M., Mayer-Smith, J. & Moon, B. (1998). “A critical analysis of the research on learning to teach: Making the case for an ecological perspective on inquiry.” Review of Educational Research, 68(2), 130-178. Zeichner, K. (1990). “Changing directions in the practicum: Looking ahead to the 1990s.” Journal of Education for Teaching, 16(2), 105-132. Zeichner, K. (2002). “Beyond traditional structures of student teaching.” Teacher Education Quarterly, 29(2), 59-64. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006, pp. 118 - 126. Teacher Candidates’ Conceptual Understanding of Conceptual Learning: From Theory to Practice Ellen A. Sigler and Julie Saam1 Abstract: Education researchers suggest that teacher education candidates be taught that meaningful learning is essential and that conceptual understanding be infused into all lessons. However, many teacher candidates are unable to successfully develop conceptual level lesson plans and some are unable to differentiate between skills and concepts. The purpose of this study was to enhance the conceptual understanding of teacher candidates and in turn help to develop their understanding of the learning process. This was accomplished by presenting the candidates with ideas concerning conceptual learning in a way that facilitated more meaningful learning and higher-level thinking. From survey analysis and evaluation of course projects, this method seems to achieve the goal of breaking down the cognitive barriers and allowing teacher candidates to conceptualize the difference between instruction that focuses on skills and instruction that focuses on conceptual understanding. I. Introduction The notion that skills instruction is so essential to students’ ability to achieve (especially in the field of mathematics) has been prevalent in the U.S. for many years (Tournaki, 2003). Drill and practice ideologies have taken the forefront in many instructional plans due to the perceived efficiency and effectiveness of the processes (Swanson & Sachse-Lee, 2000). Especially with the onslaught of high stakes testing in the classroom, teachers often feel that direct instruction of skills is the only way to ensure that students are to achieve academically (Case, 2002; Gulek, 2003). The research, however, downplays the effectiveness of rote learning and suggests a more meaningful, conceptual understanding in order to promote problem solving and increase knowledge generalization. In addition, teachers who (as students) learned using drill and practice may find it difficult to break free of this more rigid paradigm. The purpose of this study was to enhance the conceptual understanding of teacher education students (teacher candidates) and in turn help to develop their understanding of the learning process. The intent was that the candidates develop insight into the critical difference between skills and concepts and the notion that conceptual understanding leads to richness and flexibility in problem solving both within and outside the classroom. II. Background Concept learning focuses on the aspect of category formation and the use of concepts to interpret experiences and solve problems (Ormrod, 2001). A variety of theories have been developed proposing processes by which concepts such as schemas, exemplars, prototypes, etc. 1 Ellen A. Sigler, Division of Education, Indiana University Kokomo; Julie Saam, Division of Education, Indiana University Kokomo. Correspondence concerning this article should be addressed to Ellen A. Sigler, Division of Education, Indiana University Kokomo, 2300 South Washington Street, Kokomo, Indiana 46904. E-mail: elsigler@iuk.edu Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 119 are formed (Anderson, 2000). Information processing theory embraces the ideas of conceptual understanding because it is the essence of meaningful learning and is the key to higher-level thinking. Additionally, in education, proponents of constructivism weigh heavily on the need for conceptual learning since it is the basis of all knowledge constructed by the learner (Mestre, 2002). In cognitive psychology, a concept is a group or category of similar events or objects (Ormrod, 2003). The ability to form concepts allows an individual to make sense of the vast amount of information processed everyday. Students taught to develop a conceptual understanding of various domains will be more proficient at problem solving, abstract reasoning, generalizing their knowledge to new situations and more likely to make connections to related information (Ormrod, 1999). The converse of conceptual learning is rote memorization and drill and practice techniques (Snowman & Biehler, 2003) which tend to emphasize a skill acquisition approach. “Skill learning” is the term used to define learning a set of steps or memorization of a procedure in order to accomplish a task (Mayer, 2002). Though there is acceptance of drill and practice methods for some domains, the consensus of many professionals in the K-12 classroom and at the university level is that learning must entail conceptual understanding for it to be meaningful and for it to facilitate problem solving (Stingler & Stevenson, 1994; Fuys & Liebov, 1997; Carlson, 1995). Convincing teacher candidates, that meaningful learning is vital and that conceptual understanding is essential for all lessons is part of almost every Educational Psychology text. The difficulty lies, however, in training teacher candidates to develop lessons that promote conceptual understanding versus ones that promote purely routine skills training. III. Specific Programmatic Concerns In our teacher education program, when teacher candidates were asked to develop conceptual lessons they consistently produced skill level lessons, even though the need for conceptual learning was part of their college curriculum. In this particular situation, all teacher candidates received training in conceptual learning early on in their program during the Educational Psychology course that preceded all teaching methods courses. Then, while in the methods classes, teacher candidates were expected to apply this information in the development of a variety of lesson plans, specifically in the areas of math and science. For the most part, they were unable to do so. Not only were the teacher candidates unable to develop conceptual level lesson plans but more simply, they were unable to differentiate between skills and concepts. It became our task to determine a reason for this lack of understanding, and then to develop a course of study to remedy this situation. It was immediately apparent that information concerning the differences between concept learning and skills acquisition was not fully understood by these candidates. Therefore, it was necessary to determine what lesson changes should be made to facilitate better understanding of these essential points. IV. Specific Conceptual Concerns While delving into the specific college level lessons it became clear that one of the main problems concerning concept formation was that the lessons themselves were not taught conceptually. By its very nature, the “terms and definitions” format presented in the Educational Psychology course lead candidates to learn material in more of a rote fashion, despite the Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 120 instructor’s efforts to initiate meaningful learning. In many instances throughout the course, techniques such as real life examples, discovery learning and imagery helped to develop the candidate’s understanding of the concepts at hand. However, when specifically dealing with the concept of concepts candidates regressed to old habits of rote memorization. That is, they were able to define the term “concept”, list examples and even recall theories, but when it came to higher level thinking skills; application, analysis, synthesis, etc. they were unable to utilize previous learning. A second observation regarding the problem of concept learning was the candidates’ inability to differentiate a concept from a skill. This seems more prevalent when working with less complex material. For example, when developing a lesson plan on a more sophisticated topic such as gravity, the candidates seemed to understand the concept of gravity must be demonstrated and understood by children before moving on to higher-level aspects of this scientific idea. However, when dealing with more simplistic notions such as addition, the concept of “what is addition?” did not occur to candidates as an essential part of the lesson. Candidates consistently moved directly into the manipulation of symbols without regard for the child’s level of understanding of the concept of addition. Apparently, the concept of addition was not perceived as novel most likely due to the candidate’s familiarity with the concept. As students often complain about professors teaching at too high a level, the candidates did not perceive a similar corresponding difference between their conceptual knowledge and that of the child in their classroom. Based on this information it seems imperative to present teacher candidates with ideas concerning conceptual learning in a way that facilitates more meaningful learning and higher- level thinking. Additionally, it is important to ensure that candidates identify the concept to be taught and develop lessons for children that clearly center on concept formation. In this study, our goal was to determine a way to integrate concept learning into the teacher education curriculum in a way that will facilitate more meaningful learning and assist in application of conceptual level lesson planning. To accomplish this, we developed a three-part intervention. In the first part, we created a “mock” elementary school lesson, acting out a skills- approach arithmetic lesson and utilizing a numeration system foreign to the candidates. For the second part, we developed a college-level lesson outlining the properties of conceptual understanding. In the last part, we assisted the candidates in applying the conceptual ideas to mathematical methodologies. V. Methodology This was an experimental program that was carefully developed, presented and reviewed at various stages. This study was developed in order to “acquire in-depth and intimate information about a smaller group of persons …to learn about how and why people behave, think, and make meaning as they do” (Ambert, Adler, Adler & 1995, p.880). However, this study would not strictly be classified as either quantitative or qualitative in nature. Brown (1992) referred to such studies as design experiments, which center more specifically on improvement of instruction as opposed to hard empirical evaluations. According to Kember (2003), designs such as this one are more likely to yield useful results if triangulation of data occurs. Triangulation is a method for utilizing multiple sources of data in order to “establish claims beyond a reasonable doubt” (Kember, 2003, p. 99). The data utilized in this study came from survey responses obtained immediately after the intervention and projects developed by the students at a later time. Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 121 A. Subjects Candidates from two teacher preparation classes were combined to participate in the experimental class. Fourteen of the candidates were from the 200 level Educational Psychology class, and the 32 were from the 300 level Mathematics Methods class. There were several reasons for combining the classes. First, as it was indicated earlier, candidates learn the basic framework for concept learning in the 200 level class, yet they must apply these theories in the 300 level class. It was unclear at the time which group would benefit more from this instruction, so both groups were presented the material simultaneously. B. Procedure We felt it essential that candidates, in order to have a more meaningful learning experience, must recall the experience of learning novel information. As stated earlier, candidates found it difficult to “think like children” and approach a familiar topic as a novice learner. It was also necessary that the candidates experience the information and come to their own conclusions without being told specifically what they should be learning. In other words, in order for the information to be meaningful, the candidates needed to construct their own understanding of what it means to be a novice learner. This would take more than simply giving examples of concept level lessons or identifying developmentally appropriate practice. The candidates needed to discover for themselves what a concept is and what it is not. Lastly, it was vital the concept taught be simplistic enough that when the lesson was complete the candidates could see the parallels between their concept learning and the concept learning desired by schoolchildren. If the candidates spent too much effort attempting to learn a sophisticated concept, they might possibly miss the point of the entire lesson. In order to do this we developed a basic, base-ten numeration system that consisted of unfamiliar symbols instead of the well-known Arabic system. We used the Wingding font (Microsoft, 2000) and simply replaced each number in the base-ten system with a symbol, as shown in Table 1. Table 1. Base ten symbolic numeration system. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Then we developed a few demonstration problems requiring addition and regrouping. We scripted the class lesson intentionally to model a less than adequate classroom lesson for elementary school children. An example of this script is: “We are now going to attempt some two digit addition problems. Does everyone remember how to regroup {short pause}, good. Now, let’s try one together, dot circle plus dot diamond equals square dot circle, alright, good, and when we regroup it leads to square plus square plus Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 122 dot square… which of course equals…anyone {short pause}, good, dot circle”. The instructor presenting the mathematics lesson did so as a confederate. That is, in no way did she indicate the lesson was scripted, and the information presented was anything less than authentic. After completing the sample lesson, the second instructor proceeded with the standard Educational Psychology lecture that normally accompanies a unit on conceptual learning. This included definitions and specific examples of the presentation of concepts in a classroom, and included such terms as feature lists, exemplars, prototypes, and schemas. The specific text used was Ormond (2001). After the presentation, the students were then “debriefed”. We explained that their frustration with the Wingding system is similar to the frustration school children have when teachers teach only the skill of addition and not the concept of addition. To illustrate, we showed Table 1 to the candidates and explained the construction of the Wingding system. We also practiced a few more addition problems. Within minutes, the candidates were as fluent in Wingdings as their instructor. We anticipated candidates would understand the connection between their frustration and their schoolchildren; however, they would still need guidance in applying the conceptual learning theories to instructional planning. We then gave an additional explanation of sample concepts specifically with simple classroom methodologies in mind, with results as shown in Table 2. C. Analysis Candidates completed a short Likert-scale questionnaire on various aspects of the class. Several of the questions were directed to the overall quality of the presentation. One question specifically asked the candidates about their perception of the need for such information as a teacher. Several questions (targeted the candidates in the methods class) asked candidates to rate their own understanding of concepts in the classroom and their ability to apply the information learned in the class. Table 2 . Sample lecture notes from class debriefing. Exemplar View • Exemplars are actual memories of specific members of a category, and we use these to compare to an item in question to see if that belongs to that category • Intersections and roads seen everyday as we travel to school Schema View • We recognize some objects simply by our own experiences, regardless of their group membership or their attributes. • Using measuring cups and spoons to “experience” fractions. Candidates were also invited to write in comments about this class at the end of the evaluation. These comments were collated and analyzed. In addition, the methods’ instructor Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 123 evaluated the lesson plans and mathematics education methodological projects created by the methods candidates. VI. Results and Discussion A. General Results For the question “This class helped me develop a skill I will need as a professional”, a Chi Square analysis was calculated for both classes combined. There were a total of 46 candidates. The χ2 obtained was significant at the α = .01 level. All students indicated agreement or strong agreement with this statement. For the upper-level methods candidates, 32 candidates participated in the class and completed the survey. For the questions: I remember studying concept learning in P250. I understood concept learning and its application prior to this class today. I found this class enriched my knowledge of concept learning compared to what was presented in P250. Candidates indicated either agreement or strong agreement for all three questions using a five point Likert Scale. Again, response were significant at the α = .01. Some of the candidates’ comments included: “Great review and reminder for actual use in the classroom. It increased application knowledge.” “Very helpful class, helped apply and put into perspective previous info learned.” “It helped give me more concrete examples of concept learning.” “This was a great experience – to see theory apply realistically to what I hope and will strive to do as a teacher.” B. Specific Results During the remainder of the semester, the methods instructor evaluated candidates’ work for examples of conceptual understanding and applications of conceptual learning. Examples were prevalent in lesson plans, presentations, and reflective papers. After viewing a model of the constructivism approach to teaching in the combined class, candidates seemed to understand that in order to teach schoolchildren concepts they needed to also allow these children to discover the concepts. One candidate shared an instructional strategy on how to teach the concept of factors. He asked children to use Lincoln Logs (stackable rods for building structures) and group them into even groups. For example, using ten logs, a child can group those logs into 1 group of 10, 10 groups of 1, 2 groups of 5, or 5 groups of 2. These group numbers are the total number’s factors. The number 10 has factors of 1, 2, 5, and 10. In another discovery-approach lesson plan a candidate used the process of grouping objects to teach the concept of division. The instructions for this lesson are reported below: Give each student manipulatives and a compartmentalized craft box or divided cardboard box. The manipulatives should be similar in some attributes and different in others (i.e. M&M’s of various colors, beads with different colors/shapes, cars and trucks, blocks with numbers/letters/colors, cards, or fruit). Let the students then play with/ manipulate the items. Walk around the room and Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 124 observe. Ask questions, “Why did you group these this way?” and “What is the same/different about these?” “How many do you have in each group?” and “How many did your start with?” “How did your separate the blocks, toys, etc.?” Bring the class together with a discussion about how they sorted/divided up their toys. The lesson ended with the concept presentation that sorting is division. Other candidates’ lesson plans included hands-on activities such as using measuring cups to help students understand the concept of equivalent fractions and using paint rollers and paint to understand the concept of a line. One candidate’s reflection indicated a complete understanding of the importance of using learning theories, particularly concept learning to enrich instructional planning. “Percents are so widely used in day to day situations, it is imperative that students develop a good understanding. Unfortunately, studies show that students and even adults do not understand the basics of percents. This tells me as a future educator that more time needs to be spent developing an understanding of percents before jumping ahead to calculating percents.” These lessons overall were far advanced conceptually, compared to projects from previous semesters where the majority of the lessons, presentations and projects consisted of skill-level lessons. Candidates in previous classes would develop lessons that consisted of showing students how to add 2-digit numbers, how to multiply fractions, or how to do long division, using only symbol manipulation for demonstration purposes. The students were required only to duplicate the pattern accurately. These lessons lacked conceptual learning and discovery, while the lessons produced by the conceptually trained students were rich in discovery, conceptualization and number sense. VI. Discussion The purpose of this study was to determine a way to integrate concept learning into the teacher education curriculum in a way that will facilitate more meaningful learning and assist in application of conceptual level lesson planning. It seems apparent from the content of the methods candidates’ projects after the conceptual learning class, candidates glean conceptual understanding necessary to teach children utilizing the constructivist approach. It also appears from the survey analysis, the method used to teach these lessons effectively achieved the goal of breaking down the cognitive barriers and allowing teacher candidates to conceptualize the difference between instruction that focuses on skills and one that focuses on conceptual understanding. There are many sources that cite the importance of concept learning and others that demonstrate examples of concept-level lessons for schoolchildren (Ormrod, 2001, 2003). However, there is little research concerning effective methods for instructing teacher candidates on how to develop concept level lessons. It is essential in the field of education that teachers have the skill of analyzing the methods used to teach children, and assess the goal of these methods to ensure meaningful learning is occurring (Mayer, 2002; Snowman & Biehler, 2003). In future studies, a wide range of cognitive domains (outside of mathematics) could be evaluated, ensuring that teacher candidates generalize the information concerning conceptual knowledge beyond the one presented in this study. Additionally, a further analysis about the Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 125 timeliness of teaching concept learning in the pre-professional teacher education courses should be investigated. VII. Conclusion It takes a leap of faith for many teachers and teacher candidates to leave the security of the, “teach to the test” philosophy. For so long, teachers’ understanding of learning has encompassed skills instruction, drill and repetition, that they no longer have the faith children can be taught to think. “Many studies document a preoccupation with transmission of information and rote application of ‘skills’, and a paucity of class time devoted to promoting students’ ability to think critically” (Case, 2002, p.11). This study demonstrated aspects of the process of teaching conceptual understanding in the classroom. It demonstrates how teacher candidates, when given a lesson that focuses on the development of their conceptual understanding, are able to see ways to incorporate conceptual understanding in their own lesson plans. With equal importance, however, this study demonstrated the need for the discovery approach, a conceptually based lesson for the teacher candidates themselves. Without the epiphany and the realization of their own roots of conceptual learning, the candidates slip too easily into a purely skills approach. With this method candidates were able to gain the realization that they know what they know, not from exacting drill and practice, but through constructing their own knowledge based on meaningful learning and conceptual understanding. References Ambert, A. & Adler, P.A. (1995). Understanding and Evaluating Qualitative Research. Journal of Marriage & Family, 57(4). Anderson, J.R. (2000). Cognitive Psychology and Its Implications (5th ed.). New York: Worth Publishers. Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141- 178. Carlson, A.D. (1995). Letters, numbers, shapes and colors. School Library Journal, 41(5), 30- 34. Case, R. (2002). Partnering to promote critical thinking. Canadian Statistics in the Classroom, 22(1), 11-12. Fuys, D.J. & Liebov, A.K. (1997). Concept learning in geometry. Teaching Children Mathematics, 3(5), 248-252. Gulek, C. (2003). Preparing for High Stakes Testing. Theory into Practice, 42(1), 42-50. Kember, D. (2003). To control or not to control: the question of whether experimental designs are appropriate for evaluation teaching innovations in higher education. Assessment and Evaluation in Higher Education, 28(1), 89-101. Sigler, E.A., and Saam, J. Journal of Scholarship of Teaching and Learning, Vol. 6, No. 1, August 2006. 126 Mayer, R.E. (2002). The Promise of Educational Psychology. Volume II: Teaching for Meaningful Learning. New Jersey: Merrill-Prentice Hall. Mestre, J.P (2002). Cognitive aspects of learning and teaching science. In S.J. Fitzsimmons & L.C. Kerpelman (Eds.) Teacher Enhancement for Elementary and Secondary Science and Mathematics: Status, Issues, and Problems. Washington DC: National Science Foundation. Ormrod, J.E. (2003). Educational Psychology: Developing Learners. (4th ed.) New Jersey: Merrill-Prentice Hall. Ormrod, J.E. (1999) Human Learning. New Jersey: Merrill-Prentice Hall. Snowman, J. & Biehler, R. (2003). Psychology Applied to Teaching. Boston: Houghton Mifflin. Stigler, J.W. & Stevenson, H.W. (2001). How Asian teachers polish each lesson to perfection. In M. Gauvin & M. Cole (Eds.) Readings on the Development of Children (3rd ed.). New York: Worth Publishers. Swanson, H.L. & Sachse-Lee, C. (2000). A meta-analysis of single-subject design intervention research for students with LD. Journal of Learning Disabilities, 33(2), 114-136. Tournaki, N. (2003). The differential effects of teaching addition though strategy instruction versus drill and practice to students with and without learning disabilities. Journal of Learning Disabilities, 36(5), 449-458. 127 JoSoTL Mission The Journal of Scholarship of Teaching and Learning seeks to encourage all instructors to engage in the discussion of the Scholarship of Teaching and Learning (SoTL), and to become involved in the sharing of knowledge and learning about the teaching-learning process. Any report about an investigation into what works (or doesn't work) for a particular teaching-learning context will be considered for publication. Those submissions that include reflective commentary about the result of the investigation will be considered of greater value to our readership and more appealing for publication. The journal shall also consider submissions that offer opinion, thoughtful reflection, commentary, or theoretical ideas related to SoTL. While we welcome submissions from people working in any discipline, we most strongly encourage submissions that provide useful insights and a significant potential impact across disciplines. Papers narrowly focused on a single discipline are unlikely to be accepted; authors of such papers should probably target discipline-specific journals. The Journal is intended to provide support for those already engaged in exploring SoTL, as well as encouraging those new to the topic to become involved. The support will come in a variety of forms: • Shared Results As with any journal,the readership shall gain by learning through the experiences of others published in JoSoTL • Editorial Process Submissions will be reviewed by at least two reviewers using a double-blind process. Feedback gained from the reviews, as well as comments from the Editorial Board, should help guide authors as they continue work in the field • Examples Published work provides models for those considering involvement in this field • Community The readership will have a means for interacting with editorial staff, authors, and each other in a convenient and timely manner. The community of scholars interested in a shared field should be greatly enhanced by the presence of the journal on the Internet. Classroom action research, descriptive or qualitative research, quantitative studies, case studies, and other forms of research addressing SoTL are all acceptable for consideration in JoSoTL. Brief reports of projects are welcome and will be considered for publication in the interest of fostering community comment on the work. The Editorial Board will evaluate the general quality of the work, the value of the reflective content included by the author, and the relative appeal of the report for the readership of the journal. SoTL is an evolving topic for study. There is continued need to simply think about it, suggest theory regarding it, or comment on existing notions about it. Therefore, JoSoTL will also accept submissions that do not report on completed research or experimentation. These submissions will also undergo a double-blind review process and editorial evaluation 128 Submissions Please see the Mission Statement for a discussion of the editorial philosophy for JoSoTL. Authors are encouraged to submit work in the following categories: Traditional Research Reports Reports on Classroom Action Research Essays on SoTL Teaching Portfolios Despite their differences, all of these types of submissions should include the author's expression of the implications their work has for the teaching-learning process. This reflective critique is central to our mission in furthering understanding of SoTL. Authors are encouraged to review the Guidelines for Reviewers in order to understand how their submissions will be evaluated. Authors are strongly encouraged to study the Reviewer's Rubric that reviewers shall apply in evaluating their submitted work. Authors are encouraged to submit papers, teaching dossiers, and other potential publications to Donald A. Coffin. Submissions must be prepared in an electronic format using Microsoft Word. Submissions are preferred as files attached to an e-mail. However, 3.5" disks sent through snail mail are also acceptable. JoSoTL will not return disks to authors. Research papers and essays should be according to the JoSoTL Style Sheet. There is no page limit, but authors should consider brevity a virtue as they prepare their submissions. Authors are expected to include proper referencing for their sources, especially URLs for web sites that might contain material of interest to our readership. Authors are encouraged to provide a few key hyperlinks that might offer readers additional information concerning their topic. Every submission must include a title page with the following information: • Title • For each author: o Name and affliation o Postal address o email address o telephone number • A brief (75-100 word) Abstract • A brief (more than two, less than eight) list of keywords related to your work. Do not include any author's name on any page other than the title page. Please number the pages. The Abstract will be included in the publication's title page, and the keywords are important for proper archiving of published articles. We also welcome the submission of electronic teaching portfolios for JoSoTL. Portfolios should be posted on the author's web site, and the URL should be forwarded to the managing editor. Reviewers will be asked to view the site and provide evaluation. If 129 accepted, JoSoTL will post a hyperlink from the journal to the author's site. Please note that it is impossible to provide double blind review for portfolios--although the reviewers' identities will not be revealed, the author's identity cannot be removed from an entire portfolio. Paper submissions accepted for publication will require authors to sign a Copyright Agreement with the Trustees of Indiana University. Authors must be prepared to sign this agreement upon acceptance of their work for publication in the journal. For more information regarding copyright, please see the statement of copyright and terms of use. If you have any questions regarding the submission policy, please e-mail or call a member of the Editorial Board. 130 Editorial Board All members of the JoSoTL Editorial Board are affiliated with FACET, the Faculty Colloquium on Excellence in Teaching, at Indiana University. Donald Coffin Associate Professor of Economics dcoffin@iun.edu http://www.iun.edu/~bnwdac/ School Of Business & Economics Indiana University Northwest, Gary, IN 219.980.6913 Eugenia Fernandez Associate Professor of Computer & Information Technology efernand@iupui.edu http://www.engr.iupui.edu/~efernand Purdue School of Engineering and Technology Indiana University Purdue University Indianapolis 317.274.6794 Joan E. Lafuze Professor of Biology jlafuze@indiana.edu Instructional Programs Indiana University East, Richmond, IN 765.973.8246 David J. Malik Associate Executive Vice President, Indiana University FACET Director Chancellor's Professor of Chemistry dmalik@iupui.edu http://chem.iupui.edu/Faculty/malik.html School of Science Indiana University Purdue University Indianapolis 317.274.6884 Julie Saam Assistant Professor of Secondary Science Education jsaam@iuk.edu Division of Education Indiana University Kokomo 765.455.9302 Ellen A. Sigler Assistant Professor of Educational Psyhcology elsigler@iuk.edu Division of Education Indiana University Kokomo 765.455.9419 131 Style Sheet for the Journal of Scholarship of Teaching and Learning Donald A. Coffin1 Abstract: This paper provides the style sheet for the Journal of Scholarship of Teaching and Learning. Manuscripts submitted for publication should adhere to these guidelines; finished manuscripts must be in this style for final acceptance. I. General Guidelines for the Manuscript. The final manuscript should be prepared in 12-point, Times New Roman, and single spaced. Submissions should be double-spaced. Margins should be 1” top and bottom, 1” left and 1” right. The text should be left-flush (ragged right). The title (in 16 pt. bold) and author’s name (in 12 pt. bold) should be at the top of the first page (the author’s name should be followed by a footnote that provides the author’s institutional affiliation and any acknowledgements), followed by an abstract. The abstract should be indented 1” left and right from the margins, and should be in italics. Paragraphs should have a 0.5” first line indent. We will renumber pages of final manuscript. Authors should number their pages so that they appear in the bottom right of the page. We will also insert a header on the first page of the article, as above. Authors need not insert headers/footers. References should be incorporated in the text as authors name and date of publication [Coffin (1993)], with a reference section at the end of the manuscript (see below for the desired format for the references). Footnotes should incorporate material that is relevant, but not in the main text. II. Section and Sub-Section Headings. A. Major Sections. Major section headings should be flush-left, bold-faced, and roman-numeral numbered. Major section headings should have a one-line space before and after. B. Sub-Sections. Sub-section headings should also be flush-left, in italics, and alpha-numbered. Sub- section headings should have a one-line space before and after. Sub-sub-sections should appear at the beginning of a paragraph (i.e., with an 0.5” indent, followed immediately by the text of the sub-sub-section), with the heading also in italics. 1 Associate Professor of Economics, Indiana University Northwest, 3400 Broadway, Gary, IN 46408. dcoffin@iun.edu. 132 III. Tables and Graphs. Tables and graphs should be inserted in the text where the author believes they best fit. (They may be moved around a little to better correspond to the space requirements of the Journal.) If necessary, tables and graphs may occupy an entire page and may be laid out in portrait or landscape orientation. Insofar as possible, tables should fit onto a single page. All tables and graphs should be germane to the paper. References Coffin, Donald A. (1993). “Using ‘The Competitive Edge’,” Journal of Economic Education, Vol. 24, No. 1, pp. 62-69. Garcia, Jaume, and Rodriguez, Placido (2002). “The Determinants of Football Match Attendance Revisited: Empirical Evidence from the Spanish Football League.” Journal of Sports Economics, Vol. 3, No. 1, pp. 18-38. Pappas, Doug (2004). “Fixing the Fan Cost Index: A More Realistic View.” http://www.baseballprospectus.com/article.php?articleid=2790. April 21, 2004. V6N1, Hoyert.pdf V6N1, Howard.pdf V6N1, Isaacson.pdf V6N1, Johnson.pdf V6N1, Knowlton.pdf V6N1, Kreber.pdf V6N1, Hughes.pdf V6N1, Sigler.pdf V6N1, Mission.pdf V6N1, Submissions.pdf V6N1, EditorialBoard.pdf V6N1, StyleSheet.pdf