CIE Template 1 Volume 2, Number 5 September 30, 1999 ISSN 1099-839X Effective Thinking: An Active-Learning Course in Critical Thinking Barry Leshowitz Kristen Eignor DiCerbo Scott Symington Arizona State University This article describes a college course in critical thinking. Offered in the Psychology Department at Arizona State University, this active-learning course provides instruction in how to apply principles of (scientific) methodological reasoning and optimum decision making to problems faced in everyday-life situations. Students learn to evaluate statistical and scientific evidence, clarify personal and societal values, and anticipate the consequences of their actions in dealing with personally significant issues. Crime and punishment, societal acceptance of the gay lifestyle, alcohol abuse, and racial stereotypes comprise a partial list of topics addressed in the class. Using Internet links to recorded classroom discussions archived on the World Wide Web, the article provides qualitative support for a three-level model of critical thinking. This model attempts to account for the progression of methodological reasoning skills and related dispositions that takes place over the course of the semester. [RealPlayer needed to view video clips in this article.] A Special Note from the Authors Our paper makes extensive use of Webcasts (broadcasts of the class transmitted over the Internet). These video clips allow the reader to see active-learning teaching methodology, student reactions, and changes in students' critical thinking skills that occurred in the course. The videos show the progress of students over time lend credibility to our analysis of the teaching and learning process. We would not have been able to display with visual images this longitudinal progression in a standard paper format. We believe that video clips are also valuable because communication is more than just words; people speak to us through facial expressions, body language, and voice tone. The Webcasts allow readers to witness for themselves what the students are communicating. One specific situation that leant itself to display in a Webcast was the development of graphic analysis of data. The readers can observe the students' progress analyzing claims in terms of dependent and independent variables. We observe students determining how the variables were measured, what subjects comprised the sample, and other details of the particular study. A standard paper format would be adequate for displaying the graph, but the videos allow for an examination of the process that led to the graph. Finally, the videos break up with some entertaining moments what might otherwise be a somewhat tedious presentation of a program of classroom instruction. Introduction "The goal of instruction should be to allow students to deal sensibly with problems that often involve evidence, quantitative consideration, logical arguments, and uncertainty; without the ability to think critically and independently, citizens are easy prey to dogmatists, flimflam artists, and purveyors of simple solutions to complex problems" (American Association for the Advancement of Science, 1989). Unfortunately, the results of any number of national and international studies indicate that few high school graduates (or entering college students) are able to http://www.real.com/ Current Issues in Education Vol. 2 No. 5 2 apply higher-order thinking skills to problems faced in everyday life (see e.g. NSF, 1996). Controlled studies in psychology and education confirm this finding. They indicate that most students have difficulty in (a) identifying and defining problems from multiple perspectives; (b) detecting gaps in knowledge and information; (c) establishing cause-effect relationships; (d) distinguishing facts from opinions or personal values; (e) accepting unfavorable information; and (f) evaluating costs and benefits of risky decisions (Arons, 1979; Baron, 1988; Fiske & Taylor, 1991; Gigerenzer, 1996; Kahneman & Tversky, 1996; Leshowitz, 1989; Leshowitz & Yoshikawa, 1996; Nisbett & Ross, 1980; Stanovich & West, 1998; Whimbey & Lockhead, 1986). In an exhaustive study evaluating the thinking of students in high school, college, and graduate school and comparison groups of nonstudents, Perkins (1985) has reached similar conclusions. Post-primary education appears to have little impact on students' reasoning about everyday events, and number of years of education is only a borderline significant predictor of reasoning ability. One explanation for the poor performance of students in reasoning and other higher-order thinking skills is the nature of the educational experience typically encountered in our classrooms. Educational researchers have long observed that instruction at all levels does not emphasize information-processing skills. Instead of making the development of reasoning a priority, most instruction forces a large fraction of students into blind memorization. For the great majority of high school and college students, education is limited to reproducing isolated facts that they have been taught are "true"(AAAS, 1989; Adey, 1988; Jungwirth & Dreyfus, 1990; Goodlad, 1983; Layton, 1986; Loria, Mantovani & Nasi, 1984; Thelen, 1987; NSF, 1966). Unschooled in the processes of inquiry, it is hardly surprising that many students do not know how to seek evidence for claims or evaluate data so as to extract meaningful conclusions in their effort to make informed decisions and solve problems. These limitations of instruction have not gone unrecognized by the educational establishment. To remedy this situation in K-12 education, state and federal departments of education under the leadership of blue- ribbon panels of educators have introduced inquiry-based standards in history, math, science, and art, among other areas. The science-education community has been especially active in educational reform. At the college level, for example, the National Science Foundation, the American Association for the Advancement of Science, and the National Academy of Sciences have cooperated closely in launching a nation-wide effort aimed at facilitating inquiry-based instruction in science, math, engineering, and technology (NSF, 1996). In the past decade, advocates of instruction in critical thinking also have been active in the education reform movement at all levels and across the curriculum. Reflecting many of the concerns of science educators, proponents of critical thinking have focused on higher- order reasoning skills and related thinking dispositions as the major goal of instruction (Paul, 1990). Halpern (1989, p. 5) has used the term "critical thinking" to describe the cognitive processes "involved in solving problems, formulating inferences, calculating likelihood, and making decisions." Lipman (1988) has written extensively on the criteria for evaluating critical thinking. He has defined critical thinking as thinking that meets accepted criteria within the discipline, is sensitive to context, is self-corrective, and is manifested as a propensity to give sound reasons in reaching judgments, making decisions, solving problems, and taking action. The close correspondence between the dimensions of critical thinking described by Halpern and Lipman and many of the practices of the inquiring scientist is noteworthy. This observation has had important implications for the development of my (the first author's) program of instruction in critical thinking and provides the point of departure for the present paper. Below we describe a course in Effective Thinking that attempts to facilitate the development of the skills of critical thinking through practical instruction in methodological reasoning. Methodological reasoning refers to the rules of the scientific method that emphasize posing questions in terms of relationships between variables, formulating alternative hypotheses, testing these hypotheses, collecting data, drawing (causal) inferences, reaching warranted conclusions, and making informed decisions. These "are the kinds of skills that people must have in some measure in order to live effectively in the world" (Lehman et al., 1988, p. 441). In collaboration with students over several years, I have developed and demonstrated programs in critical thinking for college and high school students (Leshowitz, Jenkens, Heaton, & Bough, 1993; Leshowitz & Yoshikawa, 1996). The cornerstone of my instruction is the application of elements of methodological reasoning to relevant, often emotionally charged problems confronted in everyday life. These practices include the methods for identifying problems, proposing alternative hypotheses, conducting scientific studies, using descriptive statistics and graphs to analyze data, doing causal analysis, and making valid inferences. In addition to instruction in methodological reasoning, my class addresses the role of personal values, ethical and moral beliefs, and cost/benefits in normative (or optimal) decision making. These personal, social, and historical factors provide the background or context for making evidence-based decisions. The overall goal of my instruction in critical thinking is to prepare students to take action based on what is objectively known and what they subjectively value. Preliminary empirical support for the effectiveness of my instruction has been presented elsewhere Effective Thinking: An Active-Learning Course in Critical Thinking 3 (Leshowitz, 1989; Leshowitz & Yoshikawa, 1996; Leshowitz, DiCerbo & Okun, 1999). (Detailed findings evaluating my program will be included in future papers.) Briefly, the results of several nonequivalent-control-group studies indicate that methodological reasoning, metacognitive insights, skepticism, reliance on intuition, cognitive bias, and resistance to attitude change, among other dimensions of critical thinking, change significantly over the course of the semester. A Psychology Course in Effective Thinking A principal focus of this article is to illustrate the active-learning, discovery-based instructional techniques that I use in my psychology course, Effective Thinking. The course is an elective that meets Arizona State University's General Studies first-level requirement for literacy and critical inquiry. A course in finite mathematics or statistics and Introductory Psychology are prerequisites for the course. About 25 students, representing psychology majors and nonmajors in about equal proportions, comprise the course's enrollment each semester. Emphasizing problems of direct concern to college students to encourage personal involvement, the course provides hands-on, real-world-based experience in dealing with everyday-life situations (de Sanchez, 1995). The following is a partial list of the personally meaningful, often highly-charged issues addressed in the course: the psychological effects of divorce on children, use of untested drugs for terminally-ill patients, formation of racial stereotypes, binge drinking in college students, and acceptance of the homosexual lifestyle in the local community. The goal of the course is to help students understand how to use methodological reasoning in gathering, analyzing, and integrating information for the purpose of making informed decisions in everyday life. To illustrate how the class learns to apply methodological reasoning to everyday problems, we present a vignette used in the class. Imagine that Pete Smith, a typical college freshman, decides to "party" excessively in response to peer pressure. In a few weeks, he reports that he has made lots of friends and attributes his acceptance to his willingness to party in the manner his pals deem appropriate. This judgment is reached despite the fact that many events have taken place during this period- he has dated a young woman in the "in" group, he is a good athlete, he has a keen sense of humor, and he seems to do well in class with minimum effort. Pete's subjective validation of the effects of partying with alcohol on his sociability is not soundly reasoned since the effects of alcohol have not been isolated. In a discovery-based, highly interactive discussion that may include role-playing, the instruction addresses Pete's flawed, "make-sense" thinking. In a rational investigation of what really is "so," the instructor might begin by asking the students to describe the scenario's major claim (or hypothesis) in terms of a relation between the antecedent (independent variable; alcohol) and the consequent condition (dependent variable; popularity). Depicting the claim pictorially in a fully labeled graph helps students focus on the underlying claim and its empirical basis. Students learn that an observed comparison between two groups is essential for establishing an association between variables. Often students' preconceived ideas about the validity of the underlying claim lead them to assume the existence of a comparison when in fact none is available. In the course of the discussion, students discover that there is little objective support for Pete's (and perhaps their own) firmly held views on the effects of alcohol and that their beliefs on the issue are essentially "illusory." During the course, students complete a plethora of written assignments. In-class and take-home exams (about 4), daily homework assignments (brief essays), at least three substantial papers of about five pages in length, and a cumulative final exam constitute a partial list of writing exercises. Students complete about 45 writing assignments, totaling well over 100 pages. Completion of these assignments, which requires about two hours of outside work for each hour of class time, involves gathering, interpreting, and evaluating information. Popular media, such as newspaper and magazine articles and videos, along with students' own personal experiences constitute the sources of information analyzed. There is no assigned textbook. Realistic communication formats, such as memoranda, letters-to-the editor, structured interviews, and investigative reports, are used whenever possible. Final grades are determined by assessing the general level of performance (numerical averages are not computed), paying special attention to progress in the late stages of the course. Asking students to confront highly controversial issues in a classroom environment can lead to frustration in some students, especially at the outset of the course. To help overcome this problem, I enlist the assistance of peer tutors. Each semester, I select outstanding undergraduates from previous semesters to serve as peer tutors. Given course credit for their participation in the program, peer tutors provide assistance in all aspects of the course. By providing detailed written comments and assessments of assignments, the peer tutors help their students explore the assumptions on which they base their personal opinions and beliefs. Peer tutors may ask for clarification of terms, reasoned support for opinions, acknowledgment of the influence of a personal bias, advantages of the opposing viewpoint, and/or consideration of values that might be affecting their thinking process. In-depth feedback on daily written assignments, we find, solidifies the relationship among the instructor, the peer tutors, and the students (MacKeachie, 1986). In this highly supportive learning environment, the peer tutors and I help our students to realize that their ideas, contributions, successes, and frustrations are taken into account by all of the participants. Current Issues in Education Vol. 2 No. 5 4 Qualitative Evidence The course relies heavily on active-learning and discovery-based techniques that facilitate development of the skills and dispositions underlying critical thinking. In an attempt to demonstrate these active-learning techniques and the students' reactions to the course's instruction, we present below Internet links to archived audio/video clips of class discussions recorded in my classroom. These segments examine problematic situations, ranging from simulated mock-jury deliberations and investigations of untested drugs, to an examination of gay marriage in a local community. The clips illustrate how I attempt to reduce active learning in critical thinking to classroom practice. These clips also serve as qualitative evidence that allows researchers to "seek explanation of what the program is doing, why it follows the course that it does, how recipients respond, and why they act in ways they do" (Weiss, 1998, p.265). We believe that video clips are more valuable than words because communication is more than just words; people speak to us through facial expressions, body language, and voice tone. The readers can witness for themselves what the student is communicating. Readers can also experience the classroom and the evolution of critical thinking with their own eyes. In effect, the clips provide people with a flavor for the environment in which we teach higher-order thinking in an active-learning environment. Elaine Yoshikawa, a returning undergraduate in Psychology, and I developed a three-level model of cognitive development for the purpose of organizing our observations of my students' thinking at various critical junctures in the semester (Leshowitz & Yoshikawa, 1996). The three levels of cognitive development were expanded into descriptive categories so the audio/video clips could be coded accordingly (Weiss, 1998). An entire semester of the class was videotaped as Webcasts (broadcasts of the class transmitted over the Internet). The Webcasts were then sorted according to which level was exemplified by the students' responses. Inclusive and exclusive examples of each level facilitated the sorting of the clips. We also analyzed the clips in terms of teaching techniques demonstrated. Techniques available for viewing include Socratic dialogues, collaborative problem-solving exercises, and role-playing scenarios. In order for a clip to be included in the paper, it had to perform one of the following functions (Guba, 1978): (a) identify new elements of importance (e.g., characterize a particular level of cognitive development; (b) reinforce existing information, but not to the point of redundancy; (c) explain information that is already available; or (d) exemplify the nature of the category. In searching for clips, we at times found ourselves examining hours of tape for a minute of relevant dialogue. In some measure this difficulty was heightened by the fact that we had a very limited set of clips from which to select tape. Also, the level of students' thinking displayed inconsistency at times, which is an inevitable consequence of the instructional process. This inconsistency made it difficult to code clips in which students exhibited various levels of thinking. However, we believe that the clips we selected for this paper do present the classroom realistically. For any particular student they show a progression in mastering the principles of critical thinking along with instances of regression to earlier ways of thinking. (To view the clips of the archived Webcasts, the user must have either a Pentium PC running Windows 95/98 or Windows NT or a Power Macintosh computer. The most current web browser must also be used; the latest version of Netscape (4.5) or Microsoft Internet Explorer version 4.0 is required. Both may be downloaded from their respective vendors free of charge. In addition, the most current version of Real Networks RealPlayer G2 software will be needed to view the clips of the class discussions. If needed, the reader may download RealPlayer G2. Select the archived clip to view by clicking on the URL. At this point, the RealPlayer software should automatically start and after a few seconds, begin playing the video. To exit the clip, click on the X at the upper right-hand corner of the RealPlayer window.) Instructional Model Guiding my program of instruction is a three- level cognitive model that describes the emergence of critical thinking and associated dispositions in most students over the course of the semester. Many of the initial observations which led to this model came from a qualitative analysis of students' comments in daily class reviews. Similar to entries in a student journal, these personal essays are analogous to contributions to portfolios and structured diaries which are currently being used as evidence of students' change, growth, and responsiveness to instruction (Collins, 1993; DeLongis, Hemphill & Lehman, 1992). This analysis led to the emergence of the theoretical framework described below. Students enrolled in my Effective Thinking course generally proceed through three levels of development. Kitchener and Brenner (1990) and Kuhn (1993; Kuhn, 1999) have proposed similar approaches to the development of reflective judgment in adulthood. Although the three levels of the model are presented here sequentially, the postulated transformations rarely occur in linear sequence. However, it is our contention that this model elucidates and clarifies the ways in which reasoning skills, cognitive strategies, and associated dispositions are modified over time. Additionally, the model provides a framework for describing the events experienced by the majority of my students, peer teachers, and their instructor in the course. Level 1: Pseudo-Knowing Student characteristics. The first level of "pseudo-knowing" reflects the typical entering student's http://www.real.com/ Effective Thinking: An Active-Learning Course in Critical Thinking 5 willingness to accept or reject new information with a modicum of questioning or critical evaluation. The first clip presents an illustrative example of Level-1 thinking. In an introductory discussion, the student expresses in a forthright manner his strong preference for opinions over facts. The student indicates that opinions are "fun," while facts are "boring." At the beginning of the semester, the great majority of students do not actively engage in questioning, evaluating, or critically analyzing information. Rather, the students passively assimilate unsubstantiated claims into their knowledge base of "known facts," especially those claims posited by "experts" or appearing in print (Leshowitz, Okun, & DiCerbo, 1999). Moreover, they are generally unaware that many of their beliefs are unsupported and speculative (Leshowitz, 1989). In the next clip, one student defends his reliance on the personal experience of respected individuals in his hometown as a source of reliable information on attending college. He believes this approach to serious decision making is preferable to gathering facts for himself. Strong, emotional, and generally unsupported opinions dominate the students' analysis of information in the early stages of the course. Consistent with these classroom observations, investigators in social psychology have empirically established that it is difficult, perhaps impossible, for many individuals to evaluate objectively information that contradicts their beliefs (Nisbett & Ross, 1980). The next clip presents an example of this common cognitive bias. In a discussion on gay issues, this student argues that no fact could change his opinion about gay marriages; what he feels (values) is on a "grander scale" that is not open to scrutiny. The student made these comments in an introductory "ice-breaking" dialogue where the students discussed various sources of information on which they rely, and rated their confidence in these sources. The aim of this dialogue is to help the students begin to identify the underlying basis of their beliefs and personal opinions. Rarely mentioning standards or criteria for evaluating information, the students make few distinctions among statements of fact, opinion, belief and theory. In large measure, the goal of the course is to help students recognize these differences in their thinking. Teaching techniques. The class uses reflective dialogue is to build scaffolding on past experiences and knowledge of the participants and to facilitate insights into past and present thinking. Toward this end, I anchor the dialogue on specific points or "windows." Allowing students to follow their own path through the dialogue's windows gives rise to a class discussion that develops spontaneously. When the class dialogue goes well, it resembles a one-on-one personal conversation. The dialogue in the next clip illustrates this instructional process. Following the presentation of a TV magazine segment on an "expert" who claims he can teach people to think "intuitively," the class assesses the usefulness of "gut instinct" in decision making. In response to the students' confidence in their powers of intuition, I direct the discussion toward the concepts of hindsight bias, rational thinking, fast (intentional) processing, and proof. I also encourage the students to examine their personal experiences as they relate to the issue at hand. The following clip presents a group discussion on the merits of using intuition in decision making. One student relates how she has used intuition (or gut-level feelings) in deciding everything from selecting a flavor of ice cream, to what college to attend, to decisions regarding whether to go out with a particular person. By rigorously challenging students' unquestioned opinions and beliefs, I try to inspire in them the desire and motivation to learn more generally accepted ways of assessing what they believe as fact. The early dialogues may appear confrontational to some students; many of these students have never had their strongly held beliefs intellectually confronted in a classroom setting. In my dialogues with students, I attempt to communicate to the individual students that their contributions are significant and valued and that they should not fear embarrassment or judgmental responses. However, even with students with whom I have established a strong mentoring relationship, I may fail to adequately affirm the student's contribution. The following interchange illustrates a situation where my effort to challenge one student's use of (limited) personal experience as a source of valid information fell short of my high standards for conducting reflective dialogues. Level 2: Destabilized Knowing Student characteristics. Uncertainty, confusion, and self-doubt regarding what or whom to believe characterizes "destabilized knowing." In the next clip, a student wonders how one goes about checking the factual basis of opinions, especially when they appear in books. In preparation for a lengthier consideration of the matter, the instructor responds, "Use your head." In Level 2, students are uncertain how to determine who, what, or why to believe. What constitutes valid and reliable information, minimal inference, expert testimony, and reasonable speculation take on major importance. Students discover that standard journalism- based criteria for evaluating information, while helpful, are too imprecise to deal substantively with issues they care about. They are uncertain about definitions of "reliable statistics," "expert witness," and "appropriate language." Adding to their discomfort and diminishing confidence in their old ways of thinking is the realization that good information is absolutely necessary for solving problems and making sound decisions. At this juncture, I find that most students are prepared emotionally to make the serious commitment to fundamentally changing their http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/031199.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/031199.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/092298.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/031199b.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/092298a.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/012899.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/031199a.smil http://cie.asu.edu/volume2/number5/index.html#level2 Current Issues in Education Vol. 2 No. 5 6 knowledge-assessment and decision-making strategies. They understand that their standard ways of thinking and deciding are in need of improvement. They know they can make better, more informed decisions. Teaching techniques. My objective at this point is to provide my students with the opportunity to become aware of and confront their misconceptions (Garfield, 1993). Toward this end, I attempt to elicit feeling-level reactions to controversial issues in an attempt to lessen the students' propensity to respond as they think is expected. Expressions of core beliefs and attitudes, I find, prepare the students to deal seriously with personally significant issues. The following illustrates the frustration of one student toward the media. He asserts that people are afraid, and the media play to this fear. Exercises in legal reasoning are especially useful in encouraging students to reconsider their ways of forming beliefs and rendering judgments. In our society, the courtroom serves as a public arena for exhibiting our thinking skills. In the legal adversarial process that takes place in the courtroom, we often display some of our best and worst examples of human cognition. From an instructional standpoint, the mock-jury exercise enables me to increase the stakes students place in developing their thinking. One particular exercise I have used with good success is a fictitious civil case involving steroid abuse in intercollegiate athletics. Many students report the resolution of this case represents a watershed in their understanding of the principles of effective thinking. In this case, a highly talented freshman football player has committed suicide immediately after abruptly ending his use of steroids. His family (the plaintiff) holds the university and coach (defendants) responsible and has instituted a lawsuit for negligence in the death of their son. In addition to the transcript of the trial, students (jurors) receive the oral testimony of a teammate role- played by a member of the class. To encourage active involvement in the trial, the jurors get an opportunity to cross-examine the teammate. This courtroom interrogation of the teammate, role-played by a female student, is depicted in the next clip. The probing questions of the "jury" indicate the high degree of student involvement elicited by this real-world, problem-based exercise. In this trial, the jury also examines several media reports that have been placed into evidence. These pieces describe the effects of anabolic steroids on the psychological well being of young men and women. As part of its deliberation, the jury must determine whether this medical evidence is methodologically sound and of probative value in the case. For example, one illustrative print article, entitled "Of Muscles and Mania," begins as follows: "Paranoia, hallucinations, delusions of grandeur, and random violent tendencies are frightening whenever they appear. . . . Bodybuilders using steroids may be prone to such psychotic and manic symptoms, according to ongoing research. . ." To underscore the claim that steroids cause harmful psychological effects, the journalist presents several vivid anecdotes of steroid use. For example, we are informed, a steroid user deliberately drove his car into a tree at 40 miles per hour while his friend videotaped the crash. Corroborating these anecdotal reports, the article presents the results of a study conducted by two psychiatrists at the Harvard Medical School. Based on interviews of 41 steroid-using bodybuilders and athletes, the study found that "psychiatric problems were far more pronounced than the physical ones . . . Fortunately, the psychotic and manic symptoms disappeared promptly when steroid use was discontinued." Lacking a control group, this study clearly does not permit one to draw any inferences about an association between steroids and psychological disorder. It can be seen in the next clip that one student found complete agreement with the article's claims of the harmful psychological effects of steroids. She reports that her past experiences with users of steroids are totally consistent with the three (anecdotal) case studies, the (cursory) statistics, and the findings of the (methodologically flawed) study reported in the article. Falling-back to earlier ways of thinking is common at this stage as students wrestle with their habitual ways of thinking. Regressing to Level-1 thinking, she confidently attributes emotional outbursts ("roid rages") she has observed in high students to the effects of steroids. In her view, her personal observations validate the purported psychological effects of the steroids. Employing a discovery method of instruction, in the next clip I attempt to facilitate a close examination of the questionable medical evidence linking steroids to psychological disorders. A student applies a methodological examination of the evidence in the next clip. Using graphic analysis, he argues that, without a control group, the study's findings do not meet the criteria for causality. Unable to establish in evidence that steroids are a "general" cause of psychological disturbance, the student concludes that steroids were not involved in the death of the student athlete. Furthermore, he argues that if steroids have not been shown in scientific investigations to cause suicide, then the coach's failure to act does not constitute legal negligence in the death of the athlete. He may have been derelict in his responsibility to the team, but he probably did not contribute in any way to the demise of the athlete. In their deliberation, the jurors consider the following questions: What do I "know?" How do I know it? What else do I need to know to reach justified conclusions about the effects of steroids and the legal negligence of the coach? Somewhat dejected by their inability to reach a consensus on answers to these questions, some students voice the view that there is little left to be said. This http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/110398.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/102298.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/102998a.smil http://cie.asu.edu/volume2/number5/index.html#level1 http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/102998b.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/102998c.smil Effective Thinking: An Active-Learning Course in Critical Thinking 7 "ruminating" and "guided frustration" are crucial for arousing students' desire to engage in the learning process. Without a strong (even emotional) commitment toward examining one's thinking process, I have found that the most one can hope for is a superficial technical understanding of the processes underlying effective thinking. Level 3: Reasoned Knowing Student characteristics. Methodological analysis of all available evidence characterizes the final level of cognitive development achieved in this course. The importance of separating evidence and values in decision making is illustrated in the next clip. The student observes that by emphasizing personal values rather than hard (tangible) evidence, the plaintiff's attorney in the steroid case loses credibility. At this juncture, students are able to abandon their reliance on unsupported opinion and untested beliefs and hunches, and understand the benefits of basing decisions on facts, evidence, and what is known, according to commonly accepted standards. Gradually, students become aware of the intrusions of their values, biases, theories, opinions, and personal beliefs on what they thought they knew as factual. Reflection on one's own thinking, which is termed metacognitive reasoning, is an essential component of reasoned knowing (Kuhn, 1991). The purpose of such "rhetorical argumentation" is to coordinate objective evidence with the theories and knowledge claims one has personally constructed to make sense of the world. Critical thinkers are individuals who not only think with their theories, but also think about their theories. In the next clip a student reflects on how her theories and intuitions should be tested against data and other information. Willing to look for facts to explain her intuition to others, she no longer is driven by her gut-level feelings. Along with the development of metacognitive- and methodological-reasoning skills, these students evince signs of a pronounced skeptical disposition. The observed changes in the attitudes of the students toward information accord closely with the theoretical categorization of methodological versus systematic skepticism (Bunge, 1991). Earlier in the course, reflecting level-2 thinking, some students refused to believe in the justifiability of any knowledge. Having witnessed numerous flawed claims and the precarious nature of what they thought they knew as fact, students displayed discomfort with information sources they had earlier accepted without hesitation. Thissystematically skeptical stance resulted in rejection of the possibility that verifiable, certain knowledge is ever attainable. As the course progresses, I typically observe that most students' systematic skepticism is replaced by more finely tuned reasoning, level-3 skills that engender thoughtful, methodological skepticism. The methodologically skeptical individual deems claims that derive strong support from empirical findings as acceptable knowledge. Without substantive proof of claims, this individual will suspend judgment and postpone taking action. Not only does the methodologically skeptical individual demonstrate a "show me" attitude toward the theories of others, which we might call "external skepticism," they also display "internal skepticism." In the next clip, students display healthy skeptical dispositions toward an "expert" who has appeared on several TV talk shows to discuss his book on the "power of intuition". Unimpressed with the expert's "BS," the students want to see evidence. The students are now about to enter the final leg of their cognitive journey. Now able to recognize that their normal ways of knowing lack precision and explicit rules for application, they ask for more powerful tools of analysis. Teaching techniques- applying practical methodological reasoning skills. In the final phase of instruction, my goal is to make explicit the principles of methodological reasoning and to show their wide applicability to virtually all areas of everyday life. In response to the ground swell of demand for better methods for evaluating evidence and rules for reaching sound judgments, I direct the class toward the principles of methodological reasoning. In a group of exercises, the students reduce the rhetoric in persuasive communications to simple claims relating the antecedent condition (independent variable) to the consequent condition (dependent variable). Depicting the claim in a fully labeled graph helps the students focus on the communication's bottom-line conclusion and its empirical justification (Leshowitz, Jenkens, Heaton, & Bough, 1993). The principal goal of methodological analysis is to determine whether the observed effects are attributable to the antecedent or causal condition. To make this determination, one must control for (or eliminate the influence of) third (confounding) variables or other possible causes (rival hypotheses).The following is an excerpt from a class discussion where the students discover that correlation between variables does not imply causation. Unable to establish the (causal) connection between obesity and success in life, the students are unwilling to recommend losing weight to an individual who wishes to attain personal success. They recognize that too many other factors may play a greater role in determining success. In classroom exercises and real-world-based investigations, the students discover that knowledge of causal relations allows one to predict and control future events. This knowledge is fundamental to making the informed decisions that underlie effective problem solving. Although the students readily accept the importance of methodological analysis for decision http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/102998.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/021899.smil http://cie.asu.edu/volume2/number5/index.html#level2 http://cie.asu.edu/volume2/number5/index.html#level3 http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/110398a.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/101598.smil Current Issues in Education Vol. 2 No. 5 8 making, they also wonder whether it will be of very much practical use. They observe that rarely are (causal) data from true experimental designs available for the kinds of rapid decision making required in everyday-life situations. Hoping to deflect this criticism, I point out that the need to research all claims is not nearly as onerous as might appear at first glance. Looking back at issues that we have examined over the semester, I ask whether there is even a shred of evidence to support the assertions of of the various purveyors of questionable claims. The goal of methodological analysis in everyday-life situations is to detect "big zero's." With a little practice, students can detect "smoke and mirrors" almost reflexively. However, there is no escaping the fact that some effort is required and that "the devil is in the details." Teaching techniques- integrating evidence and values in decision making. After considerable practice, frustration and soul-searching, most students come to realize the benefits of extracting and analyzing an issue's empirical or evidential support. At this point, I return to the students' questions about the practical value of methodological reasoning. Pulling the intellectual rug out from under them, I ask whether decision making based on principles of rational analysis of evidence is too limited. To underscore this point, I often ask the class to identify situations where a great deal of statistical/scientific evidence points to a certain course of action that is ethically or morally unacceptable. For example, I ask them whether they would be willing to implement the policies of certain other countries that severely punish, and often execute, all distributors, and sometimes users of illegal drugs. Somewhat confused, but generally relieved, students examine the possibility that critical thinking entails a methodological analysis of evidence as well as an assessment of non-evidential factors such as personal values, ethical/moral principles, and cost/benefits of decision outcomes. The need to integrate all relevant evidence- and value-based information in problem solving has been noted by others (see e.g., Arambula-Greenfield, 1996). To facilitate this process, I present to the class the following formula: E x V = A. Useful in decision making, this formula integrates independent estimates of values and evidence. The quantity E represents the strength of the empirical evidence assessed through methodological analysis. The quantity V represents the strength of all non-evidential factors, such as personal values of the decision maker, and cost/benefits of the outcome of the contemplated action (A). Although the assumption of complete independence between evidence and values is difficult to maintain in many situations, my students report that it helps clarify and categorize the objects of their thinking. According to this qualitative model, if A exceeds the individual's "threshold" for action, the particular course of action is recommended or warranted. When alternative actions are under consideration, that alternative yielding the highest value of A is recommended. In the next clip, a student grapples with the relationship between values and evidence in decision making. Holding very strong (religious) values about the negative consequences of using marijuana, alcohol, and tobacco, she cannot envision any kind of evidence (displayed in a graph) that could alter her beliefs or behavior. Capstone Dialogue. One topic that generates a powerful confrontation between reason and emotion- based thinking is the treatment of terminally ill patients. In an effort to bring together the principles of critical thinking and informed decision making, I ask the students to confront a life-and-death scenario. In the course's capstone dialogue, I ask the students to react to the following article: Alzheimer's Dilemma This week the U.S. medical research community dropped a bombshell guaranteed to cause thousands of people to call their family doctors for more information. A researcher at a major university announced that a drug that he had administered to 17 patients with Alzheimer's disease had caused significant improvement in 16 of them. "Of these 17," he said, "four of them got dramatically better, seven got clearly better, and five of them got better to anybody's eye." The results with the drug, called THA, were indeed remarkable. One Alzheimer's victim is again playing golf; another went back to work part time. Another drives her car again, cooks and takes care of her own house. These results are simply mind-boggling to the many people in this country who have become familiar with the dreadful nature of Alzheimer's disease. Between 1.5 million and 3 million people have it; about 100,000 people die of it annually. Quite obviously the announcement will mean that husbands, wives, and children of Alzheimer's disease will call the family doctor to find out how soon the drug will be available to them. And the family doctor will tell all of them essentially the same thing. You can't get it. The doctors will say that the drug hasn't been approved for sale yet by the U.S. Food and Drug Administration. The FDA requires many tests to assure that the drug is safe and effective. And that the process can't be completed for years. Many of the callers to their doctor will reply, "But my father will be dead by then, and he and we are already living in hell with this disease." In this exercise, I ask the students to imagine they had a father in the early stages of Alzheimer's disease. Based on the information in the article, would http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/111298.smil Effective Thinking: An Active-Learning Course in Critical Thinking 9 they be willing to take their father to this physician for treatment? Through role playing, first in self-directed exercises in small collaborative groups and later in a larger class discussion, the students explore the issue through the perspective of the patient, his family, the FDA administrator, the family doctor, an Alzheimer- disease activist, and an ordinary citizen presently unaffected by this affliction. This role-playing exercise seeks to create a moral dilemma between what they know (scientifically) and what they feel and value. The primary question addressed by the family in this exercise is the following: Should they give their father a "shot" and allow him to receive an untested, "promising" drug? At the outset of the discussion, moved by their desire to preserve life at all costs, two students recommend to a loved one that they take this totally untested drug. In Table 1, we apply the E x V = A model to the Alzheimer's dilemma in an attempt to understand how of the parties can reach justified (though different) decisions. Table 1. E x V = A model for the Alzheimer's dilemma * * Although this theoretical framework is not a computational model, for purposes of illustration, we present quantitative estimates of E and V. Role E V Action Patient .01 Life maintained 10 6 Family member .01 Doing everything possible 10 5 Doctor .01 Conflicted between "interests" of patient and scientific evidence 10 3 - 10 - 3 FDA official .01 Scientific evidence, public safety 10 -3 Citizen (unaffected directly) .01 Compassion 10 ? First, we assume that the results of the physician's preliminary study (E) can be evaluated objectively. The small size of this study's sample, biased observations of the drug's effectiveness, lack of a control group, and a failure to consider side effects all suggest the evidence is very weak at best. Of importance to the application of this model is the assumption that estimates of E (based on a rational evaluation of the evidence) do not change as from participant to participant. The first student in the next clip cannot imagine ever being able to disregard his role in society in evaluating evidence. A second student, however, disagrees. She readily accepts the need to evaluate evidence objectively, and declares "The evidence doesn't change….Those are the facts." Next, we examine the (non-evidential) value factors (V). Clearly, the parties to the dilemma will differ greatly in their perception of moral obligations, legal rights of individuals, compassion, and societal benefits of rigorous medical testing. In contrast to estimations of E, there will be no agreement on the V factor. Thus, there will be no consensus on an appropriate course of action (as seen in Table 1). Indeed, often individuals will re-evaluate their initial commitment to act as they continue to evaluate the possible outcome of their decision. The class addresses this point in the next clip. You will note that we have expanded the discussion to include the related controversy issue of untested AIDS drugs. Students understand that there is little or no evidence of the drug's effectiveness. However, the students have different values about how long they would wish to live with the disease, and therefore reach different courses of action. Paralleling this personal dilemma is the consideration of the government's policy on compassionate (early) release of experimental drugs to terminally ill patients prior to fully establishing efficacy and safety through the arduous procedure of randomized clinical trials. The class examines the ethical/moral values of this societal concern in the final two clips (Clip 1 & Clip 2). Are the rights of the patient violated by the government's interest in preserving scientific purity? How do we reconcile http://hitchcock.dlt.asu.edu/media1/leshowitz/120898.smil http://cie.asu.edu/volume2/number5/index.html#table1 http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/120898b.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/120898b.smil http://cie.asu.edu/volume2/number5/index.html#table1 http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/120898c.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/120898d.smil http://hitchcock.dlt.asu.edu:8080/ramgen/media1/leshowitz/120898e.smil Current Issues in Education Vol. 2 No. 5 10 feelings of compassion and rational thought in matters of life and death? Students (i.e., unaffected citizens in the role play) express varying opinions about whether they would sign a petition mandating that the FDA release untested drugs to terminally ill patients: One student says "no" based on the lack of evidence showing effectiveness. A second student believes it is her right to be treated in the manner she deems most appropriate. Another student wants to "look into it." And the final student concludes "it would not be in the best interest of everybody" and that these drugs should not be released. Final Comment Students in the class are led through instruction that results in conceptual change. They move from a position of unquestioning acceptance of information to one of methodological skepticism and assessment of non- evidential factors. Table 2 summarizes the changes in students' conceptions of knowledge, metacognitive strategies, and methodological reasoning. Table 2. Levels of Development Level Knowledge claims Source of Knowled ge Dispositio n Instructio nal Events Level of Epistemolog ical Understandi ng (Kuhn, 1999 ) Critical Thinking 1. Pseudo- knowing Undifferentia ted fact/ opinions/ values; truth is knowable Direct observatio n (experienc e); authority Biased information assimilation Reflective dialogue and confrontatio n of present ways of thinking Absolutist Little or none 2. Destabiliz ed knowing Opinions - only accountable to their owners; nothing can be known Diversity of sources; all knowledge is equal; subjective Systematic skepticism Expose beliefs/valu es in real- world, problem- based exercises Multiplist Thought to be irrelevant 3. Reasoned knowing Statements of evidence and opinion are distinguishe d; evidence meets criteria; uncertain Knowing is a process Methodolog ical skepticism Methodolog ical (scientific) reasoning exercises; Integrating evidence and values in optimal decision making Evaluative Valued to promote informed decision making, effective problem solving, metacogniti on, and personal control http://cie.asu.edu/volume2/number5/index.html#table2 Effective Thinking: An Active-Learning Course in Critical Thinking 11 That such conceptual change can occur in a classroom environment may be surprising to some readers. Thagard (1992) wrote that conceptual change can occur by discovery or instruction. In the case of instruction, the instructor must, "develop rules and procedures that are sufficiently coherent... to supplant the existing rules that otherwise would take precedence" (p.58). Thagard explains that consistently using the new conceptual system builds up the strength of all of the rules of the system to the point where they are stronger than the pre-existing rules. In this case, using the rules of methodological reasoning repeatedly strengthens them so the students come to rely on them instead of intuition and gut reaction. Conceptual change does not occur easily in all students. As my role of instructional leader is reduced and I am no longer perceived as the primary source of knowledge, some students may express feelings of incertitude, frustration, and self-doubt. However, once they conceptually organize their ideas about these new thinking processes and begin to get the "big" point of the discussions, the students experience gratification and self- satisfaction. This observation is consistent with the line of research supporting the constructivist approach to education, which posits that active discovery, construction and integration is the basis for effective learning (Garfield, 1993; Gill-Perez, 1996; Resnick, 1987; von Glaserfeld, 1987). In order to produce the conceptual changes described here, I have found this active discovery method essential. Summing up, this course in Effective Thinking does not provide shortcuts to solving problems. Rather, it encourages participants to perceive problematic situations through the perspective of others and encourages an enlightened pluralism in the context of the critical thinking process. References Adey, P. (1988). Cognitive acceleration: Review and prospects. International Journal of Science Education, 10(2), 131-134. American Association for the Advancement of Science. (1989). Science for all Americans. Washington, D.C. Arambula-Greenfield, T. (1996). Implementing problem- based learning in a college science class. Journal of College Science Teaching, 88, 26-30. Arons, A. B. (1979). Some thoughts on reasoning capacities implicitly expected of college students. In J. Lochhead & J. Clement (Eds.), Cognitive process instruction. Philadelphia: Franklin Institute Press. Baron, J. (1988). Thinking and deciding. Cambridge, England: Cambridge University Press. Becker, H.J. (1998). Running to catch a moving train: Schools and information technologies. Theory into practice, 37, 20-30. Bunge, M. (1991). A skeptic's beliefs and disbeliefs. New Ideas in Psychology, 9, 131-149. Collins, A. (1993). Performance-based assessment of biology teachers: promises and pitfalls. Journal of Research in Science Teaching, 30, 1103-1120. Gil-Perez, D. (1996). New trends in science education. International Science Education, 19(2), 889-0901. Goodlad, J.I. (1983). A place called school. New York: McGraw Hill. de Sanchez, M. (1995). Using critical-thinking principles as a guide to college-level instruciton. Teaching of Psychology, 22, 72-74. Garfield, J. (1994). How students learn statistics. International Statistical Review, In press. Gigerenzer, G. (1996). On barrow norms and vague heuristics: A reply to Kahneman and Tversky (1996).Psychological Review, 103, 684-704. Halpern, D.F. (1989). Thought and knowledge: An introduction to critical thinking. Hillsdale, NJ: Lawrence Erlbaum Associates. Jungwirth, E. & Dreyfus, A. (1990). Diagnosing the attainment of basic inquiry skills: The one hundred year old quest for critical thinking. Journal of Biological Education, 24, 42-49. Kahneman, D., & Tversky, A. (1996). On the reality of cognitive illusions. Psychological Review, 102, 582-591. Kitchener, K. S., Brenner, H. G. (1990). Wisdom and reflective judgment: Knowing in the face of uncertainty. Sternberg, R. J. (Ed.), Wisdom: Its nature, origins and development. Cambridge: Cambridge University Press. Kuhn, D. (1991). The skills of argument. New York: Cambridge University Press. Kuhn, D. (1999). A developmental model of critical thinking. Education Researcher, 28, 16-26. Layton, D. (1986). Science, technology and society courses: Problems of implementation in school systems. In D. Layton (Ed.), Innovations in Science and Technology Education, Vol. 1. Paris: UNESCO. Leshowitz, B. (1989). It's time we did something about scientific illiteracy. American Psychologist, 44 (8) 1159-1160. Leshowitz, B., Jenkins, K., Heaton, S., & Bough, T. (1993). Effectiveness and benefits of an instructional program for developing critical thinking skills in students with learning disabilities. The Journal of Learning Disabilities, 26,483-490. Current Issues in Education Vol. 2 No. 5 12 Leshowitz, B. & Meyers, J.B. (1996). Application of decision theory to DUI assessment. Alcoholism: Clinical and Experimental Research, 20, 1148- 1152. Leshowitz, B., Okun, M, & DiCerbo, K.E. (1999). The effects of methodological reasoning on biased information processing and attitude polarization. Manuscript submitted for publication. Leshowitz, B. & Yoshikawa E. (1996). On instructional model for critical thinking. Inquiry, 15, 17-37. Leshowitz, B., Reich, J., Meyers, J, Brown, S & Chavez, A. Some consequences of knowing how to know. Submitted to: Inquiry. Lipman, Matthew (1989). Critical thinking: What can it be? Resource Publication Series 1, No. 1. Upper Montclair, NJ: Institute for Critical Thinking. Loria, A., Mantovani, S., & Nasi, R. (1984). A Piagetian analysis of the cognitive levels of science teachers.European Journal of Science Education, 4, 199-208. McKeachie, W. J. (1986). Teaching tips. Boston, MA: D.C. Heath and Company. National Science Foundation. (1996). Shaping the future: New Expectations for undergraduate education in science, mathematics, engineering, and technology. Nisbett, R.E., & Ross, L. (1980). Human Inference. Englewood Cliffs, NJ: Prentice-Hall. Paul, R. (1990). Critical thinking: What every person needs to survive in a rapidly changing World. Rohnert Park, California: Center for Critical Thinking and Moral Critique, Sonoma State University. Perkins, D. N. (1985). Postprimary education has little impact on informal reasoning. Journal of Educational Psychology, 77 (5), 562-571. Thagard, P. (1992). Conceptual Revolutions. Princeton, NJ: Princeton University Press. Thelen, L. V. (1987). Values clarification: Science or nonscience. Science Education, 71 (2), 201-220. Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161- 188. Whimbey, A., & Lochhead, L. J. (1986). Problem Solving and Comprehension. Hillsdale, NJ: Erlbaum, Inc. Effective Thinking: An Active-Learning Course in Critical Thinking 13 1999 Article Citation Leshowitz, B., DiCerbo, K. E., & Symington, S. (1999, September 30). Effective thinking: An active -learning course in critical thinking. Current Issues in Education[On-line], 2(5). Available: http://cie.ed.asu.edu/volume2/number5/. Author Notes The research described here was supported by a grant from the College of Liberal Arts and Sciences, Arizona State University. The authors thank Marc Van Horne in the College of Extended Education for his help with editing and archiving the video clips.Department of Psychology, Arizona State University,Tempe, AZ 85287-1104. Correspondence may be sent to the first author: Barry Leshowitz. Barry Leshowitz is an Associate Professor of Psychology at Arizona State University where he teaches courses in Effective Thinking and Research Methods. He actively researches, develops, and demonstrates college and high school programs of instruction that enhance the inquiry, critical thinking, and problem-solving skills of students. Kristen Eignor DiCerbo is a doctoral student in School Psychology at Arizona State University. Kristen Eignor DiCerbo can be reached via e-mail at kristen.dicerbo@asu.edu. Scott Symington is currently enrolled in the Ph.D. program in Clinical Psychology at Fuller Theological Seminary in Pasadena CA. Scott Symington can be reached via e-mail at symington@aol.com. Note from the 2011 Executive Editor, Lori Ellingford October 3, 2011. This article was first published at the original Current Issues in Education website, located at http://cie.asu.edu/articles/index.html. In 2009, CIE changed online platforms to deliver the journal at http://cie.asu.edu. The original CIE website is now only used as an archival repository for published articles prior to Volume 12. Efforts to make the current CIE website inclusive of past publications have necessitated the repurposing of this article into the published format you are viewing now. All content from the original publication has been preserved. No content edits occurred. Spelling, grammar, and mechanical errors that may be found were present in the original publication. The CIE logo and publisher information in use at the time of the article’s original publication is unaltered. Please direct questions about this article’s repurposing to lori.ellingford@asu.edu. Questions related to all other aspects of the journal may be addressed to cie@asu.edu. 2011 Article Citation Leshowitz, B., DiCerbo, K. E., & Symington, S. (2011). Effective thinking: An active-learning course in critical thinking. Current Issues in Education, 2(5). Retrieved from http://cie.asu.edu/ojs/index.php/cieatasu/ article/view/831 http://cie.asu.edu/volume2/number5/index.html mailto:leshowitz@asu.edu mailto:kristen.dicerbo@asu.edu mailto:symington@aol.com http://cie.asu.edu/articles/index.html http://cie.asu.edu/ lori.ellingford@asu.edu cie@asu.edu http://cie.asu.edu/ojs/index.php/cieatasu/article/view/820 http://cie.asu.edu/ojs/index.php/cieatasu/article/view/820 Current Issues in Education Vol. 2 No. 5 14