Weinstock-Infactispax-final Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 56 Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ Epistemic Understanding and Sound Reasoning Skills that Underlie Effective Democratic Engagement Michael Weinstock Ben-Gurion University of the Negev Beer-Sheva Israel Epistemic Understanding and Sound Reasoning Skills that Underlie Effective Democratic Engagement A voice in government, mostly through voting in elections, is commonly considered one of the hallmarks, if not the central activity, of citizenship in a democracy. However, the mere act of voting is a minimal expression of democracy. This point is made all too clear in countries with elections but without freedom of expression that cannot rightly be called democratic. What stands behind voting as a true democratic act is deliberation about for whom and what to vote. Such deliberation depends not just on the freedom to do so, but the ability to think critically and provide valid justifications of choices. Deliberation in a democracy has relevance beyond voting in elections. Many of the tasks required of citizens involve complex reasoning about everyday, non-schooled problems. One might imagine that reasoning ability would influence one’s effectiveness in a range of civic functions such as the advocacy and evaluation of policies and juror decision making. This paper reviews research on the relationship between epistemic understanding and skills of argument construction and evaluation. As defined by Perkins1, epistemic understanding is knowledge of knowing how to apply justification and explanation in subject matter. The research reviewed here demonstrates how such understanding underlies skills in 1 See David N. Perkins, “Reasoning as It Is and Could Be: An Empirical Perspective,” In Thinking across cultures: The Third International Conference on Thinking, eds. Donald M. Topping and Doris C. Crowell (Mahwah, NJ: Erlbaum, 1989, and David N. Perkins, Smart Schools: From Training Memories to Educating Minds (New York: The Free Press, 1992). Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 57 knowledge justification that are essential in effective deliberation. Although there are many arenas in which such skills are important, the emphasis here will be how they are essential in the realm of citizenship in a democracy. In particular, epistemic understandings that there may be multiple, competing, possibly legitimate knowledge claims, and that knowledge claims can and need to be justified in the context of alternative claims would be particularly relevant to deliberation in a democracy. Moreover, such epistemic understandings have been found to be related to intellectual values that dispose people to engage in considering complex, uncertain social problems.2 Such values would clearly contribute to effective citizenship. So as not to overstate the case by claiming that personal epistemologies underlie democracy, it should be made clear that this paper is confined to the deliberative process that contributes to effective democratic engagement, and is not concerned with other important characteristics and values of democracy such as the protection of equal rights and the rule of law. Nevertheless, there is a view within political science and philosophy that places deliberation at the center of democracy.3 Among the proposed outcomes of the process of deliberative democracy are increased tolerance for others’ views and positions that are better thought through and justified. From the perspective of the development of epistemic understanding, these outcomes may well be developmental outcomes from engagement in deliberation, such as in educational contexts as well as the necessary antecedents to the sound reasoning required for effective deliberation.4 Epistemic Understanding and Its Development The field of epistemology is concerned with how we come to know things, the source of knowledge, standards of knowledge justification, the limits of knowledge, and disagreements about knowledge. As the topic of concern of here is the everyday reasoning capacity of ordinary citizens, the forms of personal epistemology discussed is distinct from the worked-out and carefully articulated positions of philosophers concerned with epistemology. Regarding the nature of personal, or folk, epistemology, Richard Kitchener wrote:5 2 Kuhn, Deanna, and Seung-Ho Park, “Epistemological Understanding and the Development of Intellectual Values,” International Journal of Educational Research 43 (2005): 111-124. 3 Regarding deliberative democracy see Benhabib, Seyla, The Claims of Culture: Equality and Diversity in the Global Era (Princeton, NJ: Princeton University Press, 2002); Cohen, Joshua, “Deliberative Democracy and Democratic Legitimacy,” in The Good Polity, eds. Alan Hamlin and Philip Pettit (Oxford: Blackwell, 1989); Dryzek, John, Deliberative Democracy and Beyond: Liberals, Critics, Contestations (Oxford: Oxford University Press, 2000); Fishkin, James, The Voice of the People: Public Opinion and Democracy (New Haven: Yale University Press, 1995); Gutmann, Amy, and Dennis Thompson, Why Deliberative Democracy? (Princeton, NJ: Princeton University Press, 2004); Habermas, Jürgen, Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy, trans. William Rehg (Cambridge MA: MIT Press, 1996); Mansbridge, Jane, Beyond Adversary Democracy (Chicago: University of Chicago Press, 1983). 4 Kuhn, Deanna, and Wadiya Udell, “The Development of Argument Skills,” Child Development 74 (2003): 1245-1260; Kuhn, Deanna, and Michael Weinstock, “What is Epistemological Thinking and Why Does It Matter?” in Personal Epistemology: The Psychology of Beliefs about Knowledge and Knowing, eds. Barbara K. Hofer and Paul R. Pintrich (Mahwah, NJ: Erlbaum, 2002), 121-144; Warren, Joshua, Deanna Kuhn, and Michael Weinstock, “How do Jurors Argue with One Another?” Judgment and Decision Making 5 (2010): 64- 71. 5 Kitchener, Richard F., “Folk epistemology: An introduction,” New Ideas in Psychology 20 (2002), 89. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 58 Folk epistemology may be defined to be the ordinary (“folk”), common sense theory of knowledge present in the average person. Like its earlier counterparts—folk psychology, folk physics, folk biology, etc.—folk epistemology may be considered to be our “untutored” views about the nature of knowledge. Just as folk psychology is our ordinary, common sense theory of the mind, so folk epistemology is our ordinary, common sense theory of knowledge. The question addressed here is how such theories of knowledge come into play when people are confronted with knowledge claims or evidence in which the task is to construct a knowledge claim. Although this is the situation of people in many contexts—school, personal medical and health decisions, consumer choices—the focus here is on how personal epistemologies come to play in tasks relevant to effective participation in democratic deliberation. Personal epistemologies are “theories in action” in the sense that we enact them when making knowledge judgments in our everyday lives.6 These are the epistemologies of lay people and not philosophers. One of the approaches to research on personal epistemology focuses on how people understand the nature and source of discrepant claims.7 The roots of the interest are in research about college student development8 that found that a salient issue among students was the realization that there were no certain truths because of the disagreements they had with other students about ideas, beliefs, and values and disagreements among experts (i.e., professors) about what is known. This approach to personal epistemology, thus, has relevance to development in educational institutions and the issue of primary focus in this article, deliberation between people with different perspectives. Researchers from this approach have found that people’s epistemic beliefs tend to group around three positions. The “absolutist” position is one in which knowledge and knowing are conceived as objective and absolute. Disagreements about knowledge occur because we do not yet have access to the right facts, or because all but one view is distorted, incorrect, or biased. Knowledge is seen as self-evidently when true when justified by presenting the correct facts. For instance, an absolutist would explain the discrepant accounts of historians by saying that one is right because of access to the right information, and the other is wrong with that access, or because the historian has distorted the information because of error or bias. The correct account would be considered as certain knowledge. The “multiplist” position regards all knowledge as subjective and relative and, therefore, indeterminate, because of there are multiple idiosyncratic points of view. Disagreements arise because knowledge is essentially opinion, and everyone has a right to his or her own opinion. As knowledge is seen as subjective construction, a multiplist may understand that knowledge needs to be justified, but might not value the effort of making arguments from evidence to justify a point of view as such points of view are ultimately biased opinions. For example, a multiplist would say that two historians give discrepant accounts because each is biased. An American historian of the Vietnam War, for instance, 6 Kuhn and Weinstock, 2002. 7 King, Patricia. M., and Karen S. Kitchener, Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults (San Francisco: Jossey-Bass, 1994); Kuhn and Weinstock, 2002. 8 Perry, William G., Forms of intellectual and ethical development in the college years (New York: Holt, 1970). Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 59 might be motivated to selectively choose or distort information to support the American side, whereas a Vietnamese historian would inevitably be biased to support the (North) Vietnamese side. The “evaluativist” position is characterized by the acceptance and integration of subjective and objective aspects of knowledge that would permit a degree of evaluation and judgment of knowledge claims. Disagreements arise because people’s perspectives have them emphasize different information and employ different knowledge construction and evaluation methods. Thus, an American historian of the Vietnam War might give a different account than a Vietnamese historian of same war, which is tellingly called in Vietnamese, the American War. No matter what their personal views of the war or their countries, they emphasize different information because they segment historical periods differently, they come with different knowledge bases and historical archives, they address different issues particularly if they are writing for American or Vietnamese audiences. In short, their accounts are based on their best interpretations of the evidence from different historical and cultural perspectives. Characteristics of epistemic understanding in each position, with particular attention to competing knowledge claims and knowledge justification, are outlined in Table 1. Table 1 Characteristics of levels of epistemological understanding Epistemological Level Characteristics Absolutist Multiplist Evaluativist 1. Nature of knowledge Absolute, objective Subjective, relative Integration of objective and subjective aspects 2. Knowledge claims Description of reality Opinions, biased perspectives Construction from knower’s’ perspective 3. Source of discrepant claims One claim is incorrect or biased Multiple viewpoints, different opinions Different evaluations, interpretations and emphases on evidence 4. Adjudication of discrepant claims With appeal to objective reality Not possible; people have a right to their opinions With evaluation of how claim explains evidence within a perspective 5. Certainty possible? Yes No, multiple possible opinions No, as knowledge is constructed, not given 6. Nature of justification Report of facts Assertion of opinion Evaluation of evidence and perspective Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 60 These positions have been suggested to be developmental levels.9 That is, people start as absolutists, become multiplist as they lose their belief in objective knowledge and, in its place adopt very relativist beliefs, and then some move on to the more conceptually relative evaluativism. Importantly, education to a greater degree than age has been found to be a factor in epistemic development.10 It should be noted that although evaluativism is seen as the endpoint of the developmental trajectory, attainment of this level is hardly universal with some studies showing that it is typical only of those with some graduate school education.11 However, other studies, using more age appropriate assessments have found that by the end of high school, a substantial number of students can be characterized as evaluativist.12 Lest the story line which has been referred to—that a type of epistemic development best enables effective democratic citizenship—sounds too perfectly prescriptive, research on cultural differences in epistemic understanding challenges the universality of proposed course of development. Epistemic positions might reflect cultural values,13 and such some cultural values might not be consistent with Western values of liberal democracy that champions individual rights and equal opportunity. The apparent relationship between epistemology and democracy might reflect a particular epistemology that can develop in a liberal democracy. If the claim is that a relativist or evaluativist way of thinking is a necessary basis for effective democratic citizenship, would this mean that cultures that do not aim toward such an epistemology cannot be democratic? For instance, research on Bedouins in Israel shows that they are decidedly less multiplist than Americans or more western Jewish Israelis.14 An assessment of cultural values found that Bedouins value autonomous decision-making, personal independence, and egalitarianism less than those from western cultures and those in western liberal democracies.15 There is a greater value of respect for authority. Nevertheless Bedouin Israelis 9 Hofer, Barbara K., and Paul R. Pintrich, “The Development of Epistemological Theories: Beliefs about Knowledge and Knowing and Their Relation to Learning,” Review of Educational Research 67 (1997): 88-140; Kuhn and Weinstock, 2002. 10 Hofer and Pintrich, 1997; King and Kitchener. 1994; Kuhn, Deanna, Richard Cheney, and Michael Weinstock, “The Development of Epistemological Understanding.” Cognitive Development 15 (2000): 309-328; Perry (1970). 11 Weinstock, Michael, and Matthew A. Cronin, “The Everyday Production of Knowledge: Individual Differences in Epistemological Understanding and Juror Reasoning Skill,” Applied Cognitive Psychology 17 (2003): 161-181: King and Kitchener, 1994; Kuhn, Deanna, The Skills of Argument (New York: Cambridge University Press, 1991). 12 Kuhn et al., 2002; Weinstock, Michael P., Yair Neuman, and Amnon Glassner, “Identification of Informal Reasoning Fallacies as a Function of Epistemological Level, Grade Level, and Cognitive Ability,” Journal of Educational Psychology 98 (2006): 327-341. 13 Kuhn and Park. 2005; Tabak, Iris, and Michael Weinstock, “A Sociocultural Exploration of Epistemological Beliefs,” in Knowing, Knowledge and Beliefs: Epistemological Studies Across Diverse Cultures, ed. Myint Swe Khine (Netherlands: Springer, 2008) 177-195. 14 Weinstock, Michael, “Cultural Values and Epistemological Development,” (paper presented at the Seminar of the Developmental Forum of the Department of Psychology, University of California at Los Angeles, Los Angeles, USA, April 26, 2010); Weinstock, Michael, “Epistemological Development of Bedouins and Jews in Israel: Implications for Self-Authorship,” in Refining Understanding of the Development and Assessment of Self-Authorship, eds. Marcia B. Baxter Magolda, Elizabeth G. Creamer, and Peggy S. Meszaros, (Sterling VA: Stylus, 2010), 117-132. 15 Schwartz, Shalom H., Gila Melech, Arielle Lehmann, Steven Burgess, Mari Harris, and Vicki Owens, “Extending the Cross-Cultural Validity of the Theory of Basic Human Values with a Different Method of Measurement,” Journal of Cross-Cultural Psychology 32 (2001): 519-542; Weinstock, Michael, “Cultural Values and Epistemological Development,” 2010. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 61 are active participants in a democracy at both local and national levels. Moreover, among Bedouin participants in a recent study, those with evaluativist epistemologies were more likely to identify argument fallacies and value engagement in consideration of complex social issues.16 Thus, underlining the essential cautiousness of the claims in this article, epistemic understanding might be one factor in effective democratic engagement as far as deliberation contributes to the process. But, the type of citizenship, and the types of social interactions characteristic of different cultures also may well influence the type of public, political, and democratic decision-making that takes place. As most of the research on the types of epistemic positions and the course or their development, and on aspects of schooling and argument that will be described, has been performed in with populations characterized by western cultural values, the validity of my conclusions can only extend to western democracies until more research has been performed on the argumentation and epistemologies of non-western cultures. Education, Democratic Citizenship, and Argument A commonly stated, often official goal of schooling is to train students to participate effectively as citizens.17 Like many well-intended, but vague platitudes that make up mission statements of schools, the goals are not widely implemented,18 expressions of the goal of educating for citizenship generally are accompanied by neither suggested means for achieving that goal, nor any clear statement of behavioral objectives that would allow teachers to enact the education or evaluate whether the goal has been achieved.19 Moreover, although stated as the goal of schooling in general, often no one has the specific task of carrying out this goal. In American schools, fostering civic participation is deemed the job of all teachers with possible special attention in regular social studies classes.20 There are some guidelines for what social studies and other teachers must teach to educate students to become effective citizens,21 although it is not clear how widely these are implemented. In this paper, I propose that beyond knowledge of democratic values and the nature and function of governmental institutions, an important aspect of effective participation in democratic society is informal reasoning ability and skilled argument, the implication being that this should be the focus of schools’ efforts to promote active, effective citizenship. One aspect of argument skills is the construction and evaluation of sound knowledge claims that are necessary for critical discussion aimed at producing the best fit between theoretical 16 Weinstock, Michael, “Cultural Values and Epistemological Development,” 2010. 17 For example, Cogan, John J., and Patricia V. Pederson, “The Development of Civic Values: Case Study of the United States of America,” International Journal of Educational Research 35 (2001): 61-76; Glickman, Carl, “Educating for Citizenship,” The School Administrator 65 (2008): 18-23; H. Res. 1804, 1994;; Wade, Rahima C., and Susan Everett, “Civic Participation in Third Grade Social Studies Textbooks,” Social Education 58 (1994): 308-11. 18 Glickman, 2008; Haynes, Charles, C., and Terry Pickeral, “Renewing the Civic Mission of Schools,” The School Administrator 65 (2008): 10-15; Wade, Rahima C., “Social Action in the Social Studies: From the Ideal to the Real,” Theory Into Practice 40 (2001): 23-28. 19 Allen, Lew, “From Plaques to Practice: How Schools can Breathe Life into Their Guiding Beliefs,” Phi Delta Kappan 83 (2001): 289-293. 20 Cogan and Pederson, 2001. 21 Torney-Purta, Judith, and Britt S. Wilkenfeld, Paths to 21st Century Competencies Through Civic Education Classrooms: An Analysis of Survey Results from Ninth-Graders. (A Technical Assistance Bulletin) (Chicago: American Bar Association Division for Public Education, 2009). Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 62 propositions and evidence for those propositions. Although this should be a part of the construction and discussion of any policy proposal, the research described is confined to the function of juror decision-making as an exemplar of a complex reasoning task that many citizens from countries with roots in the English legal system (e.g., the United States, Canada, the United Kingdom, Ireland, Australia, New Zealand, Jamaica, Belize, Malawi, etc.), as well as other countries, (e.g., Spain, Brazil, and Russia), might be called upon to perform. Another aspect of argument skill that will be considered here concerns the ability to criticize fallacies in argument. Politicians and advocates for positions do what they can to persuade others of the merits of their viewpoints. Arguments for positions might be persuasive, but fundamentally unsound. Hopefully, citizens in a democracy will be equipped to think critically about arguments. Juror Reasoning The empirical claim I test is that skills of argument underlie effective participation in one institution, the jury, found in a number of democratic countries. The research presented here investigates how differences in the general cognitive skill of argument might influence performance on a task that is, on one hand, a specific instance of democratic participation and, on the other, representative of complex, everyday reasoning tasks that people perform without the benefit of specific education. Complex tasks that impact on democratic participation would include evaluating the statements of politicians or arguments about public referendums that accompany sample ballots and appear in the media before an election. Argument Skills The cognitive skills directly relevant to juror reasoning tested in the studies are defined in Table 2. They are illustrated with examples taken from the protocols of interviews used in this research project. The interviews consisted of participants’ justifications for their verdicts in two abridged jury trials that were based on actual cases. The reference to “Richards” in the Table concerns a case in which a teenage boy killed his abusive father after getting in a fight with him in their house. The reference to “Johnson” concerns a case about a killing in the parking lot of a bar in which the victim and defendant allegedly fought with blades. Table 2 Definitions, examples, and explanations of argument skills in the juror task Skill and Definition Example and explanation A. Representation of verdict and evidence Representation of verdict [As this is a numerical variable summarizing the whole Correct identification of category interview, there is no example given. It consists of a verdict choices. count of the number of verdicts referred to by the verdict criteria named in the judge’s instructions.] Evidence coverage [As this is a numerical variable summarizing the whole Amount of testimony accounted interview, there is no example given. It consists of a Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 63 for. count of every piece of testimony mentioned in the course of the participant’s verdict justification] B. Argument skills Simple argument [Participant 22 in the Richards case refers to piece of Testimony related to verdict testimony in relation to a category criterion for M1] category criteria It would be 1st degree. I’d say he really had intention to kill his father, because he went out and brought a gun in the house. Counterargument [P44 in the Richards case chose SD—but here Reference to evidence that supports spontaneously offers evidence in support of M1] an alternative verdict choice or I was tempted to say that it could be murder in the 1st discounts one's own verdict choice. degree because the fact that he did have a gun, and the (table continues) Skill and definition Example and explanation fact that it seemed like he had a kind of a plan because he had put the dogs in the basement. Discounting of alternative verdicts [P170 in Richards, who chose M1, discounts a criterion Explanations of why alternative for SD referring to evidence] verdicts were not chosen. The only problem with self-defense is he didn’t leave. In other fights that he had with his father, he ran to the girlfriend’s house. Justification of alternative verdicts [P58 in Richards, who chose SD, uses evidence in Anticipating how other jurors relation to category criterion and offers an argument that might argue for alternative someone might make for M1] verdict choices I think that someone might choose1st degree because of the hatred in the1st degree. Definitely the boy would have hated his father for the bad things he had done. . And the gun that was there before it happened. I guess those two would lead someone to choose 1st degree. Judgment of evidence [Plausibility: P90 in Johnson, judges the plausibility of Evaluation of the plausibility or testimony that the victim threatened the defendant with credibility of pieces of testimony. a razor, was stabbed, and then put the razor in his pocket]: It’s not self-defense. The razor was inside his pocket, I can’t see him, if he gets stabbed, that’s a bad stab (table continues) Skill and definition Example and explanation Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 64 wound. Going down, you don’t have time to have a razor and put it back in your pocket. [Credibility: P160 in Johnson, judges the credibility of testimony]: The policeman was 75 feet away, the bartender said there was a neon light there and they couldn't exactly see what really happened, so everybody really had a little problem focusing in on what really took place. Note. SD = self-defense, MS = manslaughter, M1 = murder in the 1st degree. Underlined words in the examples are statements of verdict category criteria. Words in italics in the examples are references to testimony. The inclusion of each skill has justification in the literature on juror or informal reasoning. As for the representation of the verdict choices and evidence (“A” in the table). Pennington and Hastie22 posit that in the first step of juror decision making, one adopts a story of what happened. The chosen story is determined in part by the amount of evidence a particular story covers (i.e., “evidence coverage” in Table 2). The next step of the process is the representation of the criteria of each of the possible verdicts (called “representation of verdict” in the table). These two steps are necessary to the final step of matching story to the most appropriate verdict choice. Although presented as relatively unproblematic by Pennington and Hastie, how much evidence is covered may depend on the skills of generating evidence and distinguishing evidence from explanation.23 Although it is expected that jurors consider all the evidence, whether they do so is open to question, as they are not necessarily expert reasoners. The representation of verdict choice has also been found problematic. Smith24 found that people do not always represent the legal definitions of verdicts correctly. They may call upon their knowledge of verdicts gleaned from the media rather than as instructed by a judge in a specific case. The argument skills (under “B” in Table 2) are derived primarily from the literature on informal reasoning. Reasoning about ill-structured problems—that is, problems that typically lack established problem-solving procedures and verifiable single solutions25 such as that faced by a juror—has been conceptualized as informal reasoning in which people must 22 Pennington, Nancy, and Reid Hastie, “Evidence Evaluation in Complex Decision Making,” Journal of Personality and Social Psychology 51 (1986): 242-258; Pennington, Nancy, and Reid Hastie, “The Story Model for Juror Decision Making,” in Inside the Juror, ed. Reid Hastie (Cambridge: Cambridge University Press, 1993), 192-221. 23 Kuhn, Deanna, 1991; Weinstock, Michael, and Robin A. Flaton, “Evidence Coverage and Juror-Reasoning Skill as Mediating Factors in a Juror’s Verdict Choice,” Journal of Behavioral Decision Making 17 (2004): 191- 212. 24 Smith, Vicki L., “Prototypes in the Courtroom: Lay Representations of Legal Concepts,” Journal of Personality and Social Psychology 61 (1991): 857-872; Smith, Vicki L., “When Prior Knowledge and Law Collide: Helping Jurors to Use the Law,” Law and Human Behavior 17 (1993): 507-536. 25 Galotti, Kathleen, “Approaches to Studying Formal and Everyday Reasoning,” Psychological Bulletin 105 (1989): 331-351. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 65 build arguments rather than resort to formal logic or scientific or mathematical proof.26 The recognition that evidence can be used to argue in support of a claim, against other claims, and against one’s own claim underlies the skill of sound argument. That is, one engaging in argument must recognize its two-sided nature.27 If argument were not at least two sided, there would be no need to argue or justify one’s claims—the truth of a claim would be self-evident. Kuhn28 argues that the sound justification of claims involves coordinating evidence with theoretical explanation within a framework of alternative claims, which are the possible verdict choices in the case of juror decision making. Thus, a juror should be able to present evidence in relationship to a verdict criterion (“simple argument” in Table 2), present evidence that might be used to argue against one’s own verdict choice (“counterargument”), explain why the other verdicts were not chosen (“discounting”), and anticipate arguments that others might make in favor of other verdicts (“justification of alternative verdicts”). In addition, one engaged in reasoning needs to assess the quality and credibility of the information as part of determining whether it can support a claim.29 Some, but not all jurors typically engage in evaluating the plausibility of testimony in relation to their real-world knowledge,30 called in Table 2 the “judgment of evidence.” For instance, in a jury case used in both those studies as well as the current study, jurors evaluated whether the defendant’s claim that the victim had a knife out is (a) credible given that the knife was found in the victim’s pocket, or (b) whether the victim could plausibly have put the knife in his pocket after being stabbed, as claimed by a defense witness. Perhaps more important than the derivation of the skills from literature on informal reasoning, each of the argument skills is essential to the work of the juror and should be assumed to part of a juror’s understanding of the nature of the task. One would assume that a juror comes to the juror decision-making task knowing that he or she would need to justify a verdict choice with reference to the evidence (simple argument in Table 2). Moreover, a juror most likely comes to the task knowing that jurors do not always agree with one another. Each of the skills of justification of alternative verdicts, discounting of other verdicts, and counterargument concern different uses of the evidence to reach different conclusions. Finally, at least anybody who has seen a courtroom drama on would know that the credibility of witness and the plausibility of claims are open questions. In sum, jurors not only come to their task with a range of argument skills and general understandings of the nature of knowledge and knowing, but particular epistemic understandings of the juror task.31 26 Kuhn, 1991; Means, Mary L., and James F. Voss, “Who Reasons Well? Two Studies of Informal Reasoning Among Students of Different Grade, Ability, and Knowledge Levels,” Cognition and Instruction 14 (1996): 139-178; Paul, Richard W., “Dialogical Thinking: Critical Thought Essential to the Acquisition of Rational Knowledge and Passions,” in Teaching Thinking Skills: Theory and Practice, eds. Jonathan B. Baron and Robert J. Sternberg (New York: W. H. Freeman and Company, 1987), 127-148; Perkins, 1989. 27 See Baron, Jonathan. “Myside Bias in Thinking about Abortion,” Thinking and Reasoning 1 (1991): 221-235; Kuhn, 1991; Perkins, 1989; Walton, Douglas N., Informal Logic: A Handbook for Critical Argumentation (Cambridge: Cambridge University Press, 1989). 28 Kuhn, 1991; Kuhn et al., 1994. 29 Kuhn et al., 1994; Pepper, Stephen C., World Hypotheses (Berkeley: University of California Press, 1942). 30 Kuhn et al., 1994; Pennington and Hastie, 1986. 31 Weinstock, Michael, “Like an Expert: Representing the Problem, Epistemic Requirements, and Competence in an Everyday Reasoning Task,” Learning and Individual Differences 19 (2009): 423-434. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 66 Verdict Choice The jury system rests on an assumption that jurors should engage in sound reasoning and effective consideration and evaluation of the evidence. Not only does our sense of justice depend on the belief that all verdicts have been considered with the best evaluation of all the evidence, but variation in reasoning skills might have an effect on the verdict someone could arrive at. Of course, to some degree this question is unanswerable because, as an informal reasoning problem, there may be no knowable right answer in the juror’s task as a standard by which to measure the success of an argument. However, Kuhn et al. found that there is a relationship between juror reasoning skill and the verdict chosen.32 In that study, the participants were presented with four possible verdicts in a murder trial, two of which— maximum guilt (first-degree murder) and innocence (self-defense)—had arguments laid out by the prosecution and defense respectively, and two of which (manslaughter and second- degree murder) were not addressed directly in the attorney’s cases. Accepting a verdict argued for by the attorneys (first-degree murder and self-defense) would demand less construction of a verdict from evidence and less consideration of alternative verdicts. Indeed, those less skilled in argument did tend to choose one or the other of the verdicts argued for by an attorney in both cases. In contrast, those who successfully discounted alternative verdicts or made plausibility judgments of evidence were more likely to choose manslaughter or second-degree murder in at least one case. Juror Reasoning and Epistemic Understanding A study of juror reasoning33 examined the relationship between personal epistemological beliefs and juror reasoning. The purpose of the study was to see if epistemological level when determined by assessing a broader range of epistemological dimensions than in previous studies would be related with the argument skills in the participants' verdict justifications. The sample for the project consisted of people serving jury duty in Brooklyn, New York at the Kings County Supreme Court or Civil Court. They were selected from the jury pool in the same random manner that prospective jurors are selected for impaneling. As such, they represented the wide diversity of ethnic groups, education levels, and income levels characteristic of that community. One hundred and eighty people (91 males and 89 females, ranging in age from 19-73) participated. Self-report of education levels showed 40% not continuing past high school, 25% with some college, 21% with a BA, and 14% having had some graduate or professional school education. Although unavailable for impaneling for part of a day, people returned to the pool after their participation for the remainder of their jury duty. The epistemological level of the participants was assessed by presenting them with two discrepant accounts of the obscure (actually unreal) Fifth Livian War between North and South Livia. Each one-page account was said to be written by an historian from the respective country. The accounts differed in the stated starting date of the war, which country was the aggressor, the underlying causes of the war, and which side seemed to have the upper hand when a third country intervened to stop the war. The participants were asked the following 32 Kuhn et al., 1994. 33 Weinstock, Michael, “Cognitive Bases for Effective Participation in Democratic Institutions: Argument Skill and Juror Reasoning,” Theory and Research in Social Education 33 (2005): 73-103; Weinstock and Cronin, 2003. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 67 questions: 1. Can you summarize what the Fifth Livian War was about and what happened? 2. Are the two historians' accounts of the war different in any important ways? Probe: In what ways are they different? 3. Could both of the historians' accounts of the Fifth Livian War be right? Probes: If no–why not? If yes–How can that be? Is one of the historians' accounts of the Fifth Livian War more true than the other? 4. Could anyone be certain of what happened in the Fifth Livian War? Probe: If yes–how? If no–Why not? 5. Would another historian's account of the Fifth Livian War be different from the accounts of the historians you heard? Probe: If yes–Why? From the responses it was possible to designate participants reliably as absolutist, multiplist, and evaluativist. A complex scheme34 was used to code the responses according to a range of epistemological issues such as the source of knowledge, the source and reconcilability of discrepancies, the need and standards for justification, the possibility of multiple accounts, and the role of bias and perspective in addition to certainty. Although, the questions did not ask the participants to address these issues directly, they emerged in the course of their responses. The description of the epistemic positions given in the introduction and Table 1 indicate the types of understandings that were included in designating which level was represented by an interview. In order to assess the argument skills, the participants heard short audio-taped enactments of two murder trials, complete with attorneys’ opening and closing statements, examination and cross-examination of six witnesses, and judge’s instructions to the jury. The participants were asked to choose among four verdicts, and to justify their choices before deliberating. Their justifications were coded for the argument skills described above and for their level of certainty about the verdict choice. Significant statistical relationships were found between each of the argument skills: the ability to justify alternative verdicts as well as to discount alternative verdicts, offer counterarguments, and evaluate evidence. Moreover, except for evidence evaluation, the level of skill was found to be associated across cases, indicating that generalized skill was a more important factor in the construction of argument than the specific content of the cases. In addition, those with absolutist epistemologies were the least skilled in each of the argument components (see Table 3). Consistent with the expected important shift between objectivist absolutism and subjectivist multiplism, less than half of the absolutists were skilled in each of the argument components, whereas a good deal more than half of the multiplists were skilled and displayed only slightly less skill and consistency between skills than the evaluativists. Whereas both epistemic understanding and educational level accounted for differences in argument skill, epistemic understanding was the predominant factor,35 and as indicated by Table 3. Table 3 34 See Kuhn and Weinstock, 2002. 35 Weinstock and Cronin, 2003. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 68 Tests for relationships of epistemological level and educational level with the argument skills Developmental Factor Skills Epistemological Level Educational Level VERD F(2, 158) = 3.68, η2 = .04* F(3, 156) = 3.25, η2 = .06* EVID F(2, 158) = 5.55, η2 = .07** F(3, 158) = 2.76, η2 = .05* SA F(2, 157) = 5.32, η2 = .06** F(3, 157) = 0.67, ns CA χ2 (4, 166) = 11.68*, γ = .35** χ2 (6, 165) = 3.69, ns DISC F(2, 158) = 5.66, η2 = .07** F(3, 158) = 2.42, η2 = .04+ JUST F(2, 157) = 7.25, η2 = .09*** F(3, 157) = 1.21, ns JUDGE χ2 (4, 171) = 16.91**, γ = .42**** χ2 (6, 170) = 8.90, ns, γ = .20* Note. VERD = verdict choice over two cases (both extreme or at least one moderate); EVID = the amount of evidence covered; SA = success at using evidence in relation to verdict criteria in arguments; CA = counterargument; DISC = discounting alternative verdict choices; JUST = justification of alternative verdict choices; JUDGE = judgments and evaluations of evidence credibility and plausibility. Continuous skill variables are tested with one-way ANOVAs with skill as the dependent variable. Categorical skill variables are tested with chi- square tests of association. Epistemological and educational level are categorical variables. + p = .068 *p < .05 **p < .01 *** p < .001 **** p < .0001 Of particular interest, the response to the certainty questions in the epistemological assessment (#4 above) was found to be correlated with certainty expressed about verdict choice.36 This justifies the assumption of an earlier juror study37 that personal certainty appears to be a function of personal epistemology and not particular content. Like in the other study, those most certain tended to choose one of the verdicts argued for by the attorneys in both cases.38 That is, those with less skill and most certain about their own point of view, and most likely to believe that one can be certain, apparently could consider only a narrow range of arguments for alternatives. Following their individual justifications of their verdicts, the participants deliberated in dyads to reach a verdict. Analyses of the dyadic interactions in deliberation found that those with more sophisticated epistemologies were more likely than absolutists were to control the discourse, making more statements, and to make meta-discourse statements that indicated reflection on the process of deliberation.39 Moreover, the quality of the discourse was higher among those with more sophisticated epistemologies. They referred more to verdict criteria than did absolutists, and they were more likely to question the claims of the other. On a group level, dyads consisting of two people with different epistemologies had 36 Weinstock, 2009. 37 Kuhn et al., 1994. 38 Weinstock, Michael, and Robin A. Flaton, “Evidence Coverage and Juror-Reasoning Skill as Mediating Factors in a Juror’s Verdict Choice,” Journal of Behavioral Decision Making 17 (2004): 191-212. 39 Warren et al., 2010. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 69 lower quality of discourse, further suggesting how epistemic understanding might impact on democratic deliberation. These findings, and others that show some polarization when people engage in debate about social issues,40 perhaps lend support to political theorists41 who worry that deliberation by less competent reasoners may not lead to effective democratic discussion, and may, in fact, lead to greater conflict and polarization than to resolution of competing claims. As applied to the larger political arena, the findings also echo those of Philip Tetlock who studied the speeches of US Senators. Similar to the jurors who chose the extreme verdicts of maximum innocence or guilt in both cases, and also tended give no real consideration to alternative verdicts, Tetlock found that American senators with right-wing ideologies presented issues with less integrative complexity and focused on simple conceptions of good vs. bad. Liberal and moderate senators displayed greater integrative complexity, pointing to different perspectives and comparing them to make the case for their own perspective.42 Conclusions: Epistemology and Juror Reasoning The studies reviewed make a strong case that people’s epistemic understandings of the requirements of knowledge justification and of the nature of knowledge are implicated in their reasoning and judgments about jury cases. Those with absolutist epistemic understandings are less likely to use make arguments from evidence for their own verdict choices, to explain why they did not choose other verdicts, to explain how others might have reached other verdicts, to found evidence that did not fit with their verdict choices, or to judge evidence. They were more likely to be certain about their verdict choices and more likely to have chosen one of the verdicts given an articulated argument by one of the attorneys. Evaluativists accounted for more of the evidence presented in the trials then either of the other multiplists or absolutists. Educational level also accounted for some of the differences in argument skills, but to a lesser degree than did epistemic position. In sum, the epistemic understandings that juror bring to their task seems to underlie the quality of their own deliberations and their deliberations with a partner. The lessons go beyond describing people’s approaches to complex reasoning task because the juror reasoning task is one that exists in many democracies and concerns social interactions and deliberations with serious social consequences. Moreover, the evidence about the relationship between epistemology and the quality of deliberation might be applied to deliberative processes that are parts of other aspects of democratic institutions. So as not to overstate the case, it should be clear that there are other aspects of deliberation that are not part of the limited juror's task. For instance, jurors do not need to seek out relevant information, and 40 Felton, Mark, and Deanna Kuhn, “The Development of Argumentive Discourse Skills,” Discourse Processes 29 (2001): 135-153; Lao, Joseph, and Deanna Kuhn, “Cognitive Engagement and Attitude Change.” Cognitive Development 17 (2002): 1203-1217. 41 Bohman, James, and William Rehg, eds., Deliberative Democracy: Essays on Reason and Politics (Cambridge, MA: MIT Press, 1997); Dahl, Robert, Democracy and Its Critics (New Haven: Yale University Press, 1989). 42 Tetlock, Philip E., ”Cognitive Style and Political Ideology,” Journal of Personality and Social Psychology 45 (1983): 118-126. It should be noted that his focus was on authoritarian political thinking, in which the ideological left might resemble the right, but in the case of the US Senate, there were no representatives of the ideological left to test this point. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 70 their task is to consider the specific case and not consider possible long range effects or larger social implications. The task of best prediction in policy making is one filled with even greater uncertainty and is arguably more difficult. Moreover, it might be argued that the specific critical thinking skills that apply to the juror task may not be transferred to other deliberation tasks.43 However, the meaning of the relationship found between epistemic understanding and argument skills is that people's understandings about knowledge construction and evaluation—their theories-in-action— and not specific learned skills that guide them when they confront tasks that involve knowledge. In an extended experiment, people who developed an understanding that coordinating theory and evidence was an aspect of knowledge justification on one task indeed applied that understanding to a different domain task.44 Moreover, as will be shown in the next section, epistemic understanding also plays a role in a very different type of knowledge evaluation task. Argument Fallacies Informal reasoning fallacies are arguments that are “psychologically persuasive but logically incorrect; that do as a matter of fact persuade but, given certain argumentative standards, shouldn’t.”45 As a case in point, relative to the issue of an informed, effective citizenship, William Safire wrote an opinion piece46 in order to argue against the claim of concocted evidence used in presenting the case for war against Iraq: No; the opponents of this genocidal maniac's removal now accuse President Bush and Prime Minister Blair of a colossal hoax. Because Saddam didn't use germs or gas on our troops, they say, that proves Iraq never had them. If we cannot find them right away, they don't exist. They believe Saddam sacrificed tens of billions in oil revenues for no reason at all. A strong majority of Americans believe he did have a dangerous program running, as he did before. This brief excerpt contains four nameable argument fallacies. The first sentence is an ad hominem argument, implying that those who belief the evidence was concocted support a genocidal maniac. This is a fallacy in that it imputes the character of the claimants without addressing the substance of their claims. The next two sentences are instances of ad ignorantiam argument, in which the existence of something is inferred from the lack of proof that it does not exist. In this statements (in the context of his argument elsewhere in the opinion piece), Safire argues that the lack of proof that that weapons of mass destruction did not exist, and lack of proof regarding the reasons for Saddam’s policy, is proof that they do (or could) exist or that there are particular reasons for the policy. His argument is echoing the belief that the lack of proof that there were or were not weapons is evidence that they were destroyed or hidden right before the war and does not acknowledge that one cannot conclude positively from a lack of proof. 43 For an argument against the possibility of transferring critical thinking skills see McPeck, John E., “Stalking Beasts, but Swatting Flies: The Teaching of Critical Thinking.” Canadian Journal of Education 9 (1984): 28-44. 44 Kuhn, Deanna, Mercé Garcia-Mila, Anat Zohar, and Christopher Andersen, “Strategies of Knowledge Acquisition,” Monographs of the Society for Research in Child Development 60 (1995). 45 Copi, Irving M., and Keith Burgess-Jackson, Informal Logic (Upper Saddle River NJ: Prentice Hall, 1996), 97. 46 Safire, William, “You Lied to Us,” New York Times, June 2, 2003, Editorial Page, National edition. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 71 The first sentence of the next paragraph makes an ad populum argument in which a claim is justified by stating that most people believe it is true. Here Safire is arguing that there must have been weapons of mass destruction because a majority of Americans believed it was the case. Again this is a fallacy because what a majority believes is actually irrelevant to sound evidence in support of the claim. He evades the issue of whether most Americans might have been wrong. It might said that Safire also makes an ad antiquarian argument in stating what was believed in the past is a justified reason for believing something now. He ignores the fact that Saddam had since lost a war and there had been decade of supervision and weapons inspections that may have produced changes in the situation and provided new evidence. It is possible that, nevertheless the weapons program continued. Evidence of this would have been a sound argument; just because it had been one way in the past is not. It should be noted that William Safire was performing his job, and perhaps doing a service in democratic debate whether or not one agrees with him, by trying to persuade as best as he could to move the public toward an ethical public good. Although I would hope that he, a noted language expert, could have done so according to the norms of critical discussion aimed at constructing sound knowledge, my concern is whether readers of this article could criticize the argument. To engage in democracy, citizens must engage in arguments such as this applying their best critical thinking. As much of the information regarding public policy issues and candidates’ platforms might come from the editorial pages or politicians’ speeches, it is important that people know how to evaluate the soundness of the claims made.47 A series of studies demonstrated that the ability to identify informal reasoning fallacies was related to epistemic understanding. Weinstock, Neuman, and Tabak48 presented adolescents with a number of scenarios in which, in the course of a critical discussion, someone makes a fallacious argument in support of a claim. The scenarios mostly included social and political issues, such as whether drugs should be legalized, special needs children should be mainstreamed in schools, the payment of taxes, and the sale of weapons to non- democratic countries. The participants were also assessed for their understanding of the norms of rational argumentation designed to construct knowledge. It was found that awareness of these norms was related to the ability to identify the specific problems in the arguments. A follow-up study49 investigated whether differences in grade-level and epistemic understanding could predict the identification of argument fallacies. The participants were presented with similar scenarios as in the earlier study with the addition of other social issues such as global warming, whether violence on TV promotes children’s violence, and whether children should be allowed to participate in the lottery. A couple of fallacy-free scenarios 47 As for politicians’ speeches, it should be noted that Safire was Richard Nixon’s chief speech writer in what was the first White House office created for the purpose of writing speeches, the Writing and Research Department, which Nixon called the "PR group." See Gelderman, Carol, All the Presidents' Words: The Bully Pulpit and the Creation of the Virtual Presidency, New York: Walker and Company. 1997. 48 Weinstock, Michael, Yair Neuman, and Iris Tabak, “Missing the Point or Missing the Norms? Epistemological Norms as Predictors of Students’ Ability to Identify Fallacious Arguments,” Contemporary Educational Psychology 29 (2004): 77-94. 49 Weinstock et al., 2006. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 72 were included to determine if adolescents could distinguish between fallacious and sound arguments. Their epistemic understanding was assessed with a paper-and-pencil assessment, in which there was a two-sentence “discussion” between two people who presented competing claims.50 The participants were asked to indicate of one of the claims had to be right and the other wrong (coded as “absolutist”), if both could be right and it could not be determined that one claim was more right than the other (coded as “multiplist”), or if both could be right to a degree, but one could be found to be more right than the other (coded as “evaluativist”). The assessment allowed for assignment to these three epistemological levels. First, it was found that most adolescents could distinguish between fallacious and non-fallacious arguments. But it is important to note that about one quarter could not. Moreover, only half could identify the specific fallacies. With each of the three fallacies used in the study (ad hominem, ad populum, and ad ignorantiam) grade level predicted the ability to specifically identify the fallacy with 11th graders doings so more than 7th or 9th graders. Epistemological level also significantly predicted the ability to identify the ad ignorantiam fallacy, controlling for grade level and cognitive ability as assessed with the Raven’s Matrices, which is a widely used test for abstract reasoning and non-verbal intelligence, such that evaluativists were the most likely to identify the fallacy. In all fallacies, the absolutists had the smallest percentage that could identify the specific problems with the arguments. Conclusion: Argument Fallacies In sum, it appears that adolescents are not all that bad recognizing problems in arguments, although there is clear room for improvement. There are differences that are related with educational level and the understanding of epistemic norms of knowledge justification. But, we are not such clever writers as William Safire. Although I do think that the results of the studies are valid, it is not hard to imagine that arguments that are not presented in the isolated manner of experiments would be harder to pick out in the contexts in which they do occur. In turn, in such non-experimental contexts, it would be harder to apply critical thinking. There are plenty of examples of straw person arguments, appeals to fear, and the above-named fallacies in political discourse. It might be that epistemic understanding of the norms of argument discourse and the standards of sound knowledge justification could be a basis for combating or criticizing the techniques of irrational persuasion commonly practiced in the political sphere. Short of—or even with—changing the discourse, a critically competent citizenship would benefit deliberation in a democracy. And as implied by the juror study, the ability to produce, and not just evaluate, sound arguments is important in democratic decision-making. The juror of citizen peers make decisions about whether to send people to jail, or whether to make substantial financial rewards in the case of civil wrongs. The ability to justify such socially-important decisions made about complex problems is essential for effective citizenship in a democracy. The evidence of the studies reviewed here indicates that understandings of the standards of knowledge justification and the recognition of multiple perspectives are important for critical, rational debate in a society. Educational Implications Schools could contribute such understandings by the implicit teaching of or 50 See Kuhn et al., 2000. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 73 supporting sound argument. As argued by Walter Parker,51 by virtue of their social diversity and the part they play in public life, schools are ideal contexts in which to teach the “art of deliberation.” However, neither standards-based education nor common current pedagogical practices in which students are encouraged to express their opinions52 and, in the name of “tolerance,” to withhold criticism because everyone has a right to his or her opinion provide teaching opportunities with true deliberation. As Blythe Clinchy points out, such tolerance is vacuous because accepting everyone's opinion as equally valid and without judgment precludes really listening to what the other is saying.53 An evaluativist perspective, which is critical, is necessary for true deliberation. But at all levels of education, argumentation among students does not commonly take place and it is not encouraged. Commentators on college students in the last decade54 have noted how students seem unwilling to engage in debate and deliberative argument with other students; they are unwilling to defend their own positions or criticize or discuss others’ opinions. The literary critic, Michiko Kakutani55 quotes Jeff Nunokawa, an English professor at Princeton: "Debate has gotten a very bad name in our culture. It's become synonymous with some of the most nonintellectual forms of bullying, rather than as an opportunity for deliberative democracy." However, placing a high value on “tolerance” for multiple perspectives, or encouraging deference to the other's perspective may not lend itself to developing critical thinking rooted in an understanding that an answer’s quality can be judged.56 Despite this, there is evidence that with some attention, students can learn to be more effective at deliberation and argument. Kuhn57 compares how two teachers use a debate format in the context of a history class. One of the teachers uses debate as a means of allowing the students to express the words and ideas of others in order to have them present the facts of the historical topic and to understand which side of the debate prevailed in history. The other teacher was less concerned with echoing the historical outcome, and more concerned with how well the students’ supported their arguments with the information available to them. The students with the first teacher were less engaged in the activity, and tended to argue their assigned point of view without truly responding to the other side’s arguments. Such was not the case with the second teacher. With an argument intervention, Kuhn and Udell demonstrated that students at a low-performing secondary school could improve their argument skills through a course on argument with the goal of a debate.58 51 Parker, Walter C., “The Art of Deliberation,” Educational Leadership 54 (1997): 18-21. 52 Berland, Leema Kuhn, and Brian J. Reiser, “Making Sense of Argumentation and Explanation,” Science Education 93 (2009): 26-33; Kuhn, Deanna, Education for Thinking (Cambridge, MA: Harvard University Press, 2005). 53 Clinchy, Blythe M., “Revisiting Women’s’ Ways of Knowing,” in Personal Epistemology: The Psychology of Beliefs about Knowledge and Knowing, eds. Barbara K. Hofer and Paul R. Pintrich (Mahwah, NJ: Erlbaum, 2002), 63-88. 54 Anderson, Amanda, The Way We Argue Now: A Study in the Cultures of Theory (Princeton: Princeton University Press, 2005); Graff, Gerald, Clueless in Academe: How Schooling Obscures the Life of the Mind (New Haven: Yale University Press, 2004); Kuhn and Weinstock, 2002. 55 Kakutani, Michiko, “Critic's Notebook; Debate? Dissent? Discussion? Oh, Don't Go There!” New York Times, March 23, 2002, Books section, National edition. 56 Anderson, 2005; Kuhn, 2005; Kuhn and Weinstock, 2002. 57 Kuhn, 2005. 58 Kuhn and Udell, 2003. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 74 Felton, Garcia-Mila, and Gilabert59 demonstrated that simply manipulating the goal of argument influenced the quality of students’ argument. Those given the goal of the argument about a social issue as one of deliberation learned more content information and had higher quality argument than those given the goal of the argument as one of disputation or those given the content materials but who did not engage in discussion with another person. In sum, by providing some regular support in the practice of argument, knowledge of argument skills, and understanding of goals of argument can improve argument quality and prompts students to engage in deliberation. It is assumed, but should be tested in future research, that either engagement in such activities will promote the development of epistemic understandings of the standards of knowledge justification and the recognition of multiple perspectives, or that those at particular epistemological levels are more likely to benefit from such activities. These examples provide a glimpse of what types of activities could take place to promote the development of deliberation skill and epistemic understanding. However, as argued in the introduction to the paper, such examples are unfortunately few and far between. Even those articles that point to examples of activities to promote effective citizenship are pointing to relatively singular examples60 or the ideal rather than the common reality.61 Interestingly, most of the promotion of the development of argumentation has been taking place in science classrooms, but as a means to improve scientific reasoning. With regard to argumentation as a means to promote epistemic development and effective citizenship there is a small but promising subfield on the inclusion of debates about socio-scientific issues in the classroom reasoning.62 If schools are to take their explicit or implicit goals of fostering the citizenship skills of their students seriously, they should build on the examples of the inclusion of argumentation in the classroom and formalize the development of argumentation skills and epistemic understanding as essential outcomes of schooling. 59 Felton, Mark, Mercé Garcia-Mila, and Sandra Gilabert, “Deliberation versus Dispute: The Impact of Argumentative Discourse Goals on Learning and Reasoning in the Science Classroom,” Informal Logic 29 (2009): 417-446. 60 For example, Glickman, 2008. 61 For example, Wade, 2001. 62 See Sadler, Troy D., “Informal Reasoning Regarding Socioscientific Issues: A Critical Review of the Literature,” Journal of Research in Science Teaching 41 (2004): 513-536; Zeidler, Dana L., and Bryan H. Nichols, “Socioscientific Issues: Theory and Practice,” Journal of Elementary Science Education 21 (2009): 49- 58; Zeidler, Dana L., Troy D. Sadler, Scott Applebaum, and Brendan E. Callahan, “Advancing Reflective Judgment through Socioscientific Issues,” Journal of Research in Science Teaching 46 (2009): 74-101. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 75 Bibliography Allen, Lew. “From Plaques to Practice: How Schools can Breathe Life into Their Guiding Beliefs.” Phi Delta Kappan 83 (2001): 289-293. Anderson, Amanda. The Way We Argue Now: A Study in the Cultures of Theory. Princeton: Princeton University Press, 2005. Baron, Jonathan. “Myside Bias in Thinking about Abortion.” Thinking and Reasoning 1 (1991): 221-235. Benhabib, Seyla. The Claims of Culture: Equality and Diversity in the Global Era. Princeton, NJ: Princeton University Press, 2002. Berland, Leema Kuhn, and Brian J. Reiser. “Making Sense of Argumentation and Explanation.” Science Education 93 (2009): 26-33. Bohman, James, and William Rehg, eds. Deliberative Democracy: Essays on Reason and Politics. Cambridge, MA: MIT Press, 1997. Clinchy, Blythe M. “Revisiting Women’s’ Ways of Knowing.” In Personal Epistemology: The Psychology of Beliefs about Knowledge and Knowing, edited by Barbara K. Hofer and Paul R. Pintrich, 63-88. Mahwah, NJ: Erlbaum, 2002. Cogan, John J., and Patricia V. Pederson. “The Development of Civic Values: Case Study of the United States of America.” International Journal of Educational Research 35 (2001): 61-76. Cohen, Joshua. “Deliberative Democracy and Democratic Legitimacy.” In The Good Polity, edited by Alan Hamlin and Philip Pettit, 17-34. Oxford: Blackwell, 1989. Copi, Irving M., and Keith Burgess-Jackson. Informal Logic. Upper Saddle River NJ: Prentice Hall, 1996. Dahl, Robert. Democracy and Its Critics. New Haven: Yale University Press, 1989. Dryzek, John. Deliberative Democracy and Beyond: Liberals, Critics, Contestations. Oxford: Oxford University Press, 2000. Felton, Mark, Mercé Garcia-Mila, and Sandra Gilabert. “Deliberation versus Dispute: The Impact of Argumentative Discourse Goals on Learning and Reasoning in the Science Classroom.” Informal Logic 29 (2009): 417-446. Felton, Mark, and Deanna Kuhn. “The Development of Argumentive Discourse Skills.” Discourse Processes 29 (2001): 135-153. Fishkin, James. The Voice of the People: Public Opinion and Democracy. New Haven: Yale University Press, 1995. Galotti, Kathleen. “Approaches to Studying Formal and Everyday Reasoning.” Psychological Bulletin 105 (1989): 331-351. Gelderman, Carol. All the Presidents' Words: The Bully Pulpit and the Creation of the Virtual Presidency. New York: Walker and Company. 1997. Glickman, Carl. “Educating for Citizenship.” The School Administrator 65 (2008): 18-23. Graff, Gerald. Clueless in Academe: How Schooling Obscures the Life of the Mind. New Haven: Yale University Press, 2004. Gutmann, Amy, and Dennis Thompson. Why Deliberative Democracy? Princeton, NJ: Princeton University Press, 2004. H. Res. 1804, 1994. Habermas, Jürgen. Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Translated by William Rehg. Cambridge MA: MIT Press, 1996. Haynes, Charles, C., and Terry Pickeral. “Renewing the Civic Mission of Schools.” The School Administrator 65 (2008): 10-15. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 76 Hofer, Barbara K., and Paul R. Pintrich. “The Development of Epistemological Theories: Beliefs about Knowledge and Knowing and Their Relation to Learning.” Review of Educational Research 67 (1997): 88-140. Kakutani, Michiko. “Critic's Notebook; Debate? Dissent? Discussion? Oh, Don't Go There!” New York Times, March 23, 2002, Books section, National edition. King, Patricia. M., and Karen S. Kitchener. Developing reflective judgment: Understanding and promoting intellectual growth and critical thinking in adolescents and adults. San Francisco: Jossey-Bass, 1994. Kitchener, Richard F. “Folk Epistemology: An Introduction.” New Ideas in Psychology 20 (2002): 89-105. Kuhn, Deanna. The Skills of Argument. New York: Cambridge University Press, 1991. Kuhn, Deanna. Education for Thinking. Cambridge, MA: Harvard University Press, 2005. Kuhn, Deanna, Richard Cheney, and Michael Weinstock. “The Development of Epistemological Understanding.” Cognitive Development 15 (2000): 309-328. Kuhn, Deanna, Mercé Garcia-Mila, Anat Zohar, and Christopher Andersen. “Strategies of Knowledge Acquisition.” Monographs of the Society for Research in Child Development 60 (1995). Kuhn, Deanna, and Seung-Ho Park. “Epistemological Understanding and the Development of Intellectual Values.” International Journal of Educational Research 43 (2005): 111- 124. Kuhn, Deanna, and Wadiya Udell. “The Development of Argument Skills.” Child Development 74 (2003): 1245-1260. Kuhn, Deanna, and Michael Weinstock. “What is Epistemological Thinking and Why Does It Matter?” In Personal Epistemology: The Psychology of Beliefs about Knowledge and Knowing, edited by Barbara K. Hofer and Paul R. Pintrich, 121-144. Mahwah, NJ: Erlbaum, 2002. Kuhn, Deanna, Michael Weinstock, and Robin Flaton. “How Well do Jurors Reason? Competence Dimensions of Individual Variation in a Juror Reasoning Task.” Psychological Science 5 (1994): 289-296. Lao, Joseph, and Deanna Kuhn. “Cognitive Engagement and Attitude Change.” Cognitive Development 17 (2002): 1203-1217. Mansbridge, Jane. Beyond Adversary Democracy. Chicago: University of Chicago Press, 1983. McPeck, John E. “Stalking Beasts, but Swatting Flies: The Teaching of Critical Thinking.” Canadian Journal of Education 9 (1984): 28-44. Means, Mary L., and James F. Voss. “Who Reasons Well? Two Studies of Informal Reasoning Among Students of Different Grade, Ability, and Knowledge Levels.” Cognition and Instruction 14 (1996): 139-178. Parker, Walter C. “The Art of Deliberation.” Educational Leadership 54 (1997): 18-21. Paul, Richard W. “Dialogical Thinking: Critical Thought Essential to the Acquisition of Rational Knowledge and Passions.” In Teaching Thinking Skills: Theory and Practice, edited by Jonathan B. Baron and Robert J. Sternberg, 127-148. New York: W. H. Freeman and Company, 1987. Pennington, Nancy, and Reid Hastie. “Evidence Evaluation in Complex Decision Making.” Journal of Personality and Social Psychology 51 (1986): 242-258. Pennington, Nancy, and Reid Hastie. “The Story Model for Juror Decision Making.” In Inside the Juror, edited by Reid Hastie, 192-221. Cambridge: Cambridge University Press, 1993. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 77 Pepper, Stephen C. World Hypotheses. Berkeley: University of California Press, 1942. Perkins, David N. “Reasoning as It Is and Could Be: An Empirical Perspective.” In Thinking Across Cultures: The Third International Conference on Thinking, edited by Donald M. Topping and Doris C. Crowell, 175-194. Hillsdale, NJ: Erlbaum, 1989. Perkins, David. N. Smart Schools: From Training Memories to Educating Minds. New York: The Free Press. 1992. Perry, William G. Forms of intellectual and ethical development in the college years. New York: Holt, 1970. Sadler, Troy D. “Informal Reasoning Regarding Socioscientific Issues: A Critical Review of the Literature.” Journal of Research in Science Teaching 41 (2004): 513-536. Safire, William. “You Lied to Us.” New York Times, June 2, 2003, Editorial Page, National edition. Schwartz, Shalom H. “A Theory of Cultural Values and Some Implications for Work.” Applied Psychology: An International Review 48 (1999): 23-47. Schwartz, Shalom H., Gila Melech, Arielle Lehmann, Steven Burgess, Mari Harris, and Vicki Owens. “Extending the Cross-Cultural Validity of the Theory of Basic Human Values with a Different Method of Measurement.” Journal of Cross-Cultural Psychology 32 (2001): 519-542. Smith, Vicki L. “Prototypes in the Courtroom: Lay Representations of Legal Concepts.” Journal of Personality and Social Psychology 61 (1991): 857-872. Smith, Vicki L. “When Prior Knowledge and Law Collide: Helping Jurors to Use the Law.” Law and Human Behavior 17 (1993): 507-536. Tabak, Iris, and Michael Weinstock. “A Sociocultural Exploration of Epistemological Beliefs.” In Knowing, Knowledge and Beliefs: Epistemological Studies Across Diverse Cultures, edited by Myint Swe Khine, 177-195. Netherlands: Springer, 2008. Tetlock, Philip E. ”Cognitive Style and Political Ideology.” Journal of Personality and Social Psychology 45 (1983): 118-126. Torney-Purta, Judith, and Britt S. Wilkenfeld. Paths to 21st Century Competencies Through Civic Education Classrooms: An Analysis of Survey Results from Ninth-Graders. (A Technical Assistance Bulletin). Chicago: American Bar Association Division for Public Education, 2009. Wade, Rahima C. “Social Action in the Social Studies: From the Ideal to the Real.” Theory Into Practice 40 (2001): 23-28. Wade, Rahima C., and Susan Everett. “Civic Participation in Third Grade Social Studies Textbooks.” Social Education 58 (1994): 308-11. Walton, Douglas N. Informal Logic: A Handbook for Critical Argumentation. Cambridge: Cambridge University Press, 1989. Warren, Joshua, Deanna Kuhn, and Michael Weinstock. “How do Jurors Argue with One Another?” Judgment and Decision Making 5 (2010): 64-71. Weinstock, Michael. “Cognitive Bases for Effective Participation in Democratic Institutions: Argument Skill and Juror Reasoning.” Theory and Research in Social Education 33 (2005): 73-103. Weinstock, Michael. “Like an Expert: Representing the Problem, Epistemic Requirements, and Competence in an Everyday Reasoning Task.” Learning and Individual Differences 19 (2009): 423-434. Weinstock, Michael. “Cultural Values and Epistemological Development.” Paper presented at the Seminar of the Developmental Forum of the Department of Psychology, University of California at Los Angeles, Los Angeles, USA, April 26, 2010. Volume 4 Number 1 (2010): 56-77 http://www.infactispax.org/journal/ 78 Weinstock, Michael. “Epistemological Development of Bedouins and Jews in Israel: Implications for Self-Authorship.” In Refining Understanding of the Development and Assessment of Self-Authorship, edited by Marcia B. Baxter Magolda, Elizabeth G. Creamer, and Peggy S. Meszaros, 117-132. Sterling VA: Stylus, 2010. Weinstock, Michael, and Matthew A. Cronin. “The Everyday Production of Knowledge: Individual Differences in Epistemological Understanding and Juror Reasoning Skill.” Applied Cognitive Psychology 17 (2003): 161-181. Weinstock, Michael, and Robin A. Flaton. “Evidence Coverage and Juror-Reasoning Skill as Mediating Factors in a Juror’s Verdict Choice.” Journal of Behavioral Decision Making 17 (2004): 191-212. Weinstock, Michael P., Yair Neuman, and Amnon Glassner. “Identification of Informal Reasoning Fallacies as a Function of Epistemological Level, Grade Level, and Cognitive Ability.” Journal of Educational Psychology 98 (2006): 327-341. Weinstock, Michael, Yair Neuman, and Iris Tabak. “Missing the Point or Missing the Norms? Epistemological Norms as Predictors of Students’ Ability to Identify Fallacious Arguments.” Contemporary Educational Psychology 29 (2004): 77-94. Zeidler, Dana L., and Bryan H. Nichols. “Socioscientific Issues: Theory and Practice.” Journal of Elementary Science Education 21 (2009): 49-58. Zeidler, Dana L., Troy D. Sadler, Scott Applebaum, and Brendan E. Callahan. “Advancing Reflective Judgment through Socioscientific Issues.” Journal of Research in Science Teaching 46 (2009): 74-101.