INFORMAL LoGIC XVLl, Winter 1994 Rationality, Judgment, and Argument Assessment PAUL HEALY Swinburne University 0/ Technology Key Words: Rationality; judgment; argumenta- tion; rule-following; know-how; expertise; con- textuality; interpretation. Abstract: In contrast to approaches to critical thinking which emphasize the importance of rules, strategies and criteria for the analysis and evaluation of arguments, this paper seeks to vindi- cate the central role which judgment plays in the assessment process. To counteract charges of ar- bitrariness or subjectivism in the exercise of judg- ment, individual and intersubjective constraints are outlined which can ensure its reliable exercise. The contextuality of argumentation, as it affects judgment, is discussed, and some conclusions are drawn about how acknowledgment of the role of judgment in argumentation can influence our conception of the reasoning process. Argument interpretation and analysis/arm an art, an art requiring insight and judgment. Trudy Govier In the teaching of critical thinking, much effort is devoted to familiarizing stu- dents with a variety of rules, strategies, and criteria for the analysis and evaluation of arguments. While the availability of such strategies and criteria are of enormous im- portance for the systematic and objective evaluation of arguments, it is often over- looked that, by themselves, these strategies are insufficient to determine the evalua- tion. As I will endeavour to show in what follows, an element of informed judgment (or more informally, acquired know-how) is an indispensible attribute of critical thinking. The need for good judgment as an attribute of effective argument analysis has received passing mention in some criti- cal thinking texts. But I believe the field could benefit from more explicit discus- sion of this topic. At any rate, in what fol- lows my aim is to raise the issue of the role of judgment in critical thinking, albeit in a preliminary and incomplete way, as a topic for debate. Thereafter. readers will be in a position to decide for themselves about the potential value of this line of analysis. I Judgment In developing the case for more explic- it recognition of the pervasive role of judg- ment in critical thinking, I want to begin by drawing attention to its intrinsic influence even in formal reasoning. Formal reasoning has often been char- acterized as a strictly rule-governed proc- ess. It embodies rules of inference (like modus ponens or modus tol/ens) which compel a conclusion with logical necessi- ty. But even in formal contexts judgment inevitably enters into the reasoning proc- ess. As noted by philosophers as diverse as Kant and Wittgenstein. among others, the application of rules inevitably involves the exercise of judgment. 2 Even if the rule in question is a simple one like modus tol- lens. we have to be able to recognize (or judge) that we can legitimately apply this particular rule at this point in the argu- ment. Moreover, it seems clear that, on pain of embarking on an infinite regress of rule-following, we cannot continually ap- peal to new rules to tell us how to apply the rules of inference. Unless we invoke, and trust, our judgment that this rule may be appropriately applied here, our thinking becomes paralysed (locked into an infinite 32 Paul Healy regress).3 Thus, even within formalized systems, effective thinking presupposes the use of judgment. This conclusion applies with still greater force in informal contexts because, notwithstanding the necessary textbook emphasis on rules and strategies for ap- praising informal arguments, the relations of ideas in this domain are inevitably more loosely textured, and the "rules" governing the reasoning process correspondingly more incomplete and contingent. These factors necessitate a greater reliance on judgment in dealing with such arguments. Moreover, on reflection, it is clear that judgment is in- trinsically involved in applying all the rules and strategies detailed in the texts-for ex- ample, in applying the guidelines for elim- inating extraneous material and spotting fallacies, for outlining the argument in its clearest and most essential form, for dia- gramming the relations between premises and conclusion, etc. Though the point is rarely explicitly raised in discussions of these rules and strategies, their application is not self-justifying. It always requires the (at least implicit) exercise of judgment. Furthermore, since the conclusion of an in- formal argument rarely follows deductive- ly from the reasons given in its support, an epistemological "gap" exists between premises and conclusion (the claim being underdetermined by the reasons given in support of it). In evaluating such argu- ments, then, we have no option but to exer- cise judgment in assessing the extent to which the reasons given provide support for the conclusion. Thus, it would seem that decision and judgment enter into all phases of the process of informal argumentation. But acknowledgment of the role of judgment in assessing informal arguments raises a problem of another sort, namely, the threat of subjectivism or even irration- ality. Under positivism, for example, the attempt was made to articulate rational de- cision-making in purely formal (algorith- mic) terms, with a view to minimizing the influence of apparently subjective elements, like judgment.4 But if, as I have argued, judgment is in fact an indispensible ele- ment in the reasoning process, its influence cannot simply be dismissed. Instead, the challenge is to determine how the objectiv- ity of reasoning and decision-making is to be preserved, given that reasoning inevita- bly involves the exercise of (individual) judgment. It is this issue that I next seek to address by reference to the concept of "good judgment." Good Judgment In this section, I propose an answer to the following question: if judgment is in- evitably involved in the reasoning process, what can be done to minimize arbitrariness or subjectiveness in its operation, and to maximize its reliability? The answer, I suggest, involves consideration of both the individual and intersubjective aspects of the reasoning process. On the individual level, reliability of judgment is enhanced by the development of expertise, as a result of training and practice, in the analysis and evaluation of arguments. At the same time, because of the fallibility of individual judgment (however expert it might be) it is necessary to provide, on the intersubjective level, for the (in principle) verifiability of individual judgments. In developing these themes, I seek to rehabilitate judgment as a reliable, as well as an indispensible, com- ponent of the reasoning process. On reflection, it seems clear that there is nothing intrinsically arbitrary or subjec- tive about the operation of judgment. In- deed, as we have seen, it is an essential element in reasoning. A more balanced view of its operation is in terms of an abili- ty which, like other abilities, we can exer- cise reliably and responsibly or unreliably and irresponsibly. If so, the important question is: what factors contribute to its reliable exercise? My primary claim here is that the effective use of judgment in argument assessment can be enhanced in much the same way as any skill can be enhanced, that is, by extended training and practice in the exercise of that skill. 5 Clearly, on the individual level, the ma- jor skill involved in argument appraisal is the ability to effectively assess how well the overall claim is supported by the evi- dence presented in its favor. This task em- braces several sub-tasks of the kind detailed in textbooks, and already alluded to above: e.g. correctly identifying premis- es and conclusion, identifying subargu- ments, standardizing and diagramming the whole, assessing the combined weight of the several pieces of evidence given in sup- port of the conclusion, etc. In this regard, the point I want to press is that the ability to effectively perform this task is an ac- quired skill, built up through (supervised) training and practice in analysing and as- sessing arguments. The development of this skill, I suggest, involves the acquisition of a facility in discerning and assessing the truly relevant pieces of information and in relating them together appropriately. The development of this skill goes well beyond the mere knowledge of rules and strategies (however valuable these might be as aids to reasoning). Beyond this, it requires ex- tended "hands-on" experience of argument analysis and appraisal-for roughly the same reasons that the development of any kind of skill (e.g. riding a bicycle, cooking a meal, fixing a car, constructing a proof in formal logic, etc.) requires extended train- ing and practice, at least if the task is to be executed in an effective and efficient way. Moreover, provided we are prepared to learn from our mistakes, practice builds on itself in the sense that by repeated practice, we can develop a skill and acquire a level of expertise which facilitates even more ef- fective performance in the future. Thus, my main point here is that repeated (super- vised) practice is an indispensible compo- nent in becoming a skilled thinker. not simply because (as has frequently been supposed) with practice we become better at applying the available rules, but primarily Rationality, Judgment, and Argument :B because with practice we develop a skill at discerning the crucial features of argu- ments and relating them together, thereby enhancing the reliability of our judgment. It is noteworthy that the exercise of judgment, though not much commented on as a crucial ingredient in argument assess- ment, has been acknowledged, at least by some commentators, as a very important component in decision-making in science. Thus, Kuhn, for example, insists on the need for exposure to significant "exemplars," if students are to become effective members of a functioning scientific community. In his discussion of this topic,6 Kuhn main- tains that to acquire skill as a problem solver in science, one must acquire not just a set of cognitive rules, but also a distinc- tively scientific way of seeing and thinking about a problem situation. Because there is more involved here than a purely cognitive appraisal of the issues, these shared ways of approaching and resolving scientific problems cannot be acquired simply by learning rules and definitions of the text- book variety, but necessarily also involve a degree of "know-how" as a result of being repeatedly exposed to the relevant types of problems and the strategies used for re- solving them. Without the acquisition of this kind of skill, which is acquired through extended training and practice, and which often defies explicit articula- tion. effective scientific decision-making is not possible. (Indeed, Kuhn goes so far as to suggest that without this type of ac- quired "know-how" it is not even possible to master the vocabulary of science.) As is well known. Polanyi has also forcefully ar- gued for the central role that "tacit knowl- edge" (or acquired know-how) plays in the effective performance of both everyday and professional tasks. 7 A classical prece- dent for this approach is even to be found in Aristotle's concept of phronesis.8 The present analysis leads to the con- clusion that the time has come to explicitly acknowledge the existence of a similar "tacit dimension" in the reasoning process. 34 Paul Healy Specifically, this tacit dimension reveals it- self in the need to exercise judgment in the appraisal of argumentative claims, because the available rules and strategies are not by themselves sufficient to determine our ac- ceptance or rejection of a claim (otherwise, as Kuhn indicates in his discussion of ex- emplars, we would simply teach the rules, and thereby eliminate the need for residual know-how or judgment (as positivism un- successfully tried to do)), Moreover, the reliability of judgment can be enhanced, and its subjectivity minimized, by training and practice in the skill of argument as- sessment. Furthermore, emphasis on the need for training and practice to enhance the reliabilty of judgment makes it clear that there is nothing mysterious about the reliable exercise of good judgment. As Kuhn and Polanyi's analysis illustrates, the skillful performance of any task is enor- mously enhanced by training and practice in dealing with a relevant range of prob- lems. In his recent work, Harold Brown appropriately refers to the development of the acquired ability to exercise judgment effectively as "expertise."9 But in pressing this point, it is impor- tant to empasize that the exercise of judg- ment does not provide a self-justifying epistemic warrant for the acceptancel rejection of an argumentative claim.lO The epistemic warrant for acceptancelrejection comes from the (strength of the) evidence presented in support of the claim. Accord- ingly, reference to judgment should not de- flect attention from the need for an appropriate evidential base. The role of judgment is to facilitate our decision as to whether the appropriate warrant obtains. It does this by sensitizing us to the factors which are most relevant in evaluating the argument and by helping us decide about their appropriate weighting in the circum- stances at hand. Judgment, thus, acts on, but does not replace, the evidential base. Furthermore, however well-informed and reliable our judgment might be, we cannot overlook the possibility that it might be in error; we are, after all, fallible. Accordingly, it must be possible for us to subsequently reappraise our original as- sessment, especially if new evidence or other relevant information comes to our at- tention. The process of critical assessment is, thus, ongoing and revisable. In addition, we must allow that our (individual) judg- ment might not actually be the final court of appeal when it comes to adjudicating the acceptability of a claim. To ensure reli- ability and objectivity, we must, I think, al- low the possibility of a more broadly- based debate about the acceptability of a claim and, indeed, about our judgment about it. In the classroom situation, this means that we should actively encourage discussion and criticism of proposed as- sessments of arguments, allowing students to see that no one perspective, including the instructor's, is self-evidently correct or beyond criticism (while at the same time showing that-and why-some appraisals have more in their favor than others). The process here is analogous to that of inter- subjective verification in, say, the sciences, whereby claims are severely tested through critical appraisal by other experts. In the case of critical thinking in particular, the process of validating one's viewpoint should take place in the context of dialogi- cal argumentation, whereby the strengths and weaknesses of anyone appraisal are made explicit, and themselves made the subject of further critical debate. This (dia- logical) aspect of critical thinking is itself a matter which warrants more attention than it has so far received, but it is not an issue which can be pursued further in the present paper.ll Here the issue has been raised in the context of affirming the need to include appeal, beyond individual judg- ment, to the larger community of inquirers in order to enhance the objectivity and reli- ability of argumentative appraisal, while minimizing the chance of subjective errors and biases. In rounding off the present discussion, brief mention should also be made of the broader issue of the contextuality of the ar- gumentative process, as this has bearing on the reliable exercise of judgment. What I want to affirm (though, again, full justifi- cation would require a separate paper) is that informal reasoning (and, thus, good judgment) must be highly sensitive to con- text. There are several respects in which context demands our attention. To begin with, our analysis and evaluation must be sensitive to the particular domain of argu- mentation. Clearly, the styles and strate- gies of argument, and the degrees of rigor required will vary significantly depending whether we are arguing in an everyday, scientific, religious, or aesthetic context. Moreover, while in formal argumentation, we are entitled to test the validity of the ar- gument, assuming the truth of the premis- es, the asssumption of the acceptability of the premises 12 is not a given in the case of informal argumentation, but is rather a context-dependent variable which itself enters into the overall evaluation of the strength of the argument. Context also sig- nificantly affects our judgment as to whether there are gaps in the argument which require the addition of a supplemen- tary premise (or conclusion) and the iden- tification of what this might be. More generally, of course, context influences our assessment of how the "principle of chari- ty" is to be applied in giving a judicious in- terpretation of how the argument is to be understood. In general, appreciation of the range of context-dependent variables at work in the analysis and evaluation of arguments carries the challenging implica- tion that, even within a given domain of argumentation, the same text (argument) can judiciously support more than one interpretation, depending on which contex- tual factors are given priority. As I will elaborate on briefly below, awareness of these factors argues the need for a pluralist approach to the analysis and assessment of informal arguments, although this need has not received much explicit recognition to date. Rationality, Judgment, and Argument 35 Some Implications In the present paper, I have been con- cerned to vindicate the role of judgment in argument assessment. I have been con- cerned too to show that judgment can func- tion in a reliable and effective way, and that charges of arbitrariness and subjectiv- ism in its operation are ill-founded. I want to conclude by pointing to some new as- pects of the reasoning process which war- rant attention in the light of the foregoing analysis. To begin with, explicit discussion of the role of judgment should make us more aware than heretofore of the part played by an active agent in argument assessment. Ultimately, of course, the acceptability of a claim is a matter of the evidential relations obtaining between premises and conclu- sion. Yet, as we have seen, the assessment of these relations involves the active exer- cise of judgment. Judgment, in turn, pre- supposes an agent who exercises it. Thus, it appears that the agent has an important (but rarely acknowledged) role to play in the assessment process. Reflection on this consideration argues the need to distance ourselves more completely from rule-gov- erned systems of appraisal, taking account instead of the dynamic way in which deci- sions (informed by good judgment) must be made by an active agent at every step of the way. For example, it is the agent who must decide about the sufficiency or insuf- ficiency of evidence, the contextual factors which influence decision, the unmen- tioned, but potentially relevant, informa- tion (supressed premises, etc) which could affect the overall assessment; it is the agent who must decide whether to accept now or seek more information before reaching an appraisal; likewise, it is the agent who must decide the tenacity (degree of convic- tion: strong, moderate, weak) with which the claim, if embraced, should be held, etc. It would appear, then, that alongside the more traditional analysis of the evidential relations obtaining between premises and 36 Paul Healy conclusion, there is an emerging need for explicit consideration of how the agent functions in the evaluation process. 13 A second noteworthy implication of the present discussion is that it makes ap- parent why, if, as has been claimed, good judgment is a skill which is developed through training and practice, the "hands- on" assessment of multiple representative arguments of various types is an essential aspect of becoming a good critical thinker. The need for much practice in working representative examples has, of course, long been recognized. What the present analysis suggests, however, is that the main function of these exercises is not just to familiarize students with a range of rules and strategies, but also (and more impor- tantly) to attune their judgment to the range of relevant factors which (beyond rule-following) must be taken account of in a judicious evaluation. A primary task then becomes that of boosting students' confidence in the exercise of judgment (as distinct from attempting to replace it with learnt rule-governed strategies), with the provision, of course, that they be able to ar- gumentatively justify their judgments, if called on to do so. Finally, when it is recognized that judgment is intrinsic to the process of argument assessment, and that, to be effective, judgment must be sensitive to the range of context-dependent variables at work in the situation, it becomes clear that more than one interpretation and evaluation of a given argument may well be justified. Often, depending on the relative priority given to the range of con- textual factors, multiple interpretations will be justified. If this conclusion is cor- rect, it goes against a tenet of received wis- dom which originated in a more formalist approach: the view, that is, that a given argument supports a unique interpretation and evaluation. As against this viewpoint, the line of argument pursued here suggests that multiple interpretations (though with- in a circumscribed range) will oft~n be justified. This conclusion fits well with the classroom experience of animated de- bate, including disagreement, about the tenability of different interpretations and evaluations of an argument. As against the more formalist approach, the present anal- ysis suggests that this pluralism in inter- pretation and evaluation is an intrinsic feature of critical thinking (because of the inevitably contextual and judgmental fea- tures involved), and that it should be en- couraged, rather than supressed in the interests of univocal analysis. In pressing this point, I do not, of course, want to sug- gest that all possible solutions to a given problem are equivalent in merit. but only that several solutions (and not just a single solution) to a given problem may demand our serious consideration. But, then, to make this point is simply to confirm the in- trinsically democratic (and nonauthoritari- an) character of critical thinking, properly so called. Notes I An early version of this paper was presented at the 1991 Reasoning Conference, Flinders Uni- versity, South Australia. A revised version, submitted to Informal Logic, was further sharpened in response to criticisms by Ralph Johnson and Harvey Siegel. My thanks are due to these critics. 2 On this point, see Trudy Govier, Problems in Argument Analysis and Evaluation (Dordrecht: Forls Publications, 1987), Ch. 10, especially pp.223-26. 3 In the work just mentioned, Govier (pp. 223- 26) makes this point nicely with reference to the stubborn Tortoise's dilemma in Lewis Car· roll's "What Achilles Said to the Tortoise," and with reference to Godet's demonstration of the necessary incompleteness of formal systems. 4 See Harold Brown's Rationality (New York: Routledge, 1988) for criticisms of purely algo- rithmic decision procedures, and an attempt to rehabilitate judgment as an integral part of ra- tional decision-making. Though I differ from Brown on a number of specific issues, my thoughts on the role of judgment in critical thinking-an issue not explicitly addressed by Brown-have been significantly influenced by his overall thesis. 5 My thinking about expert judgment as a skill acquired through extended training and prac- tice owes much to Harold Brown's analysis (Rationality, Ch. 4). 6 See The Structure of Scientific Revolutions, 2nd ed. (Chicago: University of Chicago Press, 1970), especially Postscript, Secs. 3, 4; also Ch. 5. 7 See Michael Polanyi, Personal Knowledge (New York: Harper and Row, 1958). It should be noted that in alluding to the work of Kuhn and Polanyi in this context, I am not taking a position on their overall approach to epistemology and the philosophy of sci- ence. Given the large and contentious litera- ture surrounding Kuhn's analysis in particular, position-taking on his overall anaysis would require a separate study. Here I am referring to specific aspects of their work in support of my claim that good thinking requires more than rule-following; in particular, it requires, in ad- dition, good judgment. 8 See Aristotle's Ethics, trans. T. Irwin (Indiana- polis: Hackett, [985), 1140a25-b29. Like phronesis, good jUdgment, as we have seen, cannot adequately be explained in terms of ru[e-following; it is acquired through training and practice; and the development of a given level of skill paves the way for still more skilled performance in the future. And just as practical wisdom becomes second nature to the [Jhronimos, good judgment becomes second nature to the accomplished reasoner. Rationality, Judgment, and Argument 37 9 See, e.g., Rationality, Ch. 4. L See too Ch. 4.3 for a naturalistic explanation of the operation of judgment. to In my view, Brown's analysis of rationality and judgment, while provocative, may be criti- cized for failing to emphasize this point. II For some preliminary reflections on this topic, see my paper, "Critica[ Reasoning and Dialec- tical Argument," Informal Logic 9 (1987), pp. 1-12. C[early, not just dialogical argumentation but also the whole process of intersubjective verification in epistemology and science are topics which require much further analysis. However, the problems attending these proc- esses cannot be tackled here. Nonetheless, I do int~nd to indicate that, despite problems re- quiring attention, I see intersubjective verifica- tion (better verifiability) as providing the most promising check on the reliability of individu- al judgment. 12 I believe that in informal logic the question of the acceptability of premises is a more impor- tant consideration than their truth as such (cf. Trudy Govier, A Practical Study of Argument, 2nd ed. (Belmont, CA: Wadsworth, 1988), Appendix B, and 3rd ed. (Wadsworth, 1992), pp. 67-69, and Ch. 5). It should be noted, how- ever, that in opting for acceptability in this context, I in no way intend to impugn the validity or worth of truth as an epistemological concept. On this issue, see further my paper, "Truth, Justification, and Nonfoundational- ism," Philosophical Studies (lrl) 33 (1992), pp.55-74. 13 Some preliminary reflections on this topic are to be found in Brown's Rationality, especially Ch. 5.2, and in William Berkson, "Skeptical Rationalism," in J. Agassi and I.C. Jarvie (eds.), Rationality: The Critical View (Dor- drecht: Nijhoff, [987), pp. 21-43. References Aristotle. Nichomachean Ethics, T. Irwin (trans.), [ndianapolis: Hackett, 1985. Berkson, W. "Skeptical Rationalism," in J. Agassi and I.C. Jarvie (eds.), Rationality: The Critical View, Dordrecht: Nijhoff, 1987, pp. 21-43. Brown, H. Rationality, New York: Routledge, 1988. Govier, T. A Practical Study of Argument, 3rd ed., Belmont: Wadsworth, 1992. Govier, T. A Practical Study of Argument, 2nd ed., Belmont: Wadsworth, 1988. Govier, T. Problems in Argument Analysis and Evaluation, Dordrecht: Foris Publications, 1987. 38 Paul Healy Healy, P. "Truth, Justification, and Nonfounda- tionalism," Philosophical Studies (Irl) 33 (1992), pp. 55-74. Healy, P. "Critical Reasoning and Dialectic Argument," Informal Logic 9 (1987), pp. 1-12. Kuhn, T.S. The Structure of Scientific Revolu- tions, 2nd ed., Chicago: University of Chicago Press, 1970. Polanyi, M. Personal Knowledge, New York: Harper & Row, 1958. PAUL HEALY DEPARTMENT OF PHILOSOPHY SWINBURNE UNIVERSITY OF TECHNOLOGY P.O. BOX 218 HAWTHORN, VICTORIA 3122 AUSTRALIA 0