INFORMAL LOGIC XIY.2&3, Spring & Fall 1992 Siegel on Competency Testing and Critical Thinking THEODORE A. GRACYK Moorhead State University Key Words: Critical thinking; competency test- ing; education. Abstract: Harvey Siegel argues that minimum competency testing (MCT) is incompatible with strong sense critical thinking. His arguments are reviewed and contrasted with positions held by John E. McPeck and Michael Scriven. Siegel's arguments seem directed against the prevailing form of MCT. However, alternative formats which allow for the aggregate and context-sensitive nature of critical thinking are not doomed to the arbitrariness Siegel finds. MCT may be a legiti- mate and useful means for furthering critical thinking as one of our educational ideals. As higher education increasingly embraces critical thinking as essential to the educational enterprise, we may find a corresponding call for comprehensive test- ing in this area. Such testing is already tak- ing place for general literacy skills, particularly of those planning to become educators. These tests frequently take the form of minimum competency testing (henceforth MCT), and there is some rea- son to believe that such testing will increase at all levels of education, and for a widening range of skills which educators claim to impart to their students. A great deal of debate has already occurred on the topic of MCT, and sharp lines have been drawn between proponents and detractors. 1 If the movement towards MCT accelerates, and if respect for critical thinking likewise gains adherents, we can expect increasing controversy concerning MCT for critical thinking skills. Those who claim to impart such skills can expect challenges (in the name of "accountabil- ity") to substantiate the claim that critical thinking courses or programs really pro- duce critical thinkers. Against this back- ground, Harvey Siegel argues that such testing is neither desirable nor pedagogi- cally useful for fostering critical thinking within the educational system. 2 While aware of general objections to the use of MCT for other skills, Siegel's primary focus is MCT's effect on critical thinking: "Does MCT help or hinder our efforts to inculcate the skills and attitudes of the critical thinker?"3 Siegel contends that MCT is a "foe" to critical thinking. Moreover, he thinks that MCT conflicts with critical thinking whether it tests criti- cal thinking. functional literacy, or any- thing else! More precisely, he holds that MCT is an indefensible and irrational edu- cational practice when assessed in terms of "our best-defended educational ideals," particularly those ideals which lead to the teaching of critical thinking. Complaining that educational policies too seldom reflect philosophical concerns about the aims of education, Siegel thinks that if these are taken into account, MCT is seen to be undesirable. However, his conclusion about MCT appears hasty, for one can endorse critical thinking as an indispensa- ble aim of education and yet regard MCT (including MCT of critical thinking) as both rational and pedagogically useful. However. to do so we must reject the assumption that MCT must always employ standardized multiple-choice exams. At the outset, nearly everyone grants that MCT can be abused and misused, and that it is no panacea for our educational ills. Yet it is hardly the enemy of critical thinking that Siegel portrays. Because his 166 Theodore A. Gracyk arguments focus on possible effects that MCT may have on critical thinking, and because the arguments derive from consid- eration of philosophically informed educa- tional ideals rather than examination of specific instantiations of MCT programs, my counterargument to Siegel is similarly focused on our goal of fostering critical thinking (particularly at colleges and uni- versities). Consequently, my argument downplays the issue of what a specific MCT program would include and exclude in order to be acceptable, but of course any conclusions reached about the compatibil- ity of MCT with our educational ideals will have consequences which may guide the design of specific instantiations. I If Siegel's position is puzzling, it is because his general characterization of critical thinking seems acceptable, albeit broadly conceived. He describes a critical thinker as "one who is appropriately moved by reasons. "4 The ideal critical thinker possesses a range of general skills, transferable across a range of disciplines and subect-matters, coupled with a habitu- ated "critical spirit" or "critical attitude. "'i We should, according to Siegel, regard critical thinking "as an educational ideal .. best thought of as a regulative ideal ... which can be used to adjudicate between rival educational methods, policies, and practices. "6 So the issue is twofold. First, is there an identifiable set of general skills which are necessary for thinking critically? Sec- ond, would minimum competency testing, whether for those specific skills or for any others, help students to progress towards our ideal of critical thinking? Siegel answers the first question affirmatively, the second negatively. In contrast, John E. McPeck answers the first negatively, and the second affirmatively.7 This divergence suggests that any connection between the two issues is hardly straightforward. Of course, there's a third way to answer the second question, namely, by saying that MCT is neutral for critical thinking. In that case, other educational goals would be par- ticularly relevant to the debate, as will become clearer below, in Part II. McPeck is well known for contending that "there is no generalized skill properly called critical thinking. "8 He has also argued that, if there is a case for such a skill or specific set of skills, the burden of proof rests upon those who advocate teach- ing such skills, but that there are very seri- ous difficulties with demonstrating the presence of such skills. 9 McPeck's chal- lenges to the practice of teaching critical thinking courses, particularly as informal logic courses, have been discussed and criticized so frequently that there is no need to look at them in depth in the present context. 1O But two points are germane to the issue at hand. McPeck emphasizes that, at best, informal logic skills are necessary for critical thinking, but are too often treated in critical thinking courses as if they were sufficient. Secondly, and more notoriously, McPeck insists that logic (for- mal and informal) is seldom relevant to the critical evaluation of arguments in practi- cal life, where the criteria of assessment are subject-specific. Like Siegel, I think that McPeck is "partly right" in these criticisms of how critical thinking is generally approached in schools. I I While McPeck is correct to emphasize the degree to which critical thought is context-dependent, he too read- ily slides from the idea that good critical thinking requires knowledge and informa- tion in order to determine an argument's strength, to the fact that many arguments draw upon highly specialized knowledge (and so can only be assessed by special- ists), and then to the very questionable view that logic courses can teach nothing very useful about argument assessment. Based on the first idea, that critical think- ing is context-dependent, McPeck rejects standardized multiple-choice tests of criti- cal thinking and defends the use of essays which are "subject-specific in an area (or areas) of the test taker's experience or preparation. "12 Siegel fastens on the most important point here: McPeek's weaker claim that specialized information is sometimes required for rea- son assessment is sustained, but the stronger claim that it is always required is not. ... McPeck does well to remind us of the importance of specialized information for critical thinking. But this in no way establishes the non-importance of non- specialized, non-technical, general infor- mation, or the inappropriateness of general criticallhinking courses. I) In brief, Siegel reminds us that there is a body of "general information" which education seeks to impart to everyone. Many arguments can be assessed ade- quately by reference to this general infor- mation, particularly to determine that they are unsound. Given Siegel's response to McPeck, educators should be concerned with critical thinking competence relative to "non-specialized, non-technical" con- texts. And so when Siegel subsequently argues that MCT is a foe of critical think- ing, we need only address its effect on crit- ical thinking in non-specialized situations. Basic understanding of some sort on the part of our audience must be assumed to obtain every time we communicate. When communicating with a more special- ized audience, one presupposes a higher proportion of subject-specific information on the part of that audience (I assume that readers of Informal Logic will understand my upcoming reference to the fallacy of division without need of any further expla- nation). Although a common core of general knowledge must be assumed of the whole audience, not every member of the audi- ence will possess all of that knowledge. Siege) and McPeck both write as if every educated person possesses the very same Siegel on Competency Testing 167 basic vocabulary and knowledge, and as if gaps only disrupt thinking in more special- ized contexts. To communicate, we must act as if our whole audience understands all that we say. But we commit the fallacy of division if we think that each member of the audience possesses all of the basic information that we presuppose of the aggregate. Each person possesses a patch- work of non-specialized knowledge. There is very little empirical information that is so subject neutral that it can be presumed known by every educated member of our society. In our increasingly polyglot and multicultural society, such obstacles to thinking multiply. Anyone who teaches is familiar with cases where a perfectly clear example bogs down when an intelligent student betrays ignorance of what seems to be an obvious, trivial point. In a recent group of forty, one student insisted that pistons are a neces- sary condition for the functioning of an automobile. Confronted with the example of solar powered cars, he retreated to the position that pistons are necessary for internal combustion engines, only to be shocked by the claim that there are rotary engines. Now, I personally know very little about automotive matters, yet I knew about rotary engines, whereas the student, who may have been an amateur mechanic, did not. Our vocabulary is no less patchwork. I recall hearing a lecture in my introductory political science course at USC; as I took notes on ambiguities in the Constitution, dozens of puzzled students around me in the lecture hall asked each other what "ambiguity" meant. In other words, because general knowl- edge is an aggregate of disparate informa- tion, it has a hit-and-miss quality that frequently disrupts critical thinking. Func- tioning individuals possess an extremely broad but shallow and unpredictable range of information on numerous subjects, com- bined with a deeper grasp in a limited number of specialized areas. And just as no adult possesses all general knowledge, 168 Theodore A. Gracyk no two possess the same range of non- technical information. An unspecifiable mass of it is quite sufficient to function. Much of this information is acquired hap- hazardly, and not through formal school- ing. I find it important to know that trash is picked up on Wednesday in my neighbor- hood, and useful to know to ask directions at gas stations when lost. I learned neither of these things in school. Formal education imparts a broad range of information, but it cannot be the goal of education that every- one acquire all of the same information, nor all of the information that will be needed to function in future life. Besides information, education imparts skills, and what has just been said about information applies to skills, as well. Siegel holds that various general skills are useful for assessing reasoning in any sub- ject area, however specialized. Logic and other critical thinking skills can be taught in school and are transferable throughout life. McPeck emphasizes that such skills are not sufficient for argument assessment and systematically fail us when we lack specialized knowledge. Both positions are correct, but McPeck is mistaken to con- clude that critical thinking courses are rel- atively worthless. Requiring such courses is compatible with the fact that these skills will fail everyone some of the time, even when dealing with relatively "non- specialized" subjects. (Straw man and equivocation are particularly dependent on background knowledge, and some weak- ness in spotting them is consistent with thinking critically.) I have stressed the aggregate nature of everyone's background knowledge because it supports the impor- tance of identifying and inculcating gen- eral, transferable skills. The further point here is that, just as general knowledge is a disparate aggre- gate, so are critical thinking skills. Some skills are directly related, involving a sequence where one is a prerequisite of the next, but this is not always the case. The ability to determine validity is a prerequi- site to determining that a specific syllo- gism is sound, and the ability to locate and distinguish between premises and conclu- sion is a prerequisite to determining validi- ty. In contrast, consider the skills involved in spotting two different fallacies, such as straw man and post hoc. No practical or theoretical link unites them; either of them can be understood and utilized in isolation from the other. Neither is essentially con- nected to formal logic. What all of these skills have in common is that each (the abilities to spot straw man, post hoc, and the validity of a syllogism) is sometimes useful in thinking critically. In the absence of a strong theory demonstrating other- wise, it seems that the full range of critical thinking skills is an aggregate of valuable but disparate skills, none of which are indi- vidually essential to thinking critically. Furthermore, different skills will be of greater and lesser value to different indi- viduals, just as the general information transmitted in our educational system is of greater and lesser value to different indi- viduals. Some of this information is retained and some isn't, yet we don't con- clude that this "general" information is worthless or should not be taught. Like- wise, while not everyone benefits identi- cally from the teaching of general skills, this fact hardly counts against a general course in critical thinking. Successful critical thinking involves an unpredictable combination of transferable skills with available background knowl- edge, and success may be blocked by either a gap in knowledge or a lack of some general skill. Critical thinking courses pro- vide the skills, but these skills will be of greater and lesser value to different stu- dents. Before exploring the ramifications of these admissions for MCT, one further aspect of critical thinking courses should be remembered. Siegel defends "strong sense" critical thinking. 14 Skills are of value only if the individual understands their value. A critical thinking course that does not motivate critical thinking is an empty addition to the curriculum. As Siegel puts it, a critical thinker needs "cer- tain attitudes, dispositions, habits of mind, and character traits" amounting to "a well- developed disposition to engage in reason assessment." Our goal is to transform stu- dents into persons who value good reasoning. IS In short, successful· critical thinking courses must convey both func- tional skills and a habituated disposition to employ these skills in every applicable sit- uation. I agree with Siegel, and yet this commitment may count towards MCT and not against it. Furthermore, recognition of the aggregate quality of our knowledge and skills will suggest what sorts of MCT are compatible with critical thinking. II Having outlined Siegel's broad charac- terization of critical thinking, how is it that he opposes MCT as a foe of critical think- ing? Supposedly, it is because the goal of producing critical thinkers embodies our commitment to a genuinely democratic society and our respect for individual learners, and thus our commitment to pre- pare them for "self-sufficient" adulthood.16 MCT, whether for critical thinking itself or for any other subject, is "incompatible" with the "best-defended" and "well- established" educational ideals, "especially" critical thinking as the regulative ideal of education. Thus, "MCT's failings are directly attributable to its inadequacy from the point of view of such ideals."I? MCT is "colossally inadequate and indefensible" because it measures for a narrow range of skills and does not foster "the autonomy to control one's life and life-decisions."18 Siegel argues that MCT is compatible with critical thinking only if "it could be shown that such testing was useful in fos- tering critical thinking."19 And it is not useful in this way, so it is "indefensible." Granted, most current MCT programs do not foster critical thinking. But Siegel's Siegel on Competency Testing 169 argument is inconclusive, since failure to foster critical thinking does not demon- strate any conflict with our goals. And Siegel fails to show any unresolvable con- flicts. IfMCT is neutral with respect to Ollr goals, it may be compatible with respect for learners and democratic life. Suppose we employ MCT for func- tional literacy. Even if the test does not actively foster critical thinking, it might foster the basic literacy which is a neces- sary condition of thinking critically in modern society. Siegel, like other critics of MCT, assumes that any emphasis on test- ing for minimum necessary conditions will result in classroom emphasis on these skills, with less attention paid "to those aspects of a student's education that are crucial to the achievement of critical thinking. "20 But basic literacy is "crucial" to critical thinking, so crucial that it is a sine qua non! How is this incompatible with critical thinking? Siegel's case that MCT hinders critical thinking is sketchy, and so I will try to develop it from his suggestions. Primarily, he speculates that MCT "may be counter- productive to the effort to develop the dis- positions constitutive of the critical thinker." Testing "may frustrate" our attempts to develop these dispositions.21 But Siegel is extremely vague as to why this will occur. Is it because standardized tests focus on discrete items of knowledge and independent skills, rather than disposi- tions and a synthesis of skills? Are savvy students thus encouraged (by their teach- ers) to put their efforts into mastering only what the tests emphasize? In other words, the tests encourage students to employ dis- crete skills (perhaps even the skills of criti- cal thinking) but do not reward them for becoming critical thinkers.22 There is no incentive to adopt new attitudes. The tests thereby impede our efforts to develop the critical spirit, that is, to develop certain sorts of persons. This objection captures the obvious truth that too many MCT programs are 170 Theodore A. Gracyk testing for too little. Part of the problem is that the standard format of the tests is bet- ter suited to measuring skills than disposi- tions. Educators favor them because they are easy to use and have an air of objectivi- ty. We have two options. On the one hand, we can abandon MCT programs (Siegel's proposal). But then we remove an incen- tive for students to acquire even "weak sense" critical thinking, insofar as the skills are demanded by the tests. And what practical incentive do teachers then have for emphasizing critical thinking of any sort, weak or strong? All the same, Siegel's argument does not show that MCT programs are inher- ently incompatible with our goal of pro- ducing critical thinkers. Consider an analogy: I give a midterm in my informal logic course, but the midterm does not measure everything I want to determine about students' progress. Students know this, and thus spend their time studying the materials that will be covered by the test. Does the midterm thus frustrate develop- ment of well-rounded critical thinkers, and should I therefore abandon the test? Per- haps if this were the sole measure of stu- dents' progress. Even then, it would only impede our goal if there were no external incentives for achievement of the goaL Thus, our second option is to include measurement relevant to our further goals at some point in an MCT program. (A good MCT program might culminate with some measure of strong sense critical thinking, using a format other than stand- ardized, multiple-choice testing.) Better yet, MCT programs should connect critical thinking to external incentives to develop the critical spirit. Siegel's argument shows that our current testing does not go far enough, not that MCT programs must, by their very nature, frustrate our goal of pro- ducing critical thinkers. A related problem is that a student's failure on a competency test is typically connected to a direct and serious conse- quence, namely, "either retesting, remedia- tion efforts, failure to be promoted to the next grade, failure to receive a diploma, or some combination of these is the result. "23 Another perceived roadblock towards graduation and the prospect of further remedial work will no doubt tum off stu- dents who do not understand our overrid- ing goals in educating them. And there is the associated political difficulty that many parents seem unwilling to accept MCT programs that reveal the failure of large numbers of their children to achieve mini- mum competency, leading to pressure to lower the standards for MCT programs. Why don't students and their parents appreciate our goals? I have suggested that testing will not frustrate development of the dispositions unless there is little or no external incentive to become critical think- ers. Given Siegel's concern with disposi- tions, it is worthwhile to wonder why professional educators must inculcate "dispositions constitutive of the critical thinker." Presumably, because it is not clear to many people that such skills are worth acquiring. Consider another skill: driving. Driver's education typically occurs during high school, yet successful completion of the course does not insure a driver's license. The state demands inde- pendent MCT before issuing a license. But MCT does not frustrate students here, because they are already disposed to put their skills into practice as soon as possi- ble. It appears that the critical thinking movement has not yet made it clear to everyone else why it is worthwhile to become a critical thinker. To summarize this stage of the argu- ment, it is unlikely that MCT itself deters anyone who is independently motivated to become a critical thinker. It is more likely that MCT frustrates our efforts when the skills and the testing are regarded as irrele- vant by students and prospective employ- ers. The incentive to drive is external to the education process, so MCT does not inter- fere with learning to drive. There is also an external incentive to achieve literacy. Yet educators have a limited success in making everyone literate, so we might expect an even lower rate of success for critical thinking. And there is even less disposition to use such skills, because there is little perceived demand for them outside academia. 24 Siegel contends that critical thinking skills are desirable for every citi- zen in a democracy, but these skills would seem considerably more attractive if they were explicitly demanded by prospective employers. This might be accomplished by teaching and testing critical thinking in terms of materials acquired from employ- ers across a wide range of occupations, and by presenting evidence that critical think- ers are rewarded in the job market. It seems to me that children arrive into the hands of educators with a desire to think critically, and that many of our edu- cational practices work against them. Surely, MeT is not the only villain here; rote learning, crowded classrooms, limited student-teacher interaction, and teachers' demands for order and "good behavior" probably share much of the blame. (Siegel's own criticisms of the tendency to indoctrinate rather than educate support the idea that our overall approach is the major problem.) To the extent that it might be a villain by concentrating on skills but not dispositions, its format is probably to blame, and I discuss this further in section III. However, if we do not devise and employ better tests of critical thinking, we are merely speculating here. We need an adequate comparison of different educa- tional environments, to see which stimu- late critical thinking and which do not. So far, MeT itself seems more neutral than negative for critical thinking. Siegel's other serious challenge is that any MeT that promotes critical thinking "would be unrecognizable as an MeT program. "25 He does not explain why. But he once again seems to be directing us to the typical design of MeT programs: standardized, multiple-choice tests. 26 Such tests do not allow us to observe the stu- Siegel on Competency Testing 171 dent's reasoning process and cannot reveal why a particular problem was answered correctly or incorrectly. More seriously, they are poor measures of ability to inte- grate distinct skills, which is essential to critical thinking. Finally, they seem inca- pable of measuring critical thinking in the strong sense, because their emphasis on testing isolated skills does not reveal whether the individual has the appropriate critical attitudes which will carryover into future situations. It is clear why these tests do not foster critical thinking. Students resent the rigidity of the format, the tightly structured testing process, and particularly their inability to ask questions, to demon- strate partial understanding, and to present alternative interpretations. 27 So prevailing tests are poor measures of "dispositions constitutive of the critical thinker," and their design does not encour- age development of such skills. But the fact that an undesirable format currently prevails does not justify Siegel's sweeping conclusions about MeT. To make MeT non-arbitrary and more consistent with our goal of producing critical thinkers, we should look beyond standardized, multiple- choice tests.28 (It does not follow that we must avoid them entirely.) And if Siegel thinks that practical necessity demands MeT with the standardized for- mat (suggested by his remark that MeT which "fosters" critical thinking "would be unrecognizable as an MeT program"), he is simply mistaken. 29 Such measures often include interactive evaluation and comple- tion of appropriate projects. By introducing assessment measures besides standardized pencil and paper tests, MeT programs could at least neutralize the purported tendency to squelch the critical spirit. Format aside, MeT seems neutral with respect to critical thinking. Other consider- ations favor the use of MeT, and testing of critical thinking in secondary and higher education might have the indirect effect of fostering critical thinking programs. Higher education still retains some 172 Theodore A. Gracyk semblance of a structured, sequential pro- gram, with graduation at the end of the line. Of two students who enter college together, it is possible for both to complete the same number of courses in four years, yet only one of the two will graduate. If one of them takes a random collection of courses and does not complete general edu- cation requirements and a major, graduation is denied. I state this obvious fact to empha- size that graduation is supposed to supply official certification that specific knowledge and skills have been acquired. MCT has gained support outside academia due to a growing sense that professional educators are so eager to graduate students that the diploma certifies little more than atten- dance. A variety of factors have created the environment that invites MCT as an objec- tive check on the quality of graduates. If a diploma could be taken seriously as certifi- cation of literacy, critical thinking abilities, and so on, MCT would not be mandated. Siegel defends critical thinking as one of our most basic educational ideals. It embodies our respect for students as per- sons, and "to treat students with respect is, moreover, to be honest with them."30 He stresses that honesty in the classroom pre- cludes "indoctrination." In order to be hon- est, teachers must invite critical thinking about the very process of teaching and learning; students must be able to demand justification at any point in the educational process. So students must be given the skills and dispositions for thinking criti- cally about their own education. In this spirit, Siegel's demand for hon- esty in education suggests a duty to pro- vide an evaluation of student progress. If one of our educational aims is to produce critical thinkers, we are dishonest unless we demonstrate our ability to produce crit- ical thinkers, and unless we can inform each individual about the degree of his or her own strength or weakness as a critical thinker. The same is true for literacy, math- ematical reasoning, and other basic skills. MCT is supported by parents and legisla- tors because they do not believe that public education is providing an honest evalua- tion of its successes and failures. Siegel's discussion of honesty empha- sizes respect for students as persons. Fair enough, but what about respect for, and accountability to. others involved in our collective educational enterprise? To ignore their input and to avoid any test of competency on the grounds that it will "frustrate" learning is itself a disrespectful form of paternalism. One of the strongest reasons to support MCT is the public stand that it requires, forcing us to make a case for our goals to parents, legislators, and future employers, and not merely to our students. Such tests involve a public com- mitment to specific educational goals and forces us to determine and publicize what level of competence is acceptable and what is not. MCT, whether of critical thinking or of other areas, commits us to identifiable goals and invites critical scrutiny (by par- ents, employers, and students) of our choice of educational goals. So MCT contributes to honesty in education. Furthermore, if we adopt Siegel's chal- lenge to treat students as individuals capa- ble of self-determination (particularly at the university level), many will choose not to develop as critical thinkers. And we can- not force them to learn what they do not want to learn, even by requiring specific courses for graduation. But if we regard critical thinking as central to education, it is imperative that we identify those who do become critical thinkers. And since none of us really thinks that mandatory critical thinking courses will transform every stu- dent into a capable critical thinker, MCT can serve a legitimate function in identifying and certifying those who acquire the skills. Without supposing that MCT is the only way to certify the achievement of graduates. MCT also focuses debate on the issue of which skills, knowledge, and atti- tudes are most valued and most worthy of certification. The National Council for the Social Studies therefore supports criterion- referenced competency testing in social studies, with particular emphasis on "reflective thinking. "31 The California State University system requires critical thinking as a component of the graduation require- ments; if the diploma is to mean "this stu- dent can think critically," we must be clear about what has been cerified. Because MCT requires an operational definition of whatever is tested, it impels educators to clarify and specify the goals which are to shape the instructional curriculum. "Fos- tering critical thinking" is hardly a precise educational goal, and coupling it with MCT demands a sharper empirical identi- fication of this goal. At the same time, it may serve as an empirical test of our abil- ity to teach critical thinking. 32 Finally, MCT of critical thinking may indirectly promote critical thinking by making it more desirable in the non- academic community. By establishing an empirical definition of critical thinking and providing certification of those who actually achieve it, critical thinking may become an identifiable skill demanded by employers. Schools that certify the attain- ment of critical thinking may thereby pro- duce students with brighter job prospects, and thus greater "self-sufficiency" in deter- mining their futures. In other words, MCT of critical thinking may work towards cre- ating a demand for critical thinkers, thereby giving students an external incen- tive for becoming critical thinkers. Given that literacy is a minimum necessary con- dition for critical thinking, the number of functional illiterates gives reason to sup- pose that formal education will be even less successful at achieving critical think- ing than at achieving literacy. Certifiable critical thinking may turn out to be a valua- ble commodity in the job market. III So far, I have argued that MCT is not "indefensible" in relation to critical Siegel on Competency Testing 173 thinking. MCT may be made neutral with respect to critical thinking, and MCT of critical thinking might indirectly foster our educational goals. But I have not fully addressed Siegel's emphasis on promoting the desired character traits and critical spir- it. In light of this goal, he offers us the fol- lowing dilemma: either MCT is arbitrary with respect to our educational ideals. or it "avoids the charge of arbitrariness only by exposing its inadequacy from the point of view of well-established educational ide- als," particularly strong sense criticd thinking.33 Siegel notes that MCT may be arbi- trary in either of two ways. "First level arbitrariness" is the problem that, for whichever skills are tested, "setting mas- tery levels is arbitrary."34 As Mich: el Scriven puts it, the problem is that we tue an imprecise standard and then "draw a sharp line where in fact there is a gray area."35 Scriven rectifies this by proposing that, on the one hand, "no one who was a clear winner or loser in terms of the old [imprecise] concept will be miscategorized by the new one. "36 On the other hand, we can mitigate arbitrariness by using multi- ple approaches over a range of items drawn from "the real world," particularly items which employers expect graduates to handle. In other words, the expectations of the non-academic world should determine the level for which we'll test. Siegel grants that skill levels can always be identified for specific jobs, such as "typist or mechanic," but the skills thus identified are too narrow. Scriven's pro- posal is thus subject to "second level" arbi- trariness, the first horn of Siegel's dilemma for MCT. Supposing that first level arbi- trariness is less serious than it seems, the second level problem is to identify specific skills which are "basic" and which make one "functional" in society. But in identify- ing specific skills. we settle on such a min- imallevel that we do not address our goals with respect to the attitudes of critical thinkers. We thus arrive at the other horn of 174 Theodore A. Gracyk the dilemma: our educational ideal demands students who are "minimally crit- ical," and not merely students who demon- strate competence with various minimal skills "for holding a place in the current economic order. "37 MeT doesn't address this ideal. We educate so as to enable the student to create her future, not to submit to it. Unless "education" is to mean "training and social- ization into pre-determined adult roles and jobs," we cannot specify in advance what a student's future will be and so we cannot specify in advance the needs of students which testing will serve . ... Education is not geared to any particular job perfor- mance; consequently, Scriven's needs assessment approach will not help our efforts to set non-arbitrary standards for MeT.l8 As long as we tie our testing to the demands of the job market, we abandon the educational ideals which encourage us to produce critical thinkers. It is disre- spectful to students and undercuts their self-sufficiency in shaping their own futures. So MeT is woefully inadequate. Siegel's argument is challenging, but incomplete. In responding to McPeck, Siegel emphasizes that critical thinking involves specifiable, transferable skills and attitudes which can be taught and which are of value across many fields. Is. he objecting to MeT because it fails to test for these? In that case, he has given us rea- son to push for the incorporation of broader objectives in any MeT program. However, Siegel goes on to deny that this argument supports MeT for critical thinking. 39 Is Siegel arguing that we cannot employ MeT for critical thinking, because we cannot "specify in advance" what the test should cover? He suggests this inter- pretation with his claim that "for the job of life ... needs assessment seems a hopeless task."40 But if specifiable thinking skills are applicable to the "job of life," they must be transferable. And if they can be inculcated by educators, we seem to be committed to the possibility of specifying what students need to learn, and therefore of some means of measuring it. Otherwise, Siegel's belief that we can foster critical thinking is merely an article of faith. And the same holds for the attitudes, habits of mind, and dispositions which he identifies with "strong sense" critical thinking. After all, his position is that some peo- ple are critical thinkers and some aren't, implying that we can identify who falls into each group. If so, there are observable differences between students who become critical thinkers and those who don't (oth- erwise, we couldn't know that we ever fos- ter these skills and dispositions). And what distinguishes the one group as critical thinkers is their disposition to think criti- cally whenever they reason. By observing how people respond across a range of situ- ations which involve reasoning, we can differentiate the two. So in principle such testing is not doomed to second level arbi- trariness; the best evidence that someone is a critical thinker is his or her propensity to transfer a specifiable assortment of skills to a range of new situations. The difference between critical and uncritical thinkers may be vague and diffi- cult to quantify. It may be the case that we seldom test for these skills and disposi- tions in prevailing MeT programs and that, as such, most MeT programs are "colossally inadequate and indefensible." However, to imply that we cannot specify and so cannot assess strong sense critical thinking conflicts with the assumption that we sometimes foster it. At best, Siegel's arguments show that concentrating on iso- lated skills does not lead to the critical atti- tudes we hope to develop, and testing for isolated skills with standardized tests is somewhat counter-productive. So Siegel evidently thinks that we can but should not employ MeT for critical thinking. However, he demands too much if he thinks that testing must itselfbe "use- ful in fostering critical thinking."41 If the second horn of his dilemma presupposes such a demand, he is setting a standard that would indict most of our current pedagogi- cal practice. (I doubt whether most infor- mal logic courses produce critical thinkers, but Siegel does not advocate abolishing such courses on this account.) Yet in the end, such a demand is Siegel's main impediment to MCT in this area. However, Siegel has slipped from his initial proposal of critical thinking as a regulative ideal to a stronger demand: that of fostering critical thinking as a necessary criterion of a· pedagogical practice. But MeT of critical thinking may meet his idea of a regulative ideal; as I argued in section II, such testing might make an indirect contri- bution to the critical thinking movement, and thus foster more critical thinking. If so, MeT meets Siegel's test of a regulative ideal for adjudicating among pedagogical practices. The most that Siegel can reasonably demand from MeT is that rethinking the tests in light of our objective-producing critical thinkers-will stimulate other ped- agogical changes intended to foster this goal. So suppose we want a competence test addressing critical thinking skills and attitudes. Suppose we are willing to mod- ify our test format. If the items are drawn from real-life situations and various sub- jects that the students have studied, and are not all multiple-choice in nature, could we mitigate second level arbitrariness? If we can, the charge that MeT is inadequate and indefensible does not carry much weight. Because lapses in background knowl- edge will sidetrack any critical thinker, we must test in ways that will mitigate this problem. It seems best to test for critical thinking across a broad range of academic subjects and, in each, for a range of trans- ferable skills. Incorporating critical think- ing into MeT programs should demand more than the recall of memorized infor- mation. We must test for transfer of skills to new situations, and must judge whether students are "appropriately moved by rea- Siegel on Competency Testing 175 sons." The behaviors indicative of critical thinkers must be incorporated into the cri- teria of success. One plausible approach would be to require students to construct critical thinking portfolios, analogous to the portfolios demanded of students in the visual arts and in writing programs, but ranging over all the subjects which the stu- dent studies. Non-academic interests can also be brought in. (Such an approach strikes me as a paradigm case of compe- tency testing.) Such projects are a form of MeT, and may foster the desired disposi- tions. I have used them for four years and student comments in course evaluations indicate that they regard. the portfolio as challenging, but also as an experience that confirms the practical rewards of critical thinking in everyday life. Of over two hun- dred students, I cannot locate a single one whose comments suggest that this means of testing is counter-productive to the goal of developing critical thinkers. At the very least, we could incorporate our testing of critical thinking into any other areas for which we employ MeT (rather than use a separate test of critical thinking). Those who teach informal logic and critical thinking should build bridges to other academic disciplines, both to get a clear idea of when reason assessment is most appropriate and to integrate informal logic and related skills with the student's education in other academic areas. (From my own experience, I sense that writing instructors are becoming more sensitive to argumentative writing as a distinct genre; an integration of composition and critical thinking instruction is probably desirable.) Of all subjects, critical thinking profits least from being taught or tested in an ivory tower. To determine whether stu- dents are "appropriately moved," we should test students across a range of argu- mentative materials involving an assort- ment of subjects (e.g., social science, biology). We might present materials rang- ing from sound to the patently unsound and ask them to identify those which 176 Theodore A. Gracyk incline them to accept the conclusion and those which do not, and to explain their response in each case. Such testing for crit- ical thinking across a variety of subjects is hardly arbitrary, since it will indicate the student's abiIty and inclination to transfer skills to novel situations and to different subjects. This is not to say that we can ignore teaching and testing of discrete skills, of the sort covered in informal logic courses, provided our testing allows for the inevita- ble gaps in competence which are consist- ent with functional competence. But we must also allow for the aggregate character of general knowledge and of functional skills. As Scriven points out, correct spell- ing is part of basic literacy, yet he knows "a very good Australian philosopher who spells very badly. "42 Siegel makes the sim- ilar point that "some members of the grad- uating class may succeed in life perfectly well, even though they are unable to calcu- late compound interest payments. "43 Being a bad speller doesn't make someone illiter- ate, provided most other literacy skills are good, just as a weakness at some sorts of calculation doesn't make one mathemati- cally dysfunctional, provided enough other mathematical skills are mastered. Like- wise, ignorance of a number of informal fallacies does not preclude critical think- ing. So any competence test of critical thinking must cover the whole gamut of skills, and passing must rely on compe- tence in many but never all of them. No student should fail because any specific skills are lacking. In short, I am adapting the conclusions of Scriven and McPeck, who observe that a successful MCT will not be a standardized, multiple-choice test. Constructed as a cri- terion-referenced test which involves inter- active evaluation, MCT need not pose a serious problem with respect to second level arbitrariness. As long as the skills and attitudes are indeed transferable, such test- ing does not conflict with our general educational goals, and might foster them. We need only take care that any "mechani- cal" skills which are tested are indeed among the many skills which are some- times needed by critical thinkers. If such testing does not actively foster critical thinking, it at least seems neutral with respect to such skills. And such testing will address the needs, however undetermined, of each student's future life. Conclusion In the final analysis, I have argued that there is no inherent incompatibility between MCT and our regulative ideal of producing critical thinkers. But there is the lingering issue that, no matter what format is employed in competency testing, it might hinder development of strong sense critical thinking by encouraging students to concentrate on mastering the minimum needed to pass the test. And this might even occur with an MCT of critical think- ing. Only experience will reveal the extent to which this will occur; at present, we simply don't know. As an objection to MCT of critical thinking, it does suggest our general failure to communicate the non-academic benefits of becoming a critical thinker. Countering Siegel's concerns are sev- eral distinct reasons favoring MCT of criti- cal thinking. If critical thinking is to be a regulative educational ideal, we must pro- vide empirical evidence of success or fail- ure. Furthermore, we should certify which students are critical thinkers. MCT of criti- cal thinking might be a legitimate means for determining these. Teaching and test- ing in terms of real-life examples drawn from a broad range of occupations is not doomed to arbitrariness. Finally, MCT can be administered in a manner consistent with the educational ideals which lead us to push for critical thinking, and appropri- ate testing may promote them.44 Siegel on Competency Testing 177 Notes I In the United States, MCT is most often used at the high school level. For an introduction to the range of the debate, see Richard M. Jaeger and Carol K. Tittle, eds., Minimum Compe- tency Achievement Testing (Berkeley: McCutchan Publishing, 1980); for a critique of MCT of high school graduates, see Mitchell Lazarus, Goodbye to Excellence: A Critical Look at Minimum Competency Testing, NAESP Studies in Education and Public Policy (Boulder, Colorado: Westview Press, 1981). 2 MCT typically involves some form of stand- ardized, multiple-choice test, where a mini- mum score is set as either a necessary condition for school graduation or as an indi- cator that remedial work is required before graduating. But the idea of MCT does not pre- clude other forms of testing, and others are sometimes used (see below, n. 28). MCT also serves the function of measuring teaching effectiveness, with local success and failure rates compared to regional or national scores. MCT may be useful as a graduation require- ment even if we find it suspect as a measure of institutional effectiveness. 3 Harvey Siegel, Educating Reason (New York: Routledge, 1988), p. 116. 4 Ibid., p. 32. 5 Despite his cntIcIsms of individual writers, Siegel falls into the camp that advocates strong sense critical thinking, with the creation of critical habits, attitudes, or dispositions regarded as essential to being a critical thinker; weak sense critical thinking is restricted to the acquisition of specific technical skills. Notable among strong sense advocates are Robert Ennis, Richard Paul, and Edward D' Angelo; an overview of strong sense critical thinking is provided by Arthur B. Millman, "Critical Thinking Attitudes: A Framework for the Issues," Informal Logic Vol. X, No. I (1988), pp.45-50. 6 Ibid., p. 46. 7 John E. McPeck, Critical Thinking and Edu- cation (New York: St. Martin's Press, 1981), particularly p. 149. H Ibid., p. 5. 9 John E. McPeck, "The Evaluation of Critical Thinking Programs: Dangers and Dogmas," Informal Logic Vol. VI, No. 2 (1985), pp. 9-13. McPeck has challenged the usefulness of two major standardized tests, the Watson- Glaser Critical Thinking Appraisal and the Cornell Critical Thinking Tests. See McPeck, Critical Thillking and Education, chap. 6. 10 Notable challenges to McPeck include Richard Paul, "McPeck's Mistakes," Infonnal Logic Vol. VII, No. \ (1985), pp. 35-43, and Richard B. Miller, "Toward an Empirical Definition of the Thinking Skills," Informal Logic Vol. VIII, No.3 (1986), pp. \13-23. II Siegel, Educatillg Reason, p. 27. 12 McPeck, Critical Thinking and Education, p.149. 13 Siegel, Educating Reason, p. 28. 14 For a comparison of Siegel and other advo- cates of "strong" versions, see Richard Smith's review article, "The Limits of Rationality," Journal of Philosophy of Education 22 (1988), pp. 277-82. 15 Siegel, Educating Reason, p. 39. 16 Ibid., pp. 55-6\. Siegel has one other desidera- tum in his case for critical thinking as our edu- cational ideal, "initiation into the rational traditions," but it appears to be irrelevant to the arguments that follow. 17 Ibid., p. II 7. 18 Ibid., p. 125. 19 Ibid., pp. 174-75 (n. 26). 20 Ibid., p. 125. 21 Ibid., p. 174 (n. 25). 22 Ibid., p. 9; the distinction comes from Robert H. Ennis. 23 Ibid., p. 117. 24 My experience has been that pre-law students and those planning on graduate work are the most motivated to become critical thinkers. 25 Siegel, Educating Reason, p. 175 (n. 26). 178 Theodore A. Gracyk 26 There are two types of standardized tests. Nonn-referenced tests rate perfonnance in comparison with others who've taken the same test (the "nonnative" group). Grading on a curve is a very crude fonn of norm-referencing. If the normative group is one that previously took the test, it is a "standardization group." Major screening exams, such as the SAT and GRE, are of this sort. In contrast, criterion- referenced tests rate performance against some fixed criterion (e.g., behavioral objectives). Competency tests are of this latter sort, yet the process of standardizing the tests relies on some nonnative group to establish that the test reliably separates the sheep from the goats. See Lazarus, Goodbye to Excellence, pp.35-41. 27 For similar criticisms, see McPeck, Critical Thinking and Education, chap. 6. 28 Standardized, multiple-choice literacy exams are avoided at some universities by using essays which are independently scored by sev- eral faculty (e.g., at the University of South Alabama). The essays are judged relative to the desired level of skill (criterion-referenced), employing holistic grading techniques. A degree of objectivity is maintained by refer- ence to model "pass" and "fail" essays, and "gray area" answers are decided by additional independent scorers. Similar exam practices could be used for all sorts of MCT, including of critical thinking. 29 Siegel, Aaucating Reason, p. 175 (n. 26); his overall characterization of MCT is on p. 117. For alternatives to the multiple-choice format, see H. H. McAshan, Competency-Based Edu- cation and Behavioral Objectives (Englewood Cliffs, NJ: Educational Technology Publica- tions, 1979), chapters VII-VIII. McAshan explicitly notes the need for teacher judgment in detennining success. Along the same lines, see Jason Millman, "Reporting Student Progress: A Case for a Criterion-Referenced Marking System," Phi Delta Kappan 52 (1970), pp. 226-30, and Jerry R. Moore, Paul L. Williams, and Richard L. Needham, "Improving Testing in the Social Studies: Organizing for Local Assessment," in Paul L. Williams and Jerry R. Moore, eds., Criterion- Referenced Testing for the Social Studies, National Council for the Social Studies Bulletin 64 (Washington, D.C., 1980), pp. 38-46. See also Robert Frahm and Jimmie Covington, What's Happening in Minimum Competency Testing (Bloomington, IN: Phi Delta Kappa, 1979), pp. 20-25; Frahm and Covington report that in 1978, the state of Ore- gon required MCT in various areas but did not establish any standardized test (testing was left to individual school districts), and the Open School plan available in St. Paul, MN left it to each student to demonstrate competency in six areas to a graduation committee. 30 Siegel, Educating Reason, p. 56. 31 Williams and Moore, Criterion-Referenced Testing For Social Studies, passim. 32 See Miller, "Toward an Empirical Definition of the Thinking Skills," pp. 117-19. Even where critical thinking is required for gradua- tion, it is usually a single course in informal logic that fulfills the requirement, in which case we must be sceptical whether the diploma genuinely certifies critical thinking. 33 Siegel, Educating Reason, p. 117. 34 Ibid., p. 123. 35 Michael Scriven, "Critical Thinking and the Concept of Literacy," Informal Logic Vol. IX, Nos. 2&3 (1987), p. JOO. 36 Ibid., p. 101. 37 Siegel, Educating Reason, p. 125. 38 Ibid., pp. 122-23. Emphasis added. 39 Ibid., p. 174 (n. 25). 40 Ibid., p. 122. 41 Ibid., p. 174 (n. 26). 42 Scriven, p. 95. 43 Siegel, Educating Reason, p. 123. 44 I am indebted to the two anonymous referees for Informal Logic, whose many suggestions contributed to numerous revisions in this paper. THEODOREA.GRACYK DEPARTMENT OF PHILOSOPHY MOORHEAD STATE UNIVERSITY MOORHEAD, MN 56563 o