Critical Thinking: Concurrent Validity JOHN FOLLMAN, CAROLYN LAVELY & NEAL BERGER University of South Florida Perspective In America in the last decade there has been an explosion ofinterest in the vener- able construct of critical thinking. Critical thinking had been in repose for years until about 1983, but is now ajuggernaut. Sternberg (1985) addressed the force behind this critical thinkingjuggernaut: "Probably never before in the history of educational practice has there been a greater push to teach children to think critically." A main plank in the rationale for enhancing critical thinking of children in general is the finding that 80% of third graders, more than half of seventh grad- ers, and 36% of eleventh graders scored minimally or inadequately when reading critically on the National Assessment of Educational Progress (Vobejda, 1988, February 26). More specifically, less than 1 % of third graders, 8% of seventh graders, and 23% of eleventh graders of36,000 public and private school stu- dents performed at the highest level on reading performance. Essential to the enhancement of children's critical thinking is a valid criterion of critical thinking so that the efficacy of critical thinking pedagogies and pro- grams, not to mention children, can be determined. Consequently it is important to examine the relationships between and among the central criteria of critical thinking. Purpose Essential evidence of the validity of instruments and of other criteria of an under- lying construct is how strongly the various criteria interrelate. The purpose of this paper is to present a comprehensive compilation of the correlations between and among the concurrent criteria of the construct of critical thinking in order to articulate the nomothetic network of the relationships constituting the concur- rent validity of the measures of critical thinking. © Informal Logic Vol. 18, Nos. 2 & 3 (1996): p-p Critical Thinking: Concurrent Validity 269 Since the construct of critical reading is both deductively deemed and also empirically evidenced to be related to the construct of critical thinking, it may properly be considered as part of the critical thinking validity nomothetic net- work and therefore the correlations between critical reading and critical thinking are also included. Empirical Evidence The following table contains the zero order correlations between and among the criterial criteria, mostly tests, of critical thinking. [See Follman (1994, N ovem- ber 11) for the critical thinking nomothetic network correlation (700+) knowl- edge base.] CRITICAL THINKING CONCURRENT VALIDITY CORRELATIONS VARIABLES (MOSTLY TESTS) r REFERENCES Critical Thinking (CT) with Critical Thinking (CT) Watson GlaserCTA& tchrs.' rats. ofpupils' CT. 52,.48,.33,.40 AC.E. Test ofCT Form G & CT in Social Science .65 Wrightstone Test ofCT -Soc. Stud. & G 6 pup.ints. .89, .84, .87 W -G & X tchrs.' rats. G 12, Eng .. 33 to .52, mdn. .44 A.C.E. CT Form G & rats. ofCT, .42 to .66, mdn. .50 AC.E. Test ofCT Form G & Pract Log., freshmen .59 A.C.E. TestofCTForm G& Sci. Reas. fresh. wmn. 39 AC.E. Form G & Crit. Anal. Read.IWrit fresh. wmn. .50 A.C.E. Form G& Crit Anal. Read.IWrit fresh. wmn. .53 AC.E. Form G & Crit Anal. Read.IWrit. fresh. wmn. .51 AC.E. Form G& Crit Anal. Read.IWrit. fresh. wmn. 32 AC.E. Form G & CT in Soc. Sci. freshmen women .69 AC.E. TestorCT Form G & CT in Soc. Sci., fresh. .66 A.C.E. TestofCT Form G & CT in Soc. Sci., fresh. .67 AC.E. TestFormG&Crit. Judg. in Lit., G 13,14 38 AC.E. TestofCT Form G& W-GCTA,fresh. wmn. .61 AC.E. TestofCT Form G& W-GCTA, freshmen .53 CT in chem.,sec.,.33-.6I ,mdn..45(corr .. 46-.87 mdn. .64) Crit. thinking test & tchr. CT rankings, G 4,5,6 unrelated AC.E. TestofCTFormG& W-GFormZM .44 TestofCTFormG&CornellCTFormZ .44 Curry TestofCT & Tchrs.' CT evaluations, G 9- 12 .61 Watson Glaser CT Form ZM & Cornell CT Form Z .48 Uncritical Inference Test & W-G CTA 25 Reflective Judgment Inv. & Cornell CT, G 13-17 .62 Reflective Judgment Inv. & W-GCTAA,G 13-17 .61 W-GCTA FormA & Cornell Crit. Think., G 12-16 .79 Reflective Judg. lntrvw.& W-GCTAFormA,G 12-16.40 Appr. Obs. Think. & Perf. A .59, B .77, Whole, sec. .68 Test on Appraising Observations & Cornell CT, sec .. 62, .49, .45,.35 Test on Appraising Observations & W-G CTA, sec. .37,.I I, .37, .41 W -G CT A & Cornell Crit. Think. Form Z, college .71 Test on Appraising Obs. A & Cornell CT Level X .48 Teston Appraising Obs. A& W-GCTA Form A 31 Test on Appraising Obs. A & verbal reports, cone. .50 Test on Appraising Obs. A & verbal reports, subs. 31 Glaser (194 I) CTFormGMan. (1951) Cotter(1951) Watson & Glaser (1952) Dressel & Mayhew( I 954) Dressel & Mayhew( 1954) Dressel & Mayhew(1954) Dressel & Mayhew(1954) Dressel & Mayhew( 1954) Dressel & Mayhew(I 954) Dressel & Mayhew( 1954) Dressel & Mayhew( I 954) Dressel & Mayhew(1954) Dressel & Mayhew(1954) Dressel & Mayhew( 1954) Dressel & Mayhew(1954) Dressel & Mayhew( 1954) Max (1954) George & Dietz (1968) Follman (1970) AC.E. Follman (1970) Curry (1971) Ennis & Millman (197I) Stewart (1979) Mines (1980) Mines (1980) Brabeck (1983) Brabeck (1983) Norris & King (1984) Norris (1989) Norris (1989) Mines et at. (1990) Norris (1990) Norris (1990) Norris (1990) Norris (1990) 270 Follman, Lavely & Berger VARIABLES (MOSTLY TESTS) r Test on Apprais. Obs. A & verb. rep., conc. & subs. .49 Test ofCrit. Think. in Biology & W·G CT A, college .64 TestofCrit. Think. in Biology & ACT Nat. Sci., coil. .68 W·G CT A & ACT Natural Science, college .78 Test on Appraising Obs. & Cornell CT, secondary .62, .48 Teston Appraising Obs. & W-G CTA, secondary .31,.35 Cornill CT Level subtests & Developing Cognitive alilow Abilities Test subtests, college Collegiate Assess. of Acad. Prof., & W -G CTA, coil. .75 REFERENCES Norris (1990) McMurrayetai. (1991) McMurrayetai. (1991) McMurray et aI. (1991) Norris (1992) Norris (1992) Farley & Elmore (1992) Pascarella etal. (1994) Critical Thinking (CT) with Critical Reading (CR) W -G CTA & Martin Reading Comp., initial, G 12 .77 Mart. Read. Comp., retest (interv.) G 12 .82 Inference & Martin Reading Compo total .75 Eval. of Arg. & Mart. Read. Compo total .77 W -G CT A Generaliz. & Martin Reading Compo total .42 W -G CTA Discr. of Argus. & Mart. Read. Compo tot. .49 W-GCTA Gen. Log. Reas. & Mart. Read. Comp. tot. .71 Martin Read. Comp. & AC.E. Psych. Exam IQ, G 12 .79 Crit. think. subtests & critical reading subtests .84 AC.E. CTFormG& Martin Reading Comp., G 12 .63 Cornell Critical Thinking Form X (CT), G 5 Intermediate Reading Test-Science (CR-SCI) Intermed. Read. Test-Social Science (CR-SS) CT:CR-SS .61 CT:CR-SCI .55 CR-SS:CR-SCI .70 Maw CT & STEP Read. A and B Crit. Read., G 6 .51 Maw CT & STEP Read. A and B Crit. Read., G 8 .59 Cal. CT Skills Test & Nelson-Denny Reading Test .49 Glaser (l94 I) W-GCTA & Glaser (1941) W-GCTA Glaser (194 I) W-GCTA Glaser (1941) Glaser (I 941) Glaser (1941) Glaser (I 941) Glaser (1941) Johnson et al. (1971) Follmanetal (1971) Follman et aI. (1972) Follmanetal. (1972) Follman eta!. (1972) Follman et aI. (1972) Follman et aI. (1972) Follman et aI. (1972) Sullivan (I 973) Sullivan (1973) Facione (1990) Keep in mind that most of these correlations are uncorrected for unreliability, but if corrected, would be higher, in most cases, materially higher. Overview of Table I indicates a substantial amount of concurrent validity research but its re- sults are mixed. The correlations are mostly moderate to strong, as noted, and would be even stronger if corrected. Another reason why these correlations are not higher is the criterion problem. Put succinctly the criterion problem is that the criteria of a construct have less psychometric integrity than its predictor tests. This has probably inhibited progress in the measurement of critical thinking. The correlations also reflect the tendency of a wide variety of researchers, with a wide variety of purposes, to perceive a wide variety of variables as representing part of or even all of the construct of critical thinking. Finally, the correlations between the critical thinking tests and the critical reading tests, ranging from .42 to .84 with a median of. 71, are impressively high, even without correction, ironi- cally considerably higher than the correlations between the critical thinking tests and the other critical thinking tests. Critical Thinking: Concurrent Validity 271 REFERENCES Cotter, K.C. (1951). A study of the critical thinking in the social studies of sixth grade children in the public schools of the City of New York. Ph.D. dissertation, New York, NY: Fordham University. Curry, 1.F. (1971). The construction and evaluation of a test of critical thinking for second- ary school students. Ed.D. dissertation, Boston, MA: Boston University, School ofEdu- cation. Dressel, P.L., & Mayhew, L.B. (1954). General education explorations in evaluation. Washington, DC: American Council on Education. Ennis, R.H., & Millman, J. (1971). The Cornell Critical Thinking Test. Urbana, IL: Criti- cal Thinking Project. Facione, P. (1990). The California Critical Thinking Skills Test: Technical Report #2- Factors Predictive ofCTSkills. ERIC Doc. No.: TM 015819. Farley, H.1., & Elmore, P.B. (1992). The relationship of reading comprehension to critical thinking skills, cognitive ability, and vocabulary for a sample of underachieving college freshmen. Educational and Psychological Measurement, 52, 921-931. Follman,1. (1970). Correlational and factor analysis of critical thinking, logical reasoning, and English total test scores. Florida Journal of Educational Research, 12,91-94. Follman, 1., Lowe, A.J., & Wiley, R. (1971). Correlational and factor analysis of critical reading and thinking test scores - twelfth grade. Twentieth Yearbook of the National Reading Conference, 20, 128-136. Follman, 1., Lowe, A.1., Johnson, R., & Bullock, 1. (1972). Correlational and factor analy- sis of critical reading/critical thinking - fifth grade. Journal of English Language Teach- ing, 4,141-150. Follman, J. (1994, November II). Cornucopia of correlations X: Critical thinking. Paper presented at the annual meeting of the Florida Educational Research Association, Tampa, Florida. George, K.D., & Dietz, S.M. (1968). The relation of teacher-pupil critical-thinking ability. Science Education, 52, 426-432. Glaser, E.M. (1941). An experiment in the development of critical thinking. New York: Columbia University, Teachers College, Bureau of Publications, No. 843. Johnson, R., Follman, 1., Wiley, R., Lowe, AJ., & Miller, W. (1971). Canonical and partial correlation of critical reading/critical thinking test scores- twelfth grade. Twentieth Year- book of the National Reading Conference, 20, 142-147. Max, H.J. (1954). The development of an instrument to measure an aspect of critical think- ing in the area of high school chemistry. Ed.D. dissertation, Urbana, lL: University of Illinois. McMurray, M.A., Beisenherz, P., & Thompson, B. (1991). Reliability and concurrent va- lidity of a measure of critical thinking skills in biology. Journal of Research in Science Teaching,28,183-192. Mines, R.A. (1980). Levels ofintellectual development and associated critical thinking skills in young adults. Ph.D. dissertation, Iowa City, lA: University oflowa. Norris, S.P. (1989). Verbal reports of thinking and multiple-choice critical thinking test de- sign.Institute for Educational Research and Development, Memorial University of New- 272 Follman, Lavely & Berger foundland, and Center for the Study ofReading, University of Illinois at Urbana-Champaign, Technical Report No. 447. Norris, S.P. (1990). Effect of eliciting verbal reports of thinking on critical thinking test performance. Journal of Educational Measurement, 27, 41-58. Norris, S.P. (1992). A demonstration ofthe use of verbal reports of thinking in multiple- choice critical thinking test design. The Alberta Journal of Educational Research, 38, 155-176. Norris, S.P., & King, R. (1984). The design ofa critical thinking Test on Appraising Observations. Institute for Educational Research and Development, Memorial University ofNewfoundland, S1. Johns, Newfoundland. Pascarella, E., Edison, M., Whitt, E.J., Nora, A., Hagedorn, L.S., & Terenzini, P. (1994). Cognitive effects of Greek affiliation during the first year of college. National Center on Postsecondary Teaching, Learning, and Assessment, University Park, PA, ED 381 053. Sternberg, R.J. (1985). Teaching critical thinking, Part I: Are we making critical mistakes? Phi Delta Kappan, 67, 194-198. Sullivan, J. (1973). The relationship of creative and convergent thinking to literal and critical reading ability of children in the upper grades. The Journal of Educational Research, 66, 374-377. Test of Critical Thinking Form G, Manual (I 951). Washington, DC: American Council on Education. Vobejda, B. (I 988, February 26). Students reported deficient in critical-thinking skills. Washington Post, A 17. Watson, G., & Glaser, E.M. (1952). Watson-Glaser Critical Thinking Appraisal Manual. Yonkers-on-Hudson, New York: World Book Company. JOHN FOLLMAN, CAROLYN LAVELY & NEAL BERGER COLLEGE OF EDUCATION UNIVERSITY OF SOUTH FLORIDA TAMPA, FL 33620-7750