A PSYCHOMETRIC ANALYSIS OF KOLB’S REVISED LEARNING STYLE-INVENTORY Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 147 A PSYCHOMETRIC ANALYSIS OF KOLB’S REVISED LEARNING STYLE-INVENTORY Thomas L. Ruble, Rider College David E. Stout, Villanova University ABSTRACT This study investigated the psychometric properties of Kolb’s 1985 revision of the Learning-Style Inventory (LSI). The LSI was revised to overcome some of the psychometric limitations of the original instrument. Results from this study suggest that the internal consistency of the LSI has been improved but some of this gain may be due to a response set. Despite gains in internal consistency, factor analysis failed to support the bi-polar assumptions of the LSI. Additional studies are recommended to clarify the validity of the LSI and the bi-polar view of experiential learning, which provides the rationale for the LSI. INTRODUCTION The Learning-Style Inventory (LSI) was introduced in the 1970s by David Kolb (Kolb, 1974) to measure an individual’s use of four different learning abilities: (1) concrete experience (CE), (2) abstract conceptualization (AC), (3) reflective observation (RO), and (4) active experimentation (AE). The LSI was based on the Experiential Learning Model (ELM), a two dimensional model for classifying learning styles corresponding to different stages in the learning process. The ELM assumed that CE and AC were opposite abilities as were RO and AE. Thus, the model proposed two separate, bi-polar dimensions in experiential learning. Since its introduction, the instrument and model have generated interest in a wide range of disciplines and over 200 references recently have been documented (Smith & Kolb, 1986). The original LSI was criticized as having a variety of psychometric limitations (Certo & Lamb, 1980; Freedman & Stumpf, 1978; Lamb & Certo, 1978; Merritt & Marshall, 1984; Ruble, 1978; Sims et al., 1986). Recently, however, the LSI has been revised to over- come some of the problems of the original version (Kolb, 1985; Smith & KoIb, 1986). The revised version, LSI-1985, has 12 blocks of four items. Each block has one item representing each of the four learning abilities. Individuals are asked to rank order the four items to describe their preferred approach to learning. The items are arranged in four columns on one page so that all items for a single scale are in the same column. The columnar format is designed to facilitate self-scoring. Early research on the revised LSI indicates that some improvement in the psychometric properties has been achieved. For example, a fundamental problem with the original LSI was the reliability of the scales. With 12 items per scale, the revised LSI is twice the length of the original version. Studies of the revised LSI indicate that coefficient alpha reliability estimates for individual scales range from .73 to .86 with an approximate average in the low .80s (Sims, Veres, & Shake, 1989; Sims, et al., 1986; Smith & Kolb. 1986; Veres, Sims, & Shake, 1987). Although these coefficients represent a substantial improvement in internal consistency over the original, it has been recognized that some of this gain may be due to a response set created by the columnar-scoring format (Sims et al., 1986; Smith & Kolb, 1986). Despite the improvement in internal consistency, studies examining test-retest reliability and classification stability have not been encouraging. The few studies to date have found rather low test- retest correlations with the time Intervals ranging from nine days to five weeks (Atkinson, 1988; Sims, et al., 1986; Veres, Sims, & Shake, 1987). However, these results may reflect the dynamic nature of different learning situations and further research is needed to clarify the classification stability of the revised LSI. After the reliability of the LSI-1985 has been examined, the next issue is construct validity. Construct validity is based on the extent to which the revised LSI measures constructs consistent with the two bipolar dimensions proposed by the ELM. Factor analyses of the original LSI yielded mixed results (cf., Ferrell, 1983; Freedman & Stumpf, 1978; Merritt & Marshall, 1984). However, some studies suggested that support for the bi-polar model was based on instrument bias due to the ranking method of scoring (Certo & Lamb, 1980; Freedman & Stumpf, 1978; Ruble, 1978). At this writing, the authors are not aware of research on the factor structure of the revised instrument. However, one study has reported a correlation matrix of the four scales, which suggests that the revised LSI may not represent two separate, bi-polar dimensions (Highhouse & Doverspike, 1987). The present study investigates the psychometric characteristics of the revised LSI. First, we examine the internal consistency of the four scales (CE, AC, RO, AE). Next, the study focuses on the construct validity of the instrument. Construct validity can be assessed in part by examining the pattern of inter-correlations of the four scales. However, factor analysis provides a more comprehensive view of the pattern of interrelationships among all the items and provides a basis for judging the efficacy of the revised LSI in representing the two-dimensions proposed by the ELM. METHOD Subjects Three hundred twelve upper division business students participated in the study. The sample was heterogeneous in terms of site of school and geographical distribution. Approximately 53% of the subjects were male. Procedure The LSI was administered with another instrument as part of a larger study. The additional questionnaire was a 60 item, forced- choice psychological instrument. This questionnaire took approximately 15 minutes to complete. Approximately half of the Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 148 subjects completed the LSI first (n149) while the remainder completed the LSI after the companion questionnaire (n163). Completing the LSI with a second instrument allows an assessment of order effects or fatigue in self-reports of learning style. RESULTS Descriptive Statistics Table 1 presents a comparison of the scale means and standard deviations of the current sample with the normative sample provided in the LSI user’s guide (Smith & Kolb, 1986). The descriptive statistics are very close to those reported by Smith and Kolb in that the order of preference (AE, AC, RU, CE) is the same and the means and standard deviations are close for the AE, AC, and RU scales. The current sample had a slightly lower mean score and higher standard deviation on the CE scale. Reliability Coefficient alpha was used to estimate the internal consistency of the scales. The coefficients for different orders of completing the Instruments are presented in Table 2. The reliabilities are very close to the reliabilities reported by other investigators (Sims, Veres, & Shake, 1989; Sims, et al., 1986; Smith & Kolb, 1986; Veres, Sims, & Shake, 1987). The effects of order had little impact on reliabilities of CE, RO, and AE. However, coefficient alpha for AC increased from .77 when the LSI was completed before the other instrument to .R6 when the LSI was completed after. Given the single-scale-per-column format, it may be easier to follow a pattern if one gets tired or bored with ranking items. Construct Validity To assess Construct validity, we examined first the intercorrelations of the four scales and then used factor analysis to provide more insight on the underlying relationships of all the items. Intercorrelations The intercorrelations of the scales are presented in Table 2. The results indicate that the four scales show moderate, negative correlations in all Cases. Given the ranking format of the LSI, the negative correlations are expected. However, given the bi-polar assumptions of the ELM, the magnitudes of the correlations are not expected. The ELM would be supported if the proposed bi-polar pairs showed stronger negative correlations with each other (CE with AC and RU with AE) and weaker correlations with the non- opposite scales. Clearly, this expected pattern did not emerge. The correlation of CE with AC was -.39 and RU with AE was -.28. The remaining correlations of the non-opposite scales ranged from -.29 to -.38. All the correlations were significant at p<.001, two-tailed. Factor Analysis Principal components factor analysis with Varimax rotation was used to assess whether the LSI measures the bi-polar dimensions proposed in the ELM. Separate analyses were performed for a two- factor solution and a four-factor solution. A two-factor solution was extracted because the ELM proposes two bi-polar dimensions. Thus, the two-factor solution would be most relevant to examine the construct validity of the LSI. A tour-factor solution was extracted also because the ELM proposes four distinct learning abilities. One criticism of the original LSI was that the two factors extracted in factor analysis accounted for a relatively small proportion of total measured variance (Freedman & Stumpf, 197P; 1980). In the present study, the two-factor solution accounted for 28.4% of the total variance. This represents a small gain over the 20.6% reported by Freedman and Stumpf (1978). However, some of this gain may be explained by a response set resulting from the single-scale-per- column format. Regardless, these results do not provide much support for the efficacy of the revised LSI as a measure of the two bipolar dimensions. In the four-factor solutions, the third factor accounted for an additional 11.5% of the total variance while the fourth factor accounted for little additional variance at 5.0%. Factor Patterns Table 3 presents the results of the rotated factor patterns extracted by the two separate analyses. The two-factor solution essentially yielded one factor of CE items and one factor of AC items. Only three items from the AE scale and none of the RU items appeared in the two-factor solution. Given these results for the two-factor solutions, it would appear that the revised LSI does not represent the expectations based on the ELM. To support the assumptions of the bi-polar model, CE items and AC items should have loaded together on one factor while RU items and AE items loaded together on the other factor. On each factor, one set of items should be positive while the other set should be negative (polar opposites). In the four-factor solution, the AC, RO, and AE scale Items separately loaded on one, and only one, factor. The CE items had weaker factor loadings and loaded on two different factors. Thus, three of the four Developments In Business Simulation & Experiential Exercises, Volume 17, 1990 149 learning abilities seem to be measured distinctly while the fourth ability, CE, is somewhat ambiguous. DISCUSSION To be valid, a psychological instrument must first be reliable. The original LSI had such low reliability estimates that assessments of construct validity were actually premature. Given the evidence of improvement in the internal consistency of the revised LSI, research can now consider construct validity. For the revised LSI, construct validity includes the issue of whether the instrument represents the two bi-polar dimensions proposed by the ELM. The results of the present study do not support the construct validity of the revised LSI. Examinations of the matrix of intercorrelations and factor patterns did not yield evidence of two separate bi-polar dimensions. Future Directions Clearly, one study can neither confirms or reject the hypothesized relations between the LSI and ELM. However, the present study raises questions for future research: (1) what is the effect of the possible response set on the LSI’s reliability? (2) Are the learning abilities two bi-polar tendencies or four separate learning styles? (3) Does the revised LSI provide the best approach for measuring learning styles? These are just some of the issues as we rethink the roles of the LSI and ELM in experiential pedagogy. REFERENCES Atkinson, G., Jr. (1988), Reliability of the Learning Style Inventory --1985. Psychological Reports, 62, 755-758. Certo, S. C., & Lamb, S. W. (1980), An investigation of bias within the Learning-Styles Inventory through factor analysis. Journal of Experiential Learning and Simulation, 2, 1-7. Ferrell, B. C., A factor analytic comparison of four learning-styles instruments. Journal of Educational Psychology, 75, 33-39. Freedman, R. D., & Stumpf, S. A. (1978), What can one learn from the Learning Style Inventory? Academy of Management Journal, 21, 275-282. Freedman, R. D., & Stumpf, S. A. (1980), Learning Style Inventory, Less than meets the eye, Academy of Management Review, 5, 445-447. Highhouse, S., & Doverspike, D. (1987), The validity of the Learning Style Inventory --1985 as a predictor of cognitive style and occupational preference. Educational and Psychological Measurement, 47, 749-758. Kolb, D. A. (1974), Management and the learning process. In Kolb, D.A., Rubin, I. M., & McIntyre, J. M. Organizational Psychology: A Book of Readings, (pp.27-42), Englewood Cliffs, NJ: Prentice-Hall. KoIb, D. A. (1985), LSI: Technical Manual. Boston: McBer and Co. Lamb, S. W., & Certo, S. C. (1978), The Learning Styles Inventory (LSI) and instrument bias. Proceedings of the 38th Annual Meeting of the Academy of Management, 28-32. Merritt, S. L., & Marshall, J. C. (1984), Reliability and construct validity of ipsative and normative forms of the Learning Style Inventory. Educational and Psychological Measurement, 44, 463-472. Ruble, T. L. (1978), A psychometric analysis of Kolb’s Learning Styles Inventory. Proceedings of the 5th Annual Meeting of ABSEL, 193-198. Sims, R. R., Veres, J. G., Watson, P., & Buckner, K. E. (1986), The reliability and classification stability of the Learning Style Inventory, Educational and Psychological Measurement, 46, 753-760. Sims, R. R., Veres, J. G., & Shake, L. C. (1989), An exploratory examination of the convergence between the Learning Styles Questionnaire and the Learning Style Inventory II, Educational and Psychological Measurement, 49, 227-233. Smith, D., & KoIb, D. A. (1986), User’s Guide for the Learning Style Inventory. Boston: McBer and Co. Veres, J. G., Sims, R. R., & Shake, L. G. (1987), The reliability and classification stability of the Learning Style Inventory in corporate settings, Educational and Psychological Measurement,47,1127-1133. Table of Contents Volume 17, 1990 The Impact of Decision Support Systems on the Effectiveness of Small Group Decisions - Revisited The Relationship Between Financial Performance and Other Measures of Learning on a Simulation Exercise Use and Effectiveness of an Analogy-Based Expert System Suggestions for Computerized Business Authors Dealing with Power: An Experiential Exercise Using Movie and Personal Diary Analysis Techniques A Model for Developing Student Skills and Assessing Outcomes Through Outdoor Training Computer-Aided Exercises Versus Workbook Exercises as Learning Facilitator in the Principles of Marketing Course An Exposition of Guilford's Si Model as a Means of Diagnosing and Generating Pedagogical Strategies in Collegiate Business Education Formal Planning and Simulation Team Performance: A Cross Sectional Approach Cases: Real Organizations in Real Time in the Classroom An Empirical Investigation of the Internal Validity of Marketing Simulation Game An Empirical Evaluation of the Pedagogical Value of Playing a Simulation Game in a Principles of Marketing Course Factors Affecting Effective Teaching of Strategic Planning: Some Preliminary Evidence An Experiential Exercise for Learning About the Relationship Between Organizational Form & the Project Management Process Accounting Communication Skills can be Taught in the Auditing Course Modeling Cost Functions in Computerized Business Simulation: An Application of Duality Theory and Sheppard's Lemma A Life Cycle Analysis of Decision Making for a Strategic Management Team What's the Problem? A Dynamic Model for Teaching Problem Solving Skills Experientially International Currency Fluctuations: Money$im, A Simulation Superstores: A Specialized Retailing Simulation Within a Specialized Marketing Curriculum Factors Affecting Student Perceptions of Learning in a Business Policy Game VC + EL = VL The Name Game: An Experiential Exercise in Intergroup Relations The Effects of Experiential Accounting Work Experience on Student Performance in Intermediate Accounting Courses The Results of Using the Experiential Activity Group Performance Evaluation in a Business Policy Setting Using a Legal Database to Describe the Legal Environment of Marketing (and Business) Matching Environmental Uncertainty and Organizational Configuration An Instructional Computer Simulation of Tampering in QC Executive Evaluation of Student Learning in the Looking Glass Simulation Group Personality Composition and Total Enterprise Simulation Performance An Expert System for Selecting Analytical Techniques for Analyzing Marketing Research Data Effects of Cognitive Styles on Responses in an In-Basket Simulation A Psychometric Analysis of Kolb's Revised Learning Style-Inventory Selecting and Developing Experiential Exercises Using Movies Application of a Real-World Strategic Management Model in the Classroom Demand Equations which Include Product Attributes Consumption as the Objective in Computer-Scored Total Enterprise Simulations The Effects of Decision Format and Evaluation on Simulation Performance, Decision Time, and Team Cohesion The Effects of Computer Related Assignments on Student Performance in Business Administration The Money Game: A Dynamic Simulation Including Random Shocks for Money and Banking Courses Methods for Evaluating Performance on Business Simulations: A Survey The Effects of Synergogy on the Policy Course: Significant Improvements in Student Learning and Teacher Evaluation Conditions and Outcomes of Trust in a Two-Person Bargaining Exercise Bankgame Enhancing Computer Business Simulation with the Use of VGA Graphics An Experiential Approach to Entrepreneurship An Advanced Simulation Method (ASM) for Multiple Objective Problems The Influence of Experiential learning Techniques on Student Recognition of Non-Primary Learning Styles Negotiating Mergers and Acquisitions: A Cocktail Napkin Approach Identification of Unintended Effects in Experiential Laboratory Exercises An Experimental Comparison of Paper and Pencil and Computer Aided Decision Support Tools Porting a Simulation from the IBM World to the Macintosh World A Transaction Cost Analysis of Experiential Learning The Development of Experiential Exercises for Courses in Entrepreneurship and Small Business Management The Assessment Center as and Experiential Classroom Exercise Pricing Strategy Algorithms for Playing Business Simulations Organizational Structures for International Operations: An Experiential Activity Simulation Emphasis in the Business School Capstone Course An Integrated Approach to Computerizing the Business Curriculum Cross-Cultural Business Negotiations Exercise Organizational Socialization and Gender Differences in Students at Work Understanding Student Work Experience: A Content-Analytic Approach Introducing Executive MBA Programs with Management Games An Analysis of Improvement in Business Decision Outcome with Sequential Use of Two Simulation Games Teaching Business Policy Utilizing Mass Lecture and Individual Case Labs Potholes Along the Road to Evaluating Learning Outcomes: The Case of Outdoor Management Training Experiential Learning for Interior Design Students: Using CADD, Lotus 1-2-3, and Wordperfect Cognitive Learning Using a Computer-Based, Qualitative Interactive Business Simulation Sex Discrimination: Does the Woman get the Job or Does the Best Man Win? A Search for Visual Aids to Support Experiential Learning Through the 1990's An Experimental Analysis fo the Effectiveness of Student Role-Playing in Sales Training How to Have Students Learn from their Term Projects Self-Evaluation Exercise (SEE): An Assessment of Class Contribution A Study of the Influence of Team Formation on Attitudes and Performance in Management Games A Hardware Based CIM Simulation Laboratory Model Test Substance Abuse in Organizations Micro Computer Training Models Teaching Forecasting, Cash Budgeting and Inventory Model Building Using SBTools A Comparison of the Effects of Experiential Learning Activities and Traditional Lecture Classes An Adjunct Writing Instruction Assistant, The Computer; With an Illustration Classroom Software for ABC Analysis Utilizing Information Processing Technology to Enhance the Business Policy Simulation Experience Progressive Cases Realistic Job Previews Vs. Traditional Job Previews: Experiencing the Differences and Understanding the Consequences Investment Analysis Using the Pragmatic Multiplier Approach A Computer Simulation Interface for Competitive and Firm Analysis Self-Assessment of Ethical Decision Making Predispositions Preparing Managers for Overseas Assignments An Inquiry into Japanese Marketing: Workshop on Teaching Japanese Marketing The Performance Appraisal Feedback Interview: A Role Play for Human Resources Management Teaching Counselor Selling techniques Using Experiential Techniques to Teach International Topics International Management Simulation Gaming: Current Status and Future Developments A Realism Comparison of Simulation Technologies/Methodologies A Time-Efficient Game to Illustrate Concepts Taught in Management Courses and Management Development Programs Teaching the Management of Technology The Concept of Face and the Applicability of Experiential Exercises in an Oriental Culture's