REPORT ON PROGRAMMATIC RESEARCH ON PERCEIVED LEARNING BARRIERS WITH SIMULATION AND EXPERIENTIAL LEARNING Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 147 REPORT ON PROGRAMMATIC RESEARCH ON PERCEIVED LEARNING BARRIERS WITH SIMULATION AND EXPERIENTIAL LEARNING Alvin C. Burns, Louisiana State University Steven P. Golen, Louisiana State University James W. Gentry, Oklahoma State University ABSTRACT The authors report the findings of programmatic research on the perceived seriousness of communication barriers to learning across five pedagogies. The theoretical basis and general method are related. Consistent findings have uncovered a generalized ordering of specific concerns as well as dimensionality of learning barriers. Discipline- specific differences do not appear to be present, but method and male-female differences are evident. The generalizability of the findings suggests that pedagogical research heretofore bound by discipline or university may be more generalizeable than considered previously. INTRODUCTION This paper is a compilation of research findings over the last two years of focused investigation on students’ perceptions of learning problems they encounter in various pedagogical situations [1; 2; 3; 5]. To be more exact, our research has been concerned with the possible learning problems business students envision with alternative pedagogies. Furthermore, we have sought to sort out the effects of: different subject matter; different types of students; different universities; and, of course, alternative teaching methods. Our findings have been surprisingly consistent, suggesting to us a measure of corroboratory validity. In any case, it is our intention to describe the theoretical underpinnings of our investigations, the methodology adopted, and the consistent findings we have discovered across studies. THEORETICAL BASIS OF THE RESEARCH We have begun with the basic premise that education requires communication as its most fundamental level. The instructor, possessing some piece of knowledge, wishes that his students possess it as well. A wide range of alternatives exists as to the method of communication. Regardless of the method selected, the transfer of knowledge requires use of a communication process. It is this process which serves as the theoretical basis of the research. Basically, all communication systems share similar characteristics. Some experience, fact, object, or idea is observed by the communicator; it is encoded into a message and the channel is determined; the message is received by an individual who decodes it into an idea or concept; action is taken based on the message interpretation; finally, feedback from the receiver is given to the communicator. An implicit assumption is the free flow of information from the communicator to the receiver. However, this assumption is not always realistic, for problems occur which inhibit the free flow. These problems, whether within the communicator, within the receiver, or external to both, are referred to as barriers to communication. They have been identified by Shannon and Weaver (1949) as forms of noise which impede the reception of the ideas being communicated. The notion of Interference has been discussed by others such as Tubbs [8] and Schramm [71. The communications literature makes clear the multidimensionality of barriers to effective communication. In fact, Chruden and Sherman [4] developed a model showing the relationship of potential barriers to the communication process as depicted in Figure 1. RESEARCH METHOD The operationalization of the variables began with a compilation of possible communication barriers based on review of the literature. Since the communication process flows back and forth between the Instructor and students as well as between students over time, a particularly large number of potential problems eventuated. Each was summarized in a descriptive phrase and pretested for clarity. Forty-six separate barriers have been used in all of our studies, and a complete listing is found in Table 1. The operationalization of various teaching methods was effected by the use of scenarios describing the basic approach of each of four methods: computer simulation, experiential exercise, case studies, and live case studies. A “straight lecture” method was included as a control benchmark. Each scenario was worded similarly and instructed the student-subject to assume that the method would be used by his or her instructor in the principles of (e.g.) marketing course and that about 50% of the class Lime would be devoted to student’s participation in the method. Next, the student- subjects were instructed to indicate the seriousness of each item as a hindrance to learning the subject matter of the course by indicating a number from 1 to 5 where “1” signified “not serious” and a “5” represented “very serious.” All of our investigations have sought to embody good experimental design. The stimulus for this goal stems from a critique of pedagogical research based on a review of over 500 studies by Kulik, Kulik, and Cohen [6] which concludes that most studies fail the test of rigorous experimentation. Consequently, all of the studies have utilized an equivalent- groups design in which students were randomly assigned to each teaching method treatment, and each student responded to only one scenario. In those instances where different course subject matter was tested, the random assignment was applied to the discipline as well. That is, each student- subject responded to only one teaching method-subject matter pair. Figure 2 summarizes the important specifics of the various studies conducted so far. Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 148 FIGURE 2 SUMMARY OF STUDIES CONDUCTED STUDY INDEPENDENT VARIABLES MODERATOR VARIABLES POPULATION I 5 Pedagogies Student Ability Marketing Students 2 Computer Simulation Major, Sex, GPA, Work Status, Class level, Etc. Business Students 3 5 Pedagogies GPA, Sex Accounting Students 4 5 Pedagogies 4 Disciplines 2 Universities, Major, GPA, Sex Business Students The analyses applied to these data sets have been fairly straightforward. Once the equivalence of the groups was assured via statistical test, determination of the relative seriousness of each barrier item has generally been undertaken with computation of grand means. This step was taken to assess variability in the data as well as to investigate the overall perceived seriousness of the learning barriers irrespective of pedagogy, subject, or moderator variables. Next, factor analysis was applied to ascertain dimensionality be reducing redundancy among the variables. Varimax rotation was applied to enhance interpretability, and Cronbach’s alpha was computed wherever appropriate to determine the reliability of each factor-dimension. Finally, analyses of variance were performed to test for significant differences between the teaching methods (or other independent variables as appropriate). Duncan’s multiple range test was adopted as the post hoc test whenever significance levels beyond .10 were found. TABLE 1 RELATIVE SERIOUSNESS OF THE COMMUNICATION BARRIERS* 4.20 1. Instructor’s Tendency Not to Listen 2. Students’ Tendency Not to Listen 4.09 3. Overload or Too Much Information for Students 4.09 4. Instructor’s Hostile Attitude 4.09 5. Personality Conflicts Between Instructor and student 4.02 6. Lack of Feedback to Instructor 4.00 7. Instructor’s Lack of Credibility 3.95 8. Instructor’s Prejudices or Biases 3.93 9. Students’ Lack of Interest in the subject Matter 3.92 10. Differences in Perception Between Instructors and Students 3.82 11. Lack of Trust in Instructor 3.82 12. Students Lack of Understanding of Technical Language 3.81 13. Students' Hostile Attitude 3.80 14. Instructors Either-Or Thinking 3.71 15. Students’ Resistance to Change 3.68 16. Students’ Inadequate Knowledge of the Topic 3.67 17, Students’ Either-Or Thinking 3.66 18. Poor Timing of Requests and Instructions 3.63 19. Lack of Feedback to Students 3.62 20. Physical Noise or Distraction 3.58 21. Excessive Size of a Group 3.58 22. Too Many Intermediate Receivers Between the Sender and Intended Receiver of the Information 3.46 23. Students’ Know- Attitude 3.46 24. Instructor’s Know-It-All Attitude 3.45 25. Students’ Poor Organization of Ideas 3.43 26. Instructor’s Emotional Reaction 3.34 27. Students’ Prejudices or Biases 3.33 28. Lack of Trust in Other Students 3.27 29. Students’ Fear of Distortion or Omission of Information 3.23 30. Personality Conflicts Between Students 3.16 31. Physical Distance Between Student Sender and Student Receiver of Information 3.16 32. Students’ Emotional Reaction 3.07 33. Students; Defensiveness 3.02 34. Differences in Perceptions of Students 2.95 35. Students’ Prematurely Jumping to Conclusions 2.84 36. Students’ Use of Profanity 2.83 37. Informal Social Groupings or Cliques 2.76 38. Instructor’s Use of Profanity 2.75 39. Students’ Lack of Credibility 2.75 40. Poor Spatial Arrangement 2.73 41. Students Inability to Understand Non-Verbal Communication 2.72 42. Students With Overly Competitive Attitudes 2.56 43. Differences Between Status of Instructor and Students 2.54 44. Differences Between Students’ Status 2.36 45. Students’ Speaking Too Loudly 2.25 46. Inappropriate Physical Appearance of Students 1.85 ‘Based on a scale of 1 ("not serious”) to 5 (“very serious”). Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 149 FINDINGS The findings, as indicated earlier, have been reasonably consistent across studies. Consequently, we will tend to report the results of our most recent study [2] as it has the greatest generalizability. The approach taken will be to itemize findings within each of several areas of investigation: (1) overall perceived seriousness of the barriers; (2) dimensionality; (3) discipline differences; (4) pedagogy differences; and (5) moderator variables differences. Perceived Seriousness of the Communications Barriers The grand means have displayed a broad range with some interesting patterns apparent upon inspection. Table 1 contains the barrier item means ranked in descending order of importance from our largest data set. The relative positions tend to reveal that concerns about the instructor and differences between the instructor and students are foremost in students’ minds. Student- student relationship problems and student-specific shortcomings tend to occupy the bottom half of the rankings, indicating their lower perceived seriousness. Dimensionality of the Barriers The factor analyses have revealed from thirteen to fifteen factor dimensions. Variance explained by principal components analysis has ranged from 61 to 69 percent. Cronbach’s alpha’s computed for those items loading with a (plus or minus) .45 or higher to assess reliability, has yielded values ranging from .40 to .92. The great majority of these coefficients have been above .60, a frequently mentioned cutoff value for exploratory research. The factor structures have tended to be consistent across studies. However, it has not been until the most recent study that a more satisfactory method could be applied, namely, confirmatory factor analysis, to compare structures between two populations (two universities). The result has been our conclusion that insufficient evidence exists to warrant treating them as separate populations. In other words, the dimensionality appears to persist across universities, in the context of our investigations. Table 2 contains the results of the rotated principal components analysis for the aggregated sample. As can be seen, thirteen dimensions evolved. The factor loadings are reasonably high and the reliability coefficients are acceptable for the most part. The dimensions represent a mix of concerns in the minds of these students, irrespective of business discipline or pedagogy. These concerns range in seriousness as well. For instance, the “Hostility, Distrust, and Personality Conflicts with the Instructor” dimension defines the area of most concern, but others are very close as well. For example, “Either-or Thinking,” “Feedback Problems,” “Students’ Lack of Knowledge and Understanding,” and “Prejudices or Biases” are certainly seen as potential problem areas. In contrast, “Status Differences” and “Use of Profanity” are of much less concern. Differences Across Disciplines Analysis of variance using discipline as the independent variable resulted in only one instance of significance beyond the .10 level. For the dimension, “Lack of Knowledge and Understanding,” post hoc tests revealed that it was perceived as relatively more serious for principles of Accounting or Finance than for principles of Management or Marketing subject matter. Apparently, in the minds of these student- subjects at least, subject matter differences have very little impact on the general difficulties of learning. Differences Across Pedagogies Tests of differences between the five pedagogies have exhibited consistency in some respects and inconsistency in others. The consistent result has been the determination of enough differences at the .10 level to insure to us that the results transcend chance. Also, there is a clear separation of straight lecture from the other four teaching approaches. In general, straight lecture evokes concerns about instructor biases, failure to listen to students, little feedback, and the like, as well as for certain environmental factors such as background noise. The “involving” pedagogies tend to give rise to fears of student interpersonal difficulties, status differences, and conflicting attitudes. The inconsistency across studies has been in our failure to find the same pattern of differences between discipline all the time. A partial explanation of this occurrence stems, we believe, from slight differences in factor structures across studies. Possibly due to sample size limitations and spurious loadings, the dimensions add or delete a few items from study to study. In other words, we have not as yet locked onto the underlying factor structure. Nor do we have perfectly reliable subscales identified yet. Nonetheless, we do believe that our most recent research comes closest in these regards. Moderator Variables Grade point average, major, working status, grade level, and sex have all been analyzed as to possible intervening effects. The persistent finding here has been about one-half of the dimensions exhibiting differences between males and females, Invariably, female students indicate more overall concern than do males. Table 3 presents the results of t-tests for female and male means. Isolated cases of significant differences in other moderator variables suggest that we have yet to tap the proper ones, if any others exist. DISCUSSION To date, the major contribution of our research appears to be in the identification of communication barrier dimensions perceived to exist across pedagogies by business students. Despite slight differences across studies, the dozen or so dimensions tend to be replicated. Our finding as to the stability of the structure across two university populations lends a certain degree of generalizability to the findings. Beyond this revelation, it is quite clear that straight lecture is perceived as a communication environment very different from experiential exercises, computer simulation, or case studies, Another positive outcome of the research is the pervasiveness of these learning barriers across disciplines. The fact that students tend to envision only slight variations from business topic to business topic (at the principles level) greatly expands the applicability of research findings in any one discipline to others. At the same time, there seems to be comparability from university to university. In short, the pedagogy rather than the business discipline or the university environment represents the central concern. Our intended direction presently includes a necessary refinement of the measuring instrument based on our findings and aided by suggestions from colleagues. From here we hope to pursue some variations of the research one of which should include the comparison of Developments in Business Simulation & Experiential Exercises, Volume 10, 1983 150 151 TABLE 2 COMMUNICATIONS BARRIERS DIMENSIONS: ITEMS AND RELIABILITY COEFFICIENTS Barriers Items Factor Loading Explained Variance Sum Score Mean* Cronbach’s Alpha 1. Hostility, Distrust, and Personality Conflicts with Instructor Instructor’s lack of Credibility Personality conflicts between instructor and students Instructor’s hostile attitudes Students’ hostile attitudes Instructors tendency not to listen Lack of trust in instructor .60 .62 .74 .55 .61 .68 20.2% 4.0 .81 2. Student Interpersonal Conflicts Personality conflicts between students Informal social groupings or cliques Lack of trust in other students .77 .66 .63 6.9% 3.1 .67 3. Use of Profanity Instructor’s use of profanity Students’ use of profanity .86 .87 4.6% 2.8 .93 4. “Either-or” Thinking Instructor’s “either-or” thinking Students’ “either-or” thinking .83 .80 4.2% 3.8 .84 5. Noise and Spatial Distraction Students speaking too loudly Poor spatial arrangements Physical distance between sender and receiver Physical noise or distractions .52 .67 .58 .63 3.6% 2.9 .64 6. Emotional Reactions and Defensiveness Students’ defensiveness Instructor’s emotional reaction Students’ emotional reaction .47 .74 .83 3.3% 3.1 .69 7. Status Differences Difference between status or position of instructor and students Difference between students’ status or position .83 .75 3.2% 2.5 .78 8. Student Know-It-Allism and Resistance Students’ know-it-all attitude Students’ resistance to change Students’ lack of credibility .55 ‘56 .56 3.0% 3.3 .45 9. Students’ Lack of Knowledge and Understanding Students’ lack of understanding of technical language Students’ inadequate knowledge of the topic .75 .77 2.6% 3.8 .64 10. Feedback Problems Lack of feedback to instructor Lack of feedback to students Students’ tendency not to listen .70 .66 .48 2.6% 3.9 .67 11. Perceptual Differences Differences in perceptions of students Differences in perception between instructor and students .70 .75 2.5% 3.4 .60 12. Prejudices and Biases Instructors’ prejudices or biases Students’ prejudices or biases .57 .67 2.4% 3.8 .71 13. Group Size Problems Excessive size of a group .70 2.2% 3.6 (Not Applicable) *Sum scores divided by number of items factor for ease of comparison 152 TABLE 3 SUMMARY OF MALE-FEMALE DIFFERENCES IN PERCEIVED SERIOUSNESS OF BARRIERS Sum Score Means Barrier Dimension Males Females Sign.* Level 1. Hostility, Distrust, and Personality Conflict with Instructor 23.6 24.3 .03 2. Student Interpersonal Conflicts 9.1 9..3 .25 3. Use of Profanity 5.2 6.1 .0001 4. “Either-or” Thinking 7.2 7.7 .0005 5. Noise and Spatial Distractions 11.4 11.4 .85 6. Emotional Reactions and Defensiveness 9.2 9.8 .001 7. Status Differences 4.9 5.0 .43 8. Student Know-it-allism and Resistance 9.8 10.0 .47 9. Students Lack of Knowledge and understanding 7.2 7.8 .0001 10. Feedback Problems 11.5 12.0 .01 11. Perceptual Differences 6.7 6.9 .04 12. Prejudices and Biases 7.1 7.4 .10 13. Group Size Problems 3.5 3.7 .11 * Based on t-test for differences between means REFERENCES [1] Buns, Alvin and Steven Golen “Perceived Communications Barriers to Business Education Across Alternative Pedagogies,” American Institute for Decision Sciences, 1981, pp. 158-160. [2] Buns, Alvin, Steven Golen, and James Gentry, “Perceived Communications Barriers to Leaning Subject Matter Across Five Pedagogies, Four Disciplines, and Two Universities,” American Institute for Decision Sciences, 1982 (forthcoming). [3] Buns, Alvin and Steven Golen, “Perceived Communication Barriers to Leaning with Computer Simulation,” under review. [4] Chruden, H. 3. and A. W. Sherman (1968), Personnel Management (Cincinnati: South-Western Publishing Company). [5] Golen, Steven and Alvin Buns, “An Analysis of Students’ Perceptions of Communications Barriers for Alternative Teaching Methods,” American Business Communication Association National Conference, Phoenix, Arizona, October 1981, pp. 117-126. [6] Kulik, 3. A., C. Kulik, and P. A. Cohen, “Effectiveness of Computer-based College Teaching: A Meta-Analysis of Findings,” Journal of Education Research, Vol. 50, Winter 1980, pp. 525-544. [7] Schramm, W., “How Communication Works,” in W. Schramm (ed.) The Process and Effects of Mass Communications (Urbana: University of Illinois Press, 1954). [8] Tubbs, S. L., “An Interpersonal Communication Model,” in L. L. Baker and R. 3. Baker (eds.) Speech Communication Behavior: Perceptions and Principles (Englewood Cliffs: Prentice Hall, 1971). Table of Contents Volume 10, 1983 Is the Computerized Business Simulation Relevant? Business Professionals Play a Student Game A Methodology For Assessing the Internal Validity of Business Simulations A Longitudinal Study of the External Validity of a Business Management Game Policy Analysis and Decision: A corporate Relocation Simulation Exercise Effective Listening: An Exercise in Managerial Communication The Symbol Exercise: An Initial Group Activity Concept Based Simulations Institutional Users of Experiential Learning Packages: A Preliminary View from Publishers' Adoption Lists How to Use Business Games in the Business Policy Course: The Students' Perspective Professors' Ratings of Business Policy Learning Methods Moot Trial: An Exercise in Trial Procedure and Evidence The Advertising Agency Game: An Experiential learning Exercise The Use of Videotaped Cases in Teaching Information Acquisition and Decision-Making Skills Experiencing Information Processing Strategies as a Means to Explore Decision Making Importance Ratings and Operations Data as Predictors of Business Game Performance Determinants of Performance in Computer Simulations Predicting Business Game Performance form Perceptions of Manager Information and actions Business Consulting: A Practicum for Undergraduate Internship as a Contingency Based Experiential Learning Program for More Effective Organizational Socialization: A Conceptual Framework MANSYM III Decision Support System Demonstration Learning the Concept of market Value Through Simulation Conflict Management for Economic Developers Development of Data Analysis Units Designed to Enhance Reasoning and Knowledge Transfer in the College Level Course BOSS: A Behavioral-Quantitative, Computer-Supported Game Entrepreneurial Potential: An Experiential Exercise in Self Analysis and Group Assessment Development of Strategists: Simulated Cases BANKRUPT: A Deceptively Simple Business Strategy Game Simulating Market and Firm Level Demand - A Robust Demand System COMPSIM A Computer Center Management Simulation Hiving Model: Assessing Management Skill Awareness The Johari Window, A Reconceptualization Role-Playing Based on Video-Tape Scenarios: An Application of Modeling to Building Supervisory Skills How to Internationalize Your Curriculum A Computerized Model of Human Behavior in a Total-Firm Management Simulation the Worksheet Approach for Simulation Game Strategy Analysis Teaching Competitive Bidding Using a DSS Generator Do We Learn from Experience? The Use of Theory Power for Increased Research Momentum in Business Simulation and Experiential Exercises Research Report on Programmatic Research on Perceived Learning Barriers with Simulation and Experiential learning An Empirical Examination of Conflict - and Nonconflict - Oriented Problem-Solving Technologies Management Curriculum: 1982 Humor as a Management Tool: Use in Formal Game Presentations Student Behavioral Change Through Teacher behavioral Change