COOPERATIVE LEARNING: WHAT ARE WE LEARNING? Developments In Business Simulation & Experiential Learning, Volume 24, 1997 COOPERATIVE LEARNING: WHAT ARE WE LEARNING? Peter M. Markulis, SUNY College at Geneseo Daniel R. Strang, SUNY College at Geneseo ABSTRACT The purpose of this paper is twofold: First, it present the findings of a study the authors conducted on cooperative learning in 1994: and, second, it discusses the strengths and weaknesses of using an ethnographic approach in conducting educational research (as opposed to a more traditional or experimental approach) in terms of the authors own experience. While the authors found that cooperative learning was beneficial both in terms of student learning and behavioral skills, they were unable to “prove” these findings using traditional’ research design methods and statistical techniques. Readers should find the paper useful both as forum for discussing traditional versus ethnographic research designs. as well as a venue for discussing the benefits and liabilities of using cooperative learning for management students. INTRODUCTION For a number years, many authors have been extolling the benefits of cooperative learning as a mechanism to enhance the learning undergraduates students (Beckman. 1990: Coffin. 1992; Cottell. 1991, Hiltz, 1990; Spare. 1991). A study conducted and reported recently (Markulis & Strang, 1994) employed an ethnographic approach and promoted this approach as an alternative research technique n contrast to the more traditional controlled experiment in which format hypotheses are tested and inferences are made. The research subjects in this case were undergraduate business students. Markulis and Strang report that cooperative learning appeared to have a positive overall impact on learning and socialization. One is left to wonder if it would have been possible to make more definitive pronouncements if the more traditional experimental techniques had been employed. This paper reports additional results obtained in the 1994 study and the various problems which arose in attempting to conduct a more traditional controlled experiment for the same research sample. Since the problems that the authors encountered are probably not institution-specific or discipline-specific, one is left to wonder if it is possible to design experiments that will give us the proof of the value of cooperative learning--or, any other kind of learning, for that matter. An article recently reported in The Chronicle of Higher Education (Shea. 1996) recognizes the existence of a debate over the appropriateness of using of the significance test in psychological’ research focusing on learning. For the devout statisticians, the suggestion that tests of significance may not be appropriate and, in fact, often may lead to false conclusions is the ultimate blasphemy. It does return researchers to the question of how to demonstrate which pedagogical techniques are superior. Although the authors had hoped to demonstrate the value of cooperative learning as a teaching technique. The real value of this research may be in the problems that the authors encountered in their attempts to perform traditional research. The models cited by Campbell and Stanley are often referred to when researchers attempt to design a traditional controlled experiment for the study (Campbell and Stanley, 1963). Their model includes factors such as 1) randomization: (2) experimental and control groups; (3) tight control over the experimental conditions; etc. Clearly these factors are desirable when it is possible to achieve them, in this research the authors encountered several hurdles. For example, it was discovered that the college policy covering human subjects did not allow experiments to be conducted in required courses because students had no choice in opting Out of an experiment. Thus is was not possible to apply one treatment to one section of a course and another treatment to another section because it would be construed that what was being done in one or both of this sections was being done for research purposes and conceivably would somehow diminish the quality of instruction. PROJECT BACKGROUND The Group. The subjects chosen for the study were 74 undergraduates taking an introductory course in microeconomics. The breakdown of the class was as follows: 6.8% Freshmen 74.3% Sophomores 12.2% Juniors 6.8% Seniors This class was chosen for two reasons: First, the instructor teaching the course had been using teams for group work for at least three years. However, student feedback suggested to the instructor that the team format was perceived as having little positive impact on learning. Second, since an overwhelming majority were freshmen and sophomores, most of these students would have had little previous exposure to team or group work in a college setting. 98 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 The Cooperative Situation: During the second week of the semester, a faculty member not teaching the course visited the crass and laid out the cooperative assignment for the semester. The students were given a lecture on what was meant by cooperative learning, why cooperation was important and the general benefits of cooperation. The lecture emphasized that today’s work force would require them to form teams in a quick fashion and that cooperative efforts would be expected by their employers. They were told that by participating in this cooperative learning project they would gain valuable knowledge and experience, which would serve as the foundation for team situations that, they would surely encounter in their futures. This theme was graphically reinforced through a college-wide lecture given by Dennis Basset, Vice- President of Human Resources at Bausch & Lomb, Inc. Mr. Basset told the audience that corporate recruiting officers were looking more and more for colleges which had given their students exposure to cooperative learning and diversity. Those institutions which had not provided this exposure would not be visited by recruiting officers. Basically, cooperation for their project was defined as follows: Each team worked on a computerized tutorial package as a team. Each team member was assigned to review a particular module’s subject matter and was told to be prepared to 'teach’ the other students the concepts relating to that module. The students took a series of 10 quizzes, which were administered and recorded by computer. The students were told that the cooperative learning project would be factored into their final grade and weighted at 25 percent. The students were then divided up randomly into 23 three-person teams and one two-person team. In the second intervention, students were reminded about the nature and purpose of the project and feedback was given regarding their journals. Teams not actively involved in cooperative learning were encouraged to do so. Some time was allocated to problem resolution and addressed 'typical' kinds of team problems as well possible solutions. The reader should be aware that more exacting rules and procedures could have been established for the cooperative project. but the literature (Cohen, 1993), as well as the consultants from the education department believed that the project should be as realistic as possible and as a consequence formal interventions should be carefully calibrated and minimized. A project that was too artificially contrived or subject to excessive intervention might adversely affect the ability of the researchers attempt to understand and appreciate the dynamics of cooperative learning. The Microeconomics Package. The students were given the microeconomics package (Economics in Action, by McTaggart et al., 1992) and told that the package would not only help them to understand microeconomics better, but that it would be used to help as the medium for cooperative effort during the semester. Each student was given access to his/her own version. The package was fully explained, along with the protocols for taking quizzes, maintaining logs, and keeping journals. The class was told that the ten-module package would follow or complement the topics given in the class lectures. The “Interventions. “ There were three so-called cooperative ‘interventions’ during the course (see TIME LINE in EXHIBIT A). Each of the three interventions was conducted during formal class time by a faculty member, but not the course instructor. Ethnographic research has demonstrated that interventions can have a positive effect on students participation in cooperative learning (Campbell, 1992). The first intervention occurred during the third week of the semester. A guest speaker (a faculty member involved in the research project) gave a formal presentation and discussed cooperative learning in a general way. Details on keeping journals were also provided at this time. The third intervention took place after several teams had been interviewed. This session was used to help resolve various kinds of team problems. DATA COLLECTION Given the nature of the study, a multifaceted approach was 99 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 Objective team measures were obtained by taking means, etc. for the members of the teams, thus the mean team GPA, the results on for each test, and for the course (quality points. were compiled and available for analysis. Table 2 shows the measures for each team. used to collect data. A number of objective measures were obtained for each student on each of the 23 teams. These measures included the follows: 1. Grade point average (GPA) prior to taking this course 2. Grades for each of three preliminary tests 3. Grade for final 4. Course grade Table shows GPA and Course grades for each individual on each team. Note: an * is used to represent missing values. 100 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 In addition, a number of more subjective measures were recorded during the semester. These measures included persona’ journal’s, peer reviews and interviews with student teams. Each of these measures has been described thoroughly in an earlier paper (Markulis & Strang. 1996). Suffice it is to say, that the personal journals, peer reviews and student interviews were used to help develop thc degree to which the teams cooperated (internally) during the semester and to help derive various themes which the authors found from reading the journal’s, and evaluating the peer evaluations and interview data. From the rich and multifaceted data source, the authors derived three definable types of cooperation (or team interaction). Type A = individual workers Type B = meeting oriented Type C = cooperative oriented The type of cooperation observed for each team was recorded at five different times during the semester and encoded for analysis. This data is presented in Table 3. Where A = Type A Cooperation B = Type B Cooperation C = Type C Cooperation NM = Not Measured APPENDIX A presents more detailed operational definitions for the required and derived themes. (A sampling of some of the major themes/issues is described below. MEETING LENGTH. The average meeting time for the first 5 modules was a hour and for the second 5 modules, it dropped to approximately 45 minutes. SCHEDULING. Scheduling was a problem for fewer teams than was anticipated. Unfortunately, for some teams, it seemed to remain a problem for the duration of the project, despite the fact that one of the authors worked with these teams trying to help them alleviate scheduling problems. TYPE OF COOPERATION. Table 4 shows a breakdown of cooperation type over the course of the 10 modules. As can be seen, for most modules, cooperation seemed to steadily gravitate toward Type C (cooperative oriented), but one might say, not dramatically so. Also, there seemed to be a tendency to "slack- off" during the last module. FREE RIDER. Free riders were a problem for several’ teams throughout the project. There were no Type C teams with free riders. The presence of free riders seemed to have a deleterious effect on both the functional as well as interpersonal’ aspects of teams. This was particularly true toward the end of the project. STUDENTS SATISFIED UNDERSTOOD CONCEPTS. For most of the teams and for most of the modules, students reported that carrying out the modules was a valuable learning experience, i.e., working through the modules in a team fashion helped in their understanding of various economic concepts. This was true much more for teams where there were good communications, no free riders, no dictators, and where the comfort level was high. It also was found to be higher for teams utilizing either Type B or C cooperative learning. 101 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 102 ANALYSIS & CRITICAL DISCUSSION Clearly the extensive objective and subjective data collected offer the researcher a rich sample pool for analysis. Traditional research methods call for a formal statement of hypotheses. In this case, the primary hypothesis would be that cooperative learning resulted in more (or better) learning (as measured by higher test scores) after controlling for factors such as academic ability (represented by prior GPA). The scalar and ordinal data available were analyzed using various parametric and non-parametric statistics. Without laboriously reiterating all the tests performed and the results, suffice it is to say that no statistically significant results arose which indicated the efficacy of cooperative learning as a pedagogical tool to improve learning of undergraduate business students. Does this literally mean that cooperative learning should be dismissed as a pedagogical tool? Since the authors have a great deal of faith in the value of cooperative learning, they are not prepared to pronounce to the world that cooperative learning is without value. It may be instructive to ask: Why did we get the results we got? It may also be appropriate to consider how likely it is that other similar research efforts might lead to the same results. Lets first consider the database used. Although the data were extensive and meticulously gathered, they suffered from the typical problems researchers encounter in trying to obtain precise measures of some rather imprecise factors. For example, although elaborate measures were employed, one is left to wonder how accurately one can measure the level of cooperation for a team of three people. Can this cooperation be reduced to an qualitative variable with three levees? Can one use the mean test score for three individuals as a measure of the learning for that team. (This, of course. ignores the bigger question--do test scores measure learning.) Are the measures that were used to crude and insensitive for the purposes of the study? Could better measures have been utilized? Although the authors readily concede some of the problems attendant to the data, they are at a loss to envision better measures. Perhaps, this represents a research opportunity for future researchers: or, perhaps we are simply “barking up the wrong tree.’ It was clear that students perceived the cooperative learning project in a positive light. However, there are several caveats to this statement. Students perceived the experience in a positive light whether they used Type B or Type C cooperative learning, while students employing Type A cooperation had the least positive things to say about the experience. Clearly, a type of Hawthorne Effect was operative throughout the project, meaning that the high and steady interest of the faculty in the process had a positive impact upon the students’ interest and involvement in the project. Those that employed Type B. were mostly scheduling type things and those that moved from Type C to Type B. were of the same type, while those of Type C seemed to either have few technical or other problems or were seriously working on them. There were some differences between those teams which used Type A Cooperation and those teams which used Type B or C Cooperation, although these differences were not statistically significant. Indeed, as stated earlier, most of our finding were discovered through the use of ethnographic techniques, and not via the traditional experimental methodology. For example, personal journals and interviews suggest that Type A Cooperative teams seemed to have difficulties in communicating, with comfortableness, but not as much with satisfaction with understanding the concepts. Type A teams had more trouble with scheduling, free riders and ‘dictators” than Type B or C teams, and these problems seemed to persist which suggests that team specific intervention strategy be employed by the instructors in the future. If there was a direction of causality between a teams type cooperation and the incidence of problems, the direction Developments In Business Simulation & Experiential Learning, Volume 24, 1997 103 was not obvious. Specifically, it is possible that a Type A orientation may have served the source of team problems, and it is also possible that the presence of team problems led to the evolution of a Type A cooperative mode. It was Interesting to note that students cooperated when told do so, and that many of them seemed pleasantly surprised at how well cooperation worked, but what was disappointing was that many teams which used Type A cooperation were not persuaded to move toward Type B or C, despite attempts to encourage Type C Cooperation. CONCLUSION This paper has attempted to do two things: First, to explore and examine the usefulness of cooperative learning as a teaching medium; and secondly, to raise questions about the use of traditional research designs vis a vis ethnographic designs in studying the efficacy of simulations and experiential learning. The nature of the conclusions that arise from an ethnographic analysis are -- or, at least. Seem--different from those that come from more traditional, controlled, research models in which hypotheses are tested and inferences are made. The ethnographic approach is clearly more subjective in nature. While this may leave some readers less comfortable than the usual pronouncements arising from traditional research methods, there is much weight, especially in the contemporary educational research literature that argues for that applicability and appropriateness of the ethnographic approach. Having said that, this study does not definitely conclude that cooperative learning is superior in enhancing both Learning and/or socialization than more traditional pedagogies. and it does, in general support the findings from previous studies which conclude that cooperative does improve learning and socialization. Finally, the grid, scoring mechanisms, working definitions of cooperative learning and various tables and modes of analysis which were developed as a result of this research may be useful in future efforts to carry this research further. APPENDIX A OPERATIONAL DEFINITIONS TYPE OF COOPERATION. Type A — individual oriented simply meet together, Little preparation may take quiz together, no concern for whether all members understand concepts. Type B -- meeting oriented, that is members wait until the meeting to discuss concepts and/or take quiz. No one person preparing for the meeting. Type C -- cooperative oriented, whereby one person is assigned to learn and teach (or help clarify) the topic to other team members. Quiz is taken together and there is an effort to make sure all understand the concepts. REFERENCES Beckman, Mary, “Collaborative Learning: Preparation for the Workplace and Democracy?’ College Teaching, 38 (4), 1990, pp. 128-134. Brown, A.D., “The Pros And Cons of Experientially Based Projects,’ Journal of Management Education, 17(4), Nov. 1993, pp. 498-504. Campbell, Cheri Ellis, “Doin Time in College: An Ethnographic Study of Power and Motivation in a Large Lecture Class,” Paper presented at the 78th Annual Meeting of the Speech Communication Association, Chicago, 11., Oct 29-Nov I, 1992. Campbell, Donald T. and Julian C. Stanley, Experimental and Quasi-Experimental Designs for Research, Chicago, Ill.: Rand McNally and Company, 1963. Coffin, Donald, “An Application of Collaborative Learning in a Course on Urban Economics,’ in Collaborative Learning; Sourcebook for Collaborative Learning in the Arts and Sciences at Indiana University, ed by Sharon J. Hamilton and Edmund Hansen Bloomington, IN: Indiana University, 1992, pp. 10-20. Cohen, Myna, L., “Listening to Students Voices: What University Students Tell Us about How They Learn, Paper presented at the Annual Meeting of the American Educational Research Association (Atlanta, GA, April 12-16, 1993) Cottell, Philip G., “Classroom Research in Accounting: Assessing for Learning,” New Directions for Teaching & Learning, #46, Summer 1991, pp. 43-54. Johnson, David, W. Roger T. Johnson, Jeanette Johnson and Douglas Anderson, Effects of Cooperative Versus Individualized Instruction on Student Prosocial Behavior, Attitudes Toward Learning, and Achievement,@ Journal of Educational Psychology 68 (4), 1976, p. 447. Markulis, P.M. & Daniel R. Strang, An Ethnographic Study of Cooperative Learning @ ABSEL Proceedings 1996, ABSEL Conference, Orlando, Shea, Christopher, “Psychologists Debate the Accuracy of Significance Test.’ The Chronicle of Higher Education, Aug. 1996, pp. A12 & A17. Slavin, Robert, E, Cooperative Learning, New York, New York: Longman Inc., 1983. Table of Contents Volume 24, 1997 Incorporating Computer Telephony into the MIS Course A Learner Oriented Infrastructure for videoconferencing Based Distance Education Courses The Identification of Temporally Related Structural Elements of the Experiential Component of Electronic Spreadsheet Tasks Does Involvement Influence Learning from Simulation Participation? Some Relationships with Helpfulness and Performance Outcomes Musings on Business Garne Performance Evaluation Performance on a TE Simulation: What does it represent? The Business Policy Game: An International Simulation - An Assessment Tool Students in Free Enterprise as Experiential Learning The Role of Computer Models in Wargames: A Practitioner's view Using Focus Groups as a Tool to Research Learning: A Demonstration and a Discussion Using Medical Simulations to Teach Multi-Cultural Diversity Using the World Game to Internationalize International Exchange Game Community Services Needs Assessment: An Innovative Approach Introducing Students to Potential Total Quality Management Ethical Dilemmas Privacy in the Workplace: A Situational Analysis Antecedents of Learning in Simulations Demonstrating the Learning Effectiveness of Simulation: Where we are and Where we need to go The Use of Computer Simulations as a Pedagogical Aid in Teaching Management Information Systems Evaluating Simulation Learning in a Distant Learning Instructional Model Financial Engineering of Global Investments The Labor History Game: Playing with the Past Perks Participation Assessing Negotiator's Proficiency with a Negotiation role-play The Art of Negotiating Enhancing Learning and Employee Development through the Assessment of Learning Pedagogy Preferences across selected Dimensions of Culture: A Preliminary Investigation Cooperative Learning: What are we Learning? Effective Use of Mastery Based Experiential learning in a Project Course to improve skills in system analysis and Design Simulations and Learning: Can we prove a relationship? (Seminar) Predicting and Reviewing NYSE Stock Prices by use of Basic Statistical Analysis and Logic Toy Car Depreciation Exercise ABSEL as Home Community: An Interactive Exploration Service Learning: Linking Academic Study to Community Development and Business Enhancement Ability of Efficient Evaluation of Knowledge-Based Management Strategies Corporate Ethics Training Programs Modeling Attributes in Demand Functions of Computerized Business Simulations: An Extension of Teach's Gravity Flow Algorithm Expert Systems Combined with Neutral Networks: Tools to Benefit the Marketing Researcher Computer Game Design: New Directions for Intercultural Simulation Game Designers Consistency in Simulation Performance over Time and Across Simulation Games The Impact of an Artificial Market Leader on Simulation Competitor's Strategies Business Plans, Case Studies, and Total Enterprise Simulations: A Natural Co-existence An Exploration into the Non-Use of Business Simulations The Market Game: Interactive Learning Through Market Simulation Plotting Brand Trajectories with the COMPLETE PPM package: A Market Segmentation Analysis and Positioning Tool Me and Mine Inc: An Exercise in Management Theory Learning to Differentiate Leadership from Managerial Position Business Ethics Survey: A Perspective from the Retail Industry Experiential Learning in Demand Analysis for an Agricultural Commodity Marketing on the Internet: A Pedagogical Exercise College Students Need Simple Computer Simulations, Especially for International Business Courses An Analysis of Student Attitudes, Performance, and Strategies in a Simulation Competition Based on a Controlled Product-Market-Entry Game Structure The Crystal Enterprise: Application of a Non-Computerized Simulation Model in a Process of Organizational Change Threshold: A Windows-Based Behaviorally Oriented Total Enterprise Simulation Coaching Business Game Teams Using a Decision Variable Optimizer Rock and Roll is here to Stay: Enhancing Experiential Pedagogy with Musical Exercises An Example of Business Process Analysis Simulation for Customer Software Support The Use of Business Gaming in Hong Kong Academic Institutions Using the Integrative International Simulation INTOPIA Mark 2000 in a Concentrated MBA Curriculum A Group Experience as an Integrated Part of the Core Management Course Empowered Learning in the Classroom The Inter-Group Interaction: An Innovative Approach to Cooperative Learning Courses that Utilize Student Team: An Approach to enhancing their Effectiveness Thoughts about the Measurement of Learning: The Case for Guided Learning and Associated Measurement Issues Measuring Student Learning Using Business Simulations: A Theory Based Perspective On the Use of PC Fingame in an Undergraduate Finance Course How Managers get Things Done: A Virtual Soundbite Internet Experiential learning in the Principals of Marketing Classroom: A Pedagogical Approach Current Student Perceptions Relative to Business Simulations Beyond Capitalism: Designing Business Policy and Social Justice The Use of Boards of Directors to Evaluate Reports and Presentations in an Undergraduate Business Policy Course Beacon Lumber: An Experiential Introduction to Financial Accounting The Application of Organizational Motivation Principles: The Experiential Business Simulation Motus Manufacturing Avoiding a Bogey: Grading Case Discussions Scientifically Communicating Consumer Behavior II: A Modified Exercise Using Personal Consumption Journals in Condensed Courses Designing Instruments for Assessing the Effectiveness of Simulations, (Seminar) 360º Performance Feedback: Appraisal vs. Assessment The Cafeteria Approach to Managing an Academic Career: Remaining Non-Perishable while Doing Your Own Thing An Experiential Exercise Related to Person-Organization Fit and it's Consequences For Today's Dynamic and Changing World, Business Simulations need to be expanded to encompass Much Greater Complexity: A Demonstration The Incident Process: A Case in Reverse Contextually-Anchored Business Simulations The Energy Factor: Building Motivation in the Simulation Gaming Environment The Design of an Internet Game