THE EFFECTS OF DIFFERENTIAL GRADE WEIGHTS ON BUSINESS GAME LEARNING LEVELS Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 159 THE EFFECTS OF DIFFERENTIAL GRADE WEIGHTS ON BUSINESS GAME LEARNING LEVELS Joseph Wolfe, University of Tulsa C. Richard Roberts, University of Tulsa ABSTRACT Three separate Business Policy sections played a fairly complex business game under different game performance grade weights. Before/after knowledge tests found that weights had an effect on knowledge levels but in a nonlinear fashion at the extremities of weights tests. Qualitative date substantiated the quantitative data collected. INTRODUCTION Business gaming research has basically followed two paths-- comparative substantive evaluations of simulations, and process-related studies of factors that determine gaming performance {2;8]. The first group compares a gaming application’s learning results with those of an alternative teaching method. The second group examines how various internal game factors influence playing quality. Research like that of Fritzsche [1], Raia [7], and Wolfe and Guth [9] characterizes the first group while the research of Lucas [4], McKenney and Dill [5], and Miesing [6] characterizes the latter. Wolfe’s [8] recent literature review summarized those gaming areas needing greater research. One major inconsistency and possible research flaw has been the amount of course-grade credit given for game performance in both teaching and experimental situations. Of the 23 articles directly studying learning outcomes, half did not even state whether grade credit was given for game performance and the others that made a disclosure demonstrated a grade range of 10% to 85%. In the first case unrevealed grade-weighting makes true replication studies impossible while simultaneously masking any possible stimulus/response relationships existing between the game and its hypothesized learning effects. In the latter case a weight of only 10% may be too nominal to inspire or reward dedicated effort while an 85% weight may be frightenly excessive. An optimal and balancing grade weight may exist between a course’s other teaching components and research should be conducted to find that point. This paper reports a study of the learning effects produced by different grade weights within a set of Identical business policy courses. Research of this type should determine what (if any) effects different grade weights have on student learning and game performance in typical teaching situations. Attendent findings should also illustrate the intrinsic motivational value of games versus the Incentives provided by such typical supportive devices as final game reports, team press conferences, and mandatory instructor counseling. HYPOTHESES It was generally hypothesized that learning levels and playing performances are affected by the amount of final grade credit given for game results. This positive teaching relationship should emanate from a number of sources: 1. The operant conditioning students have received which tells them to respond only to those elements that affect their welfare; 2. The attention focus supplied by the weights the instructor has attached to each course teaching component; and 3. Art equity sense that dictates that grade weight should approximate work effort. Accordingly H1: was stated as: H1: Learning levels increase with grade weight increases. Business policy courses have traditionally been asked to functionally integrate business school coursework and the simulations used in those courses typically require this integration for successful play. Grade weighting could reinforce this requirement by encouraging students to use more than just their functional major for game success. Higher grade weights should heighten integrative knowledge while lower grade weights should increase the variance of within- team functional knowledge. Accordingly the last hypotheses were stated as: H2: Integrative knowledge levels increase with grade weight increases. H3: Within-team variances in functional knowledge decrease with grade weight increases. METHODOLOGY Three separate and statistically-identical (see Table 1) sections of a senior level business policy course played Jensen and Cherrington’s [3] relatively complex top management game for 10 decision rounds under different game performance grade weights. One section’s teams received nominal 10% credit (NOM) for game results as measured by total profits and rates-of-return on assets and equity. Another section received moderate 50% credit (MOD) for the same criteria while a third section received a heavy 85% credit (HEA). All other course and game-playing elements were controlled. Students played on randomly- assigned 3-member teams within 8-company industries beginning the semester’s third week. Students were exposed to a technical game-orientation session before beginning game play and no trial runs were employed. Two class periods were used for instructor counseling and decision preparation while all other class periods were used to discuss 13 assigned cases. Students were expected to participate In the case discussions and they submitted 3 of their 13 cases for grading. Learning was measured by performance on a before/after objective test of game knowledge. Twenty-three ques- Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 160 tions of two types were employed--one type covered decisions about the company’s functional areas in isolation, i.e., Marketing, Finance, Accounting, Production, Economics and Management; the other type required the integration of two functional areas for the correct answer. An after-course attitudinal questionnaire was also administered to better understand the different learning environments possibly associated with each grade weight. RESULTS The first hypothesis that learning increased with grade weights was tested by comparing the proportions of correct responses between groups. For the functional questions differences in proportions of correct answers were found in four cases at the p= .05 level between NOM and HEA with no differences between NOM and MOD. See Table 2. In the NOM/HEA case the hypothesis was accepted for questions 15, 19 and 22 but the learning result operated in an opposite fashion for question 5; a heavy grade-weight had no differential effect for eight other questions. The second hypothesis regarding integrative knowledge was tested in a similar fashion. As shown in Table 3 grade weights had relatively little influence and in the case of question 21 the result was in the opposite direction. The third hypothesis proposed that within-team functional knowledge would be less variable as grade weight increased. This hypothesis was mildly supported as the MOD teams showed a significantly lower variance than the NOM teams on four of eleven questions. This result was not as striking when the variances of NOM were compared with HEA where the latter had a lower variance on only two of ten calculable questions. As shown in Table 4 a U-type function appears to exist across the mean variance scores by grade weight. Shown another way, Table 5 reveals that variations in aggregated responses significantly exceed those of the NOD treatment group on sets of both functional and integrative questions as well as for the total set of questions. DISCUSSION For experimental purposes, extreme grade-weight positions were employed in this study. Qualitative data was also collected and the responses to questions 1 and 2 in Figure 1 show that game/case grade tradeoffs negatively affected the students especially with the NOM group. Despite the grade distortions employed, both course components were favorably received. All groups felt that their learning levels were equal even though in reality the (1) HEA group outscored the NOM group on five questions, and (2) within- team knowledge variances were quite high for NOM teams. This result should cast further doubt on the use of perceptual data to evaluate gaming’s learning benefits. Students’ feelings were valid as shown in questions 1 and 2 but these feelings did not reflect true learning outcomes. Overall, a basically moderate view of game-weighting emerges from this study. Once above a relatively low threshold level, grade weights do not appear to be more motivational than the intrinsic qualities of a good simulation. An amount of grade weight, however, should be given in both teaching and experimental situations. This should be done, if for no other reason than the equities of the situation. Given 3-member teams, this study’s students spent from 110 to 160 collective hours on the game, and the instructor used 4 of 30 class periods for game-related activities and these allocations should be recognized in any fair grading/reward system. Based on Figure l's qualitative data an approximate 45%/55% grade weight split should be employed between the game/cases. Although game-performance outcomes were not this studys Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 161 objective, Mann-Whitney U-tests of ranked earnings reinforced the moderate view shown here. While all teams performed well both financially and test-wise, the MOD group outperformed the others as shown in Table 6. Areas for further study should include tests for business policy concepts and skills, and scoring procedures for decision-making tests. The instrument used in this study tested only game-related technical knowledge and not the acquisition of the broader objectives of the business policy course. Research should also be conducted to determine if technical game master’. translates into strategic management knowledge. Another issue revolves around the scoring procedures used in this study. All analyses were performed on the proportions of correct answers and a blank or its equivalent, “I don’t know,” response was just as negative as a wrong answer. It could be argued that in strategic decision-making a wrong answer is better than no answer at all. If such Is the case this study’s scoring procedures were partially incorrect. CONCLUSION One should conclude from this study that business games are relatively robust teaching instrumental. Large grade weights were not required to motivate learning and only minimal weights were needed to bring grading equity to the situation. In this study extremely high or low grade weights were often associated with different functional learning levels and with within-team knowledge variances; extreme grade weighting did not substantially differentiate integrative knowledge. Accordingly, game grade weights should be kept in balance with a course’s other teaching devices while distorted behaviors and learning outcomes arrive only at extreme levels not normally employed in most teaching applications. REFERENCES [l] Fritzsche, David J .,“The Lecture vs. the Game,” in James Kenderdine, and Bernard Keys (editors), Simulation, Games and Experiential Learning Techniques: On the Road to a New Frontier, 1974, pp. 41-46. [2] Greenlaw, Paul S. and F. Paul Wyman, “The Teaching Effectiveness of Games in Collegiate Business Courses,” Simulation & Games, Vol. 4, No. 3, 1973, Developments in Business Simulation & Experiential Exercises, Volume 12, 1985 162 pp. 259-294. [3] Jensen, Ronald L. and David J. Cherrington, The Business Management Laboratory (Dallas; Business Publications, 1977). [4] Lucas, Henry C., Jr., “Performance in a Complex Management Game,” Simulation & Games, Vol. 10, No. 1, 1979, pp. 61-74. [5] McKenney, James L. and William R. Dill, “Influences on Learning in Simulation Games,” The American Behavioral Scientist, Vol. 10, No. 3, 1966, pp. 28-32. [6] Meising, Paul, “Qualitative Determinants of Team Performance in a Simulation Game,” in David J. Fritzsche and Lee A. Graf (editors), Developments in Business Simulation & Experiential Exercises, 1982, pp. 228-231. [7] Raia, Anthony P. , “A Study of the Educational Value of Management Games,” Journal of Business, Vol. 39, No. 3, 1966, pp. 339-352. [8] Wolfe, Joseph, “The Teaching Effectiveness of Games in Collegiate Business Courses; A Ten-Year Update,” Simulation & Games, Vol. 16, 1985 (in press) [9] Wolfe, Joseph, and Gary R. Guth, “The Case Approach vs. Gaming in the Teaching of Business Policy; An Experimental Evaluation,” Journal of Business, Vol. 48, No. 3, 1975, pp. 349-364. Table of Contents Volume 12, 1985 One Step Beyond Naiveté: Laboratory Simulation of Strategic Management Predicting Performance Over the Course of the Simulation Congruency of Critical Performance Factors in Simulated and Real World Environments: An Exploratory Study Stimulating Learning and Creating Course Involvement Through the Use of Debates A Novel Approach to Writing Texts ABSEL Revisited: In-Class Student Involvement Increases Learning Bus-Sim Micro: An Introductory Business Simulation Demonstration Solutions Diversified: And Experiential Introduction to Management Information Systems Simulating Rigorous Analysis in Stimulation Gaming The Use of Decision Support Systems (DSS) and Operations Research/Management Science (OR/MS) Techniques to Enhance the Learning Experience of Students Participating in Computerized Simulation Problems Associated with the Administrative Use of Student Ratings Enthusiasm Awareness in the Experiential Classroom Participation Expectations of Students in Experiential Settings Using the Computer as a Planning Aid in an Applied Marketing Course Integrating Microcomputers in the Marketing Curriculum Through the Use of Marketing COMPUPROBS Giving Students Experience in Team Report Writing Description and Results of an Experiential Approach in the Teaching of Business Policy Using Real World Experiences as an Experiential Activity: the Page Technique A Guide to Writing Microcomputer Simulations The Development of a Microcomputer Distributed Processing Business Game A Monte-Carlo Approach to Interactive Gaming Single Case Experimental Design as an Alternative to the Conventional Approach Learning Theory and Research Design: How Has ABSEL Fared? Criticisms of the Use of Simulations in Economics: A Rebuttal Using the General Management Game in the Business Policy Course General Management Policy Simulations: Which Games are Popular, and Why do Professors Stop Using Games? Business Policy Content and Teaching Methods: Comparisons of the Hegarty and Summers and Boyd Findings Let's Talk to the Experiential Authors A Computerized Logistics Game for Micros Effects of Contextual Factors on Bargaining Decisions of Experiential Group Representatives The Linking of a Collective Bargaining Simulation and a Business Simulation Game - A Case Study Students are Surrogates in Arbitration Studies AIRWAYS: A Microcomputer Simulation of a Service Industry A Proposed Interactive Inventory Control Simulation Microcomputer Demonstration Exercises for a Course on Creativity and Problem Solving Processing Skills for Teachers, Trainers and Managers Developing the Competencies of Resistance to Stress and Accurate Self-Assessment Raising Elizabeth: Socializing Girls to Aspire to Careers as Managers, Secretaries or Corporate Wives Factors Influencing Perception: Gaining Understanding Through Experience Painless Computer Simulation The Effects of Differential Grade Weights on Business Game Learning levels An Investigation of the Validity of a Recommendation for Experiential Exercise Debriefing Simulation and Experiential Practices of Faculty: A Data-Based Workshop The Live Case Study: Filling the Gap Between the Case Study and the Experiential Exercise A Reactive Microcase Pedagogy Advantages of a Multigame Simulation Course Using the Logistic Function to Score Computer-Supported Games A Simulation Model to Calculate Session Time for Running Simulation Models in a Shared Resource Environment On the Core Size of Small Task Groups and the Cause for the Core Size to Emerge