AN EXPLORATORY STUDY OF THE EFFECT OF STRATEGIC EMPHASIS IN MANAGEMENT GAMES ON ATTITUDES, INTEREST, AND LEARNING IN THE BUSINESS POLICY COURSE Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 178 AN EXPLORATORY STUDY OF THE EFFECT OF STRATEGIC EMPHASIS IN MANAGEMENT GAMES ON ATTITUDES, INTEREST, AND LEARNING IN THE BUSINESS POLICY COURSE Robert E. Schellenberger, East Carolina University Joseph A. Hill, East Carolina University R. B. Keusch, East Carolina University ABSTRACT This study compared the effect of a strategic emphasis versus an operational emphasis on learning and attitudes among players in four classes of a Business Policy course. The results show that game performance as measured by return on equity (ROE) was higher for students playing a game with a strategic emphasis than students playing a game with an operational emphasis. These findings are clouded by the fact that the student group with the strategic emphasis contained proportionately more accounting and finance majors. BACKGROUND During the last ten years or so, there has been a change in the objectives of the Business Policy Course. In 1978 Schellenberger (1978) identified the key educational goals for the policy text as: 1) an understanding the concepts of strategy, 2) the development of problem solving skills, 3) development of the administrative point of view, and 4) an understanding of the interrelatedness of the functions of business. In 1986 Pearce (1986) focuses on “a comprehensive model of the strategic management process”. Pearce indicates that this approach will provide “an executive Level perspective”. A glance at almost any sample of policy textbooks in 1975 versus 1988 would show a significant increase in the amount of text material devoted to strategic management. Many business policy games were developed prior to the change in emphasis in the policy course. The objective of this research was to investigate whether the level of strategic emphasis in the game would have any impact on attitudes and learning. This issue seems sufficiently important to have been addressed prior to the change in emphasis in the business policy course. With the change in emphasis the issue seems even more important. A review of the literature (especially Greenlaw (1973) and Wolf (1986)) does not reveal any studies which focus on the impact of the level of strategic decision making on attitudes and learning. Wolf (1978) focused on the influence of game complexity. However, the level of complexity is not a proxy for the level of strategic decision making. This research will focus on the impact of the level of strategic decision making on student attitudes and learning. Management games are designed with differing emphasis on strategic decision making as opposed to operational and administrative decision making. A game that does not permit firms to occupy alternative market niches fails to give teams the fundamental option of seeking a particular market niche. Since every game operates over some time horizon, many of the decisions are short run operational decisions. The longer the time horizon impacted by the decision and the more resources committed, the more strategic the decision. The least strategic decisions are the decisions regarding the quantity to produce and the quantity of raw materials to purchase for the immediate time period. For the purposes of this research we looked for either one game that permits differing levels of emphasis on strategic, administrative, and operational decisions or two games so that each would represent different ends of the spectrum relative to the level of decisions built into the game. The advantage of two games is that the emphasis would clearly be different. The disadvantage is that differences in attitudes, interest and learning could be attributed either to differences in strategic emphasis or to the particular nature of the two games. MANSYM (Schellenberger (1986)) was used both because it permitted varying levels of strategic emphasis and because it was a familiar product at the school where the study was being conducted. EXPERIMENT Four classes of the Business Policy course at a midatlantic comprehensive regional university were involved in this research. Two of these classes were taught by instructor A and two were taught by instructor B. Each of the four classes played a management game that represented 25 to 30 percent of the course grade. Decisions covered a simulated three year period of time. Quarterly decisions were submitted weekly. All four classes played the two- product Small Kitchen Electrical Appliance version of MANSYM (Schellenberger (1986)). However, two classes were given a computer program that generated the eight short-term production decisions -- these classes were considered to have a strategic emphasis. The remaining two classes were required to make the short-term production decisions by what ever means they could -- these classes were considered to have an operational emphasis. In addition the two classes with the strategic emphasis were required to submit computer-generated proforma statements. The computer program to generate proforma statements is available to all students but its use was required by the two classes with the strategic emphasis. The requirement to use the proforma option served not only to provide a more strategic emphasis but also to equalize the work load between the two different emphases. Each instructor taught one class with a strategic emphasis and one class with an operational emphasis. Thus the classes using the computer package to make the short term production decision and the proforma computer package are labeled “strategic emphasis” and the remaining classes labeled “operational emphasis”. While these labels may overstate the differences they clearly distinguish between the two groups. Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 179 DATA GATHERED If differences are found it is necessary to ensure that these differences are attributable to the experimental variable (i.e. strategic emphasis). Thus sex, age, grade point average and marital status were found. In addition, the Ghiselli (1971) Self Description Inventory and Minor (1976) Sentence Completion Scale were administered to ensure comparability of the experimental and control groups. As a final check on comparability, data on hours devoted to the game and to other aspects of the course were gathered. Performance data included overall grade in the course, grade on the game , grade on analysis of cases, and return on equity (ROE). These indices were considered one measure of learning. A second measure of learning was students’ perceptions of the fulfillment of the eight objectives of the course. The last measure of learning related to the students ability to classify the various elements of game decisions by strategic, administrative, and operational levels. It is believed that a management game should reinforce these concepts. An attitudinal questionnaire was administered at the end of the semester. This questionnaire was taken from the Raia (1966) and Boseman (1974) studies. Attitudinal data included attitudes toward the instructor and the course in the belief that more favorable attitudes enhance learning. in addition, student perceptions of learning, interest, and motivation from the class were gathered. The attitudinal questionnaire was administered during the last week of the semester. This questionnaire was collected by a student from the class and retained by that student until after the final grades were posted. In this manner students were assured that they would not be penalized by candid responses. Total sample size was 113 students: 56 with the strategic emphasis and 57 with the operational emphasis. HYPOTHESIS The hypotheses center around the performance, learning and attitudinal differences between the strategically oriented game classes and the operationally-oriented classes. A review of the literature does not suggest what differences might be found with a strategic emphasis versus an operational difference. The Raia (1966) study does focus on differences related to the level of game complexity. The authors do not feel that the level of complexity will differ between the two groups. Moreover, the Raia study found no significant differences in the attitudes of students related to game complexity. The most likely differences would occur in performance variables. However, these researchers have no basis on which to hypothesize where such differences, if they exist, will occur. Since this is exploratory research, it is reasonable to start with the hypothesis that no differences will be found. The Mann-Whitney U test was used because the assumption of a normal distribution could not be made for the data. The .05 level of significance was used. RESULTS Attitudinal Results It was hypothesized that student attitudes would be similar for both groups of students (i.e. no statistically different attitudes toward the course or instructor). Table 1 below shows these results. TABLE 1 ATTITUDES TOWARD COURSE AND INSTRUCTOR (7 point scale) Strate- Opera- Signif- Item gic tional icance Importance of your instructor 4.56 4.44 .98 Importance of course 4.59 4.65 .82 Enjoyment of your instructor 4.54 4.28 .29 Enjoyment of course 4.37 4.24 .73 These attitudes are similar. Perceptions of Learning, Interest, and Motivation It was hypothesized that student perceptions of learning, interest, and motivation toward the policy course would be similar for both groups of students (i.e. no statistical difference). Table 2 below shows these results. TABLE 2 PERCEPTION OF LEARNING, INTEREST, AND MOTIVATION IN POLICY COURSE COMPARED TO OTHER COURSES (7 point scale) Strate- Opera- Signif Item gic tional icance Your interest in course 3.79 3.85 .94 Your motivation in course 8.55 3.60 .68 Your learning in course 4.01 4.05 .90 Perceptions of learning, interest, and motivation toward the policy course are indeed similar. Results Relative to Learning This study has gathered three different sets of data on learning. These three sets f data are shown in tables 3, 4, and 5 respectively. The first set of data looks at learning as measured by performance in various aspects of the course. Two measures of performance in the game were gathered. One is the computer-generated grade for game play. This grade is made up from nine criteria. Two of these criteria (representing 50% of the weight) refer to profit measures. The remaining seven criteria (representing 50% of the weight) focus on managerial dimensions such as stockouts, cash management etc. A second measure of game performance is return on equity (ROE). It was hypothesized that no statistically significant differences would be present. Table 3 below shows these results. TABLE 3 LEARNING AS MEASURED BY PERFORMANCE Strate- Opera- Signif- Item gic tional icance Final grade in course (scale is C- = 1 to A+ = 9 3.74 3.89 .53 Game grade (scale is 0 to 100) 88.8 87.8 .40 Case grade (scale is 0 to 100) 85.7 85.9 .62 ROE in game (after tax) 15.8% 13.8% .002 Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 180 A large statistical difference on ROE occurs with the strategic group earning a higher ROE. This difference is large enough to expect a statistically significant difference if the study were replicated. It is possible that the absence of the short-run production decisions encouraged students to devote more emphasis to strategic issues and thus obtain a higher ROE. The second set of data looks at learning as measured by student perceptions of the fulfillment of course objectives. Again it was hypothesized that no statistically significant differences would be present. Table 4 shown below gives these results. TABLE 4 FULFILLMENT OF COURSE OBJECTIVES (4 point scale) Strate- Opera- Signif Item gic tional icance Familiarization with managerial environment 2.89 2.89 .90 Ability to apply concepts and techniques of management 2.44 2.72 .065 Ability to analyze complex business problems 2.48 2.52 .68 Understand the function of managers 2.85 2.93 .48 Develop top management point of view 2.76 2.81 .73 Obtain practice in problem solving 2.56 2.70 .39 Understand the concept of strategy 3.04 2.98 .75 Examine a variety of Businesses 2.87 2.83 .73 No statistically significant differences were found. The last set of data on learning looks at students’ perceptions of the level of decision making in the game. This data can be analyzed by comparing the two groups or by comparing each group against some norm. The decision or cluster of decisions are listed in rank order (i.e. from most strategic to least strategic). A cluster of decisions is a group of related decisions. For example, there are nine marketing decisions in addition to the pricing decision. These nine decisions are listed as a cluster of decisions. Again the hypothesis is that no statistically significant differences will be found. Table 5 shown below shows this data. TABLE 5 LEARNING BY LEVEL OF DECISION MAKING IN GAME (1 = Strategic decision, 2 = administrative decision, 3 = operational decision) Strate- Opera- Signif Item gic tional icance Identify targeted market niche 1.09 1.07 .46 Establish Pricing Policy 1.37 1.35 .95 Establish other marketing policies 1.52 1.31 .01 Expansion of plant or equipment 1.69 1.86 .24 Make changes in long term loans 1.72 2.03 .08 Set price this quarter 1.90 1.84 .72 Make other marketing decisions this quarter 1.91 1.98 .65 Make production decisions this quarter 2.50 2.52 .99 Make changes in short term loans 1.96 2.0 3 .76 One statistically different result is found. The classes with the strategic emphasis view the establishment of marketing policies (other than pricing policies) as more strategic than the classes with the operational emphasis. Perhaps the use of multiquarter proforma statements permit this prospective for the strategically oriented classes. These decisions or decision clusters are listed in the order of their strategic importance (i.e. the numerical value of the first decision should be lower than the numerical of the second and the second should be lower than the third etc.). The students with the strategic emphasis have only one decision out of order whereas the operational emphasis has four out of order. The lack of consistent ordering by students with the operational emphasis raises some questions about their understanding of the role and meaning of strategy. COMPARABILITY OF GROUPS The major difference between the two groups is the higher ROE for the group with a strategic emphasis. Before this difference can be attributed to the strategic emphasis it is necessary to demonstrate that the strategic group is fundamentally the same as the operational group. One comparison is the amount of time devoted to the course. If the strategic group devoted more time to the game, then the higher ROE may be merely the result of extra time. Table 6 shows the amount of time devoted to the various aspects of the course. TABLE 6 TOTAL HOURS DEVOTED TO THE POLICY COURSE Strate- Opera- Signif- Item gic tional icance Total hours devoted to game 27.16 37.89 .003 Hours devoted to written cases 10.17 18.87 .006 Total hours devoted to course 66.66 80.55 .14 Those students with a strategic emphasis obtain a higher ROE with less time. The major benefit of the strategic emphasis seems to be greater efficiency (i.e. as such learning -- if not more -- with less effort). Before it can be concluded that the strategic emphasis provides comparable results with less time it is necessary to be sure that the demographic variables are fundamentally the same. Table 7 below shows the demographic data. TABLE 7 DEMOGRAPHIC VARIABLES Strate- Opera- Signif- Item gic tional icance Percent Males 46% 61% .17 Age 22.7 22.4 .65 Grade Point Average 2.95 2.83 .65 Percent married 4% 72 .51 This table does not reflect any statistically significant differences. However, it does not include majors since majors cannot be measured on a numerical scale. Thus it was necessary to perform two-way analysis of variance on the data on majors. The two-way analysis of variance produced a .085 level of significance. This difference occurred because one of the classes with the strategic emphasis had a Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 181 disproportionately high number of finance and accounting students. Using the .05 level of significance, this would not be a statistically significant difference. It does make the researchers uncomfortable and they decided to Investigate further. It is now necessary to ask whether the improved efficiency was due to the strategic emphasis or the higher proportion of accounting and finance majors. The data from the Ghiselli (1971) Self Description Inventory and Minor (1976) Sentence Completion Scale can be used to compare the managerial potential of the two groups. If the managerial potential is similar, it is possible that the higher proportion of accounting and finance majors would not produce a different level of performance. Table 8 shows the data for the key scale values (i.e. those with the strongest relationships to managerial success). TABLE 8 MEASURES OF MANAGERIAL POTENTIAL Strate- Opera- Signif- Item gic tional icance From Ghiselli Self Description Inventory Supervisory ability 27.59 28.80 .27 Intelligence 38.18 39.33 .33 Need for Occupational Achievement 37.27 37.73 .89 From Minor Sentence Completion Scale Total (managerial motivation) score 8.64 8.24 .80 This data does not suggest that the students with the strategic emphasis have different managerial abilities and motivations. This suggests that the differences in performance were due to the difference in emphasis rather than the differences in majors. CONCLUSIONS No meaningful differences in attitudes were found. The fundamental difference in learning was the higher ROE attained by the strategically oriented group. One could question whether this is a true learning outcome. Even if this outcome were unimportant and no differences were sound, the significant difference in time required to attain the same outcome is a major benefit to the strategic emphasis. On the basis of the evidence the strategic emphasis seems to offer clear advantages over the operational emphasis. The only weakness in this conclusion is the distribution of majors. While it is possible that the disproportionately high number of accounting and finance majors in one of the classes with a strategic emphasis may have contributed to a higher ROE, it is difficult to believe that this difference would contribute to the lower time usage. Thus we feel quite comfortable with the conclusion that the strategic emphasis produces at least the same learning with a significant reduction in time. Obviously these findings apply only to schools and locations with similar populations. Students at the school where the study was conducted must have a 2.5 GPA to gain admittance to the School of Business. The authors do not believe that their students are any different than the majority of AACSB-accredited schools. FUTURE RESEARCH Given the questions raised by the disproportionate number of accounting and finance students this study should be repeated with this factor under control. If the findings are supported, the selection of the game to be used in the business policy course can influence at a minimum the time required for the game. It is quite possible to find a game with much more of an operational emphasis than the one used. Thus the authors believe that this study may not have produced as significant a difference between the strategic and operational emphasis as is possible. In other words, the absence of differences may have been because both versions of the game were too similar. Thus further research with different games or greater differences in the same game may produce significant differences in learning. REFERENCES Boseman, F. G. and R. E. Schellenberger (1974), “Business Gaming: An Empirical Appraisal,” Simulation and Games, Vol. 5, No. 4, pp. 382-402. Ghiselli, E. (1971), Exploration in Managerial Talent, Santa Monica, CA, Goodyear Publishing Company. Greenlaw, P.S., and F.P. Wyman (1973), “The Teaching Effectiveness of Games in Collegiate Business Courses,” Simulation and Games, Vol. 4, 259-294. Minor, J.B. and M. G. Minor (1976), “Managerial Characteristics of Personnel Managers,” Industrial Relations, Vol. 15, 225-234. Pearce J. A. and Robinson R. B. (1986), Strategic Management, Homewood, IL, Richard D. Irwin, Inc. Raia, A. P. (1966), ”A Study of the Educational Value of Management Games,” Journal of Business, Vol. 39, 339-352. Schellenberger, R.E. and F. G. Boseman (1978), Business Policy and Strategy Management, New York, NY, John Wiley & Sons. Schellenberger, R.E. and L.A. Masters (1986), Mansym IV, (New York, NY, John Wiley & Sons. Wolf, J. (1986), “The Teaching Effectiveness of Games in Collegiate Business Courses: a Ten-year Update” Wolf, J. (1978), “The Effects of Game Complexity on the Acquisition of Business Policy Knowledge,” Decision Sciences, Vol. 9, 143-155. Table of Contents Volume 16, 1989 Quality Control Circles (QC™s): Towards a Computerized Simulation The Canadian Hospital Executive Simulation System (CHESS) The Impact of Using Group Performance Evaluation as an Experiential Exercise The Impact of Leader and Team Member Characteristics Upon Simulation Performance: A Start-Up Study Planning for Career Success: Is Where you are Going Where you Really Want to Be? The Production Frontier: Modeling Production in the Computerized business Simulation A Study of the Need for Valid Business Game Algorithms Modeling the Human Component of Business Simulations A Stimulating Simulation in International Business Business Ethics, Experiential Exercises and Simulation Games Collective Bargaining Simulation: Adding Reality Through Point Scoring The Use of Experiential Teaching Techniques: Creativity vs. Conformity Visualization and Guided Imagery in the Organization Behavior Class: An Experiential Exploratory Approach Arranging an Agenda: An Activity on Running Better Meetings Harried Harry: An Experiential Capstone for Students of Organizational Behavior Coping with Stress: An Experiential Exercise Fairness in the Classroom: An Empirical Extension of the Notion of Organizational Justice A Study of the Relationship Between Student Final Exam Performance and Simulation Game Participation Competency Based Development: A Management Development Exercise Simulation Performance Revisited: The Fit Between Instructor Style and Learning Style An Evaluation and Application of an Instrument for Measuring Pedagogical Effectiveness A Knowledge Based System to Support Reasoning by Analogy for Business Simulation Gaming using Forecasting Accuracy as a Measure of Success in Business Simulations The Development of Algorithmic Functional Business Games Strategy Design, Process and Implementation in an Unstable/Complex Environment: A Second Exploratory Study Simulation Integration Contrasts Between MBAs and Undergraduates in the Capstone Policy Course An Investigation of the Real World Usefulness of a Strategy and Policy Course Using a Business Simulation Framework Duel (sic) Views of Internships, as Experiential Learning The Impact of Decision Support Systems on the Effectiveness of Small Group Decisions - An Exploratory Study An Investigation of the Relationship Between Formal Planning and Simulation Team Performance Under Changing Environmental conditions Sensitivity Analysis with the Complete IFPS/Personal Student Analysis Package: A marketing Decision Support System A New Approach to Teaching Salesmanship using Persona, Microskills, and a Sales Process A Rational Case for Synthetic Experience as a Prime Ingredient in the Marketing Curriculum SalesHire: A Microcomputer-Based Salesperson Selection Exercise TRANSECON: An Interactive Program for Learning Transportation Economics Hypercard as a Construction Tool for Short Instructional Exercises A Game to Introduce Accounting Information Systems Students to Certain Internal Control Concepts "Commitments" - A Demonstration Proposal An Analysis of Popular Games as Experiential Models for Corporate and Collegiate Management Education An Exploratory Study of the Effects of Strategic Emphasis in Management Games on Attitudes, Interest, and learning in the Business Policy Course Predicting Individual Decision Making Performance in a Business Simulation: An Empirical Study Strategic Planning And Organizational Performance In A Business Simulation: An Empirical Study, PAM (Planning Action Management) Simulation of a District Sales Territory Lifelong Learning and ABSEL: An Inquiry on Definitions and Relationships A Review of Salient Trends in Proceedings: A Fifteen year (1974 - 1988) Review of ABSEL Contributorship