ENHANCING STRATEGIC MANAGEMENT GOAL- SETTING SKILLS IN THE BUSINESS POLICY COURSE Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 22 ENHANCING STRATEGIC MANAGEMENT GOAL- SETTING SKILLS IN THE BUSINESS POLICY COURSE Walter J. Wheatley, University of West Florida Robert W. Hornaday, University of North Carolina at Charlotte Tammy G. Hunt, Baylor University ABSTRACT This research project employed a pretest-posttest control group design to investigate the efficacy of business simulations in the enhancement of the goal- setting skills of business policy students. The study utilized a large sample size and controlled for the effects of the demand characteristic. Results strongly suggest that the use of a management simulation is an effective means to enhance goal-setting skills among business policy students. INTRODUCTION Although the use of managerial simulations in business policy courses has lost much of its luster, gaming is still a popular method of instruction [18]. Estimates of the percentage of Academy of Management members currently using games range from 32.7% [1] to about 50% [14;7]. The popularity that business games once enjoyed [31] seems to have leveled off [21]. Two major reasons for the diminished glamour of gaining stand out in the literature. Some instructors quit using games due to mainframe problems. It simply wasn’t worth the time and effort to fight the technical, political, and budgetary battles that had to be won to properly administer mainframe simulations. The expanding use of microcomputers for gaming [5;6] is rapidly eliminating most problems associated with mainframes [9; 11]. The other reason for the decline in prestige of gaining is more serious. Many are skeptical about the instructional value of games [14]. While gaming zealots are convinced gaming is a beneficial method of instruction they have been unsuccessful in empirically demonstrating the pedagogical advantages to the satisfaction of the skeptics. The debate over the instructional value of gaming rages unabated. The purpose of this paper is to present both sides of this debate and to offer some empirical evidence on the effect of gaming on goal-setting. BUSINESS GAMING EFFECTIVENESS RESEARCH A review of the literature on the benefits of business simulation offers conflicting results. On the positive side, there Is evidence that gaming appears to support the objectives of strategy-making, goal- setting, and decision- making (34] while enhancing student learning, interest and motivation (24]. Other evidence suggests that gaming is beneficial under conditions requiring high organizational learning and adaptation [35] and is fulfilling the “capstone” purpose of undergraduate policy courses by providing a “worthwhile learning environment capable of teaching many of the policy decision-making elements of a Business Policy course” [36]. Business games are effective learning mediums for international business concepts [16] and can be successfully used to create varied learning environments [30]. On the negative side, Neuhauser [19] states that there “is little evidence that games are in any sense efficient or effective methodological devices” (p. 124). Gaming has no positive effect on decision effectiveness [20] or on student class performance [8;2;27]. Finally, of major importance, Norris and Snyder [22] could not find a relationship between game performance and subsequent career success. In sum, the literature on the use of business simulations in business policy courses seems to show that students enjoy gaming and that games are good for teaching complex concepts. But student performance both in the classroom and in later life is not enhanced by participating in management simulations. Faculty research design seems to plague those who try to evaluate the effectiveness of business policy gaming. Butler, Markulis, and Strang [4] criticize existing gaming research for not following generally accepted research design guidelines. Their review of eight years of annual ABSEL Proceedings concluded that only seven per cent of the articles used all three characteristics of good research design (control group, randomization, and experimenter control of the treatment variable) [4]. Another major criticism of efforts to demonstrate the benefits of business policy gaming is the lack of empirically verifiable performance variables [28]. The few studies which have attempted to investigate the effectiveness of simulations usually use purely academic performance measures such as the student’s mastery of policy principles and fail to control for the resulting experimenter bias [34;36]. This study used a pretest-posttest research design that meets the three standards established by Butler et al [4], and employed an important performance variable (goal-setting) that is measurable. RESEARCH DESIGN Randomization. Data were collected from 230 Business Policy students (all seniors) at a large, Southeastern university. All students who happened to register for seven Spring semester, 1985, policy sections took part in the testing. Control Group. Four sections (treatment group) taught by one instructor participated in a simulation as the major focus of their business policy studies. No case analyses were Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 23 used in these four sections. Another instructor taught the remaining three sections (control group) using the case method approach. Demographic data collected from all students showed little difference between the backgrounds and capabilities of the 134 students in the treatment group and the 96 students in the control group (Table 1). Control of Treatment Variable. The design provided clear control of the treatment variable by the researchers. The treatment group participated in a simulation and did not analyze cases. The control group did the reverse. Performance Variable. Goal-setting is a critical skill for top level managers. In a 1983 survey, Boyd and Summers [3] reported that goal- setting tied for first place with strategic planning as items that should be included in business policy courses. In this study goal-setting was used as the performance variable to evaluate performance between the treatment group (simulation) and the control group (case method). Specifically, goal-setting ability was measured by counting the number of organizational goals each student could generate. The number of items generated (ideational fluency) is an accepted outcome variable used in the evaluation of divergent thinking and creativity [13,15]. Furthermore, Hargraves and Bolton [12] and Parnes and Meadows [23] found that when using this measure (ideational fluency) there is a high positive correlation between the number of items generated and the quality of these items. In a study of goal generation Wheatley [32] also found a strong correlation (.989) between the quantity and quality of goals generated. Thus, there is evidence that subjects who generate a large number of goals also generate a large number of quality goals. Both theoretical and empirical supports exists for counting goals generated to measure goal-setting performance. Control for the Demand Characteristic. In this type of research, a common cause of serious bias comes from respondents telling researchers the things the researchers seem to want to hear [25]. This “demand characteristic” poses a special threat when using student subjects. Students love to play games and will try to “win” if they can figure out the objective of the exercise. To minimize the effects of demand, the students in this study did not know the true purpose of the two goal-setting tests, except that the more goals they generated, the higher their course grade. In addition, the administration of other questionnaires during the semester served as a “bogus pipeline” to eliminate any demand compliant responses [26]. The Simulation. The simulation used was developed by Carl Gooding [10], presently at East Carolina University, and further modified by Dan Voich at The Florida State University. Called ENSIM (Environmental Simulation), the game is a highly competitive general management simulation with dynamic environmental constraints. ENSIM offers a realistic simulation of a manufacturing firm producing two products in competition with up to 19 other firms. Student groups select pricing, marketing, inventory, and purchasing strategies, establish debt, equity and dividend policies. Students make 29 decisions each monthly operating cycle. ENSIM has been used for Intercollegiate MBA competition in the Southeast on several occasions. In the present study, students in the treatment group were randomly assigned to two-member firms for the ENSIM competition. Case Analysis. The text used by the control group was Wheelan and Hunger’s Strategic Management and Business Policy [33]. This book is one of the most popular business policy texts [1]. The text contains a mix of high quality cases that illustrate important strategy and policy issues. In the control group, cases were analyzed on both an individual and group basis. In every analysis the students analyzed external and internal issues, defined problems, and made recommendations concerning various strategic management issues. Individual cases were analyzed with the assistance of the instructor. The instructor intervened in group cases only after the group had presented their analysis to the class. In summary, the pretest-posttest research design provided for randomization, a control group, control of the treatment variable, a measurable performance variable, and demand characteristic control. DATA COLLECTION At the beginning of the semester all of the students received instruction on the importance of goal- setting as a major component of the strategic management process and how to prepare quality goals. The students were then asked to generate as many goals as they could that they felt necessary for a business to attain in order to survive and prosper. The criteria for quality goals [29] that they were instructed to follow were: 1. Concrete and specific with the ability to lead and motivate. 2. Attainable but requiring competitiveness, imagination, and hard work to achieve. 3. Understood by those who are to develop means to achieve them. 4. Conforming to ethical and social codes accepted by society. 5. Mutually supportive of all other goals. This initial goal generation task constituted the pretest with the number of goals generated representing the measure of this variable. To avoid clues as to how many goals a student was expected to generate, the students wrote each goal separately on an unlimited number of three by five cards. At the conclusion of the semester the goal generation task was repeated as a posttest. RESULTS Analysis. Results of statistical analyses are shown in Table 2. In both pretest and posttest the treatment group outperformed the control group, 25.16 vs. 21.89 and 38.77 vs. 28.01 respectively. In comparing differences in goal-setting ability recorded by students participating in a simulation and those using the case method, improvement between the pretest and the posttest is the important measure. Table 2 shows the mean improvement in goal generation is 13.60 for the treatment group compared to a 6.12 improvement for the control group. This difference generated an F-ratio of 25.487 and a Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 24 corresponding p-value of less than .001. This is significant evidence, that in this research project, the use of simulation provided greater improvement in goal-setting than did the case analysis method. Any pretest can be expected to be a major influence on the results of a posttest and in this case there was a difference in the pretest scores of treatment and control groups (Table 2). To control for the effects of the pretest scores, another ANOVA was performed on the data treating the results of the pretest as a co-variate was negative, indicating that scores on the pretest were negatively correlated with improvement scores. Discussion. The utilization of a pretest-posttest design by this research project coupled with the balanced representation existing between the treatment group and the control group offers strong assurance against threats to the integrity of this study. The large sample size employed in the testing of goal-setting, an important strategic management skill, along with the efforts to control for the effects of demand characteristics add to the rigor of the study. The significant statistical findings strongly suggest the superiority of gaming over case analysis in enhancing the goal- setting ability of business policy students. A major weakness of the study is the significant difference in pretest scores between the treatment and control groups. It is possible, of course, that the instructor of the treatment group sections was unconsciously “teaching” goal-setting techniques, while the instructor of the control group sections was not. All that can be said to address this issue is that both instructors took part in the research design, received their training from the same university concurrently, and had the same amount of experience at teaching business policy. Their purpose in participating in the study was not to “prove” or “disprove” the superiority of either method of instruction, but to better understand the effectiveness of two different pedagogies. In any case, the limitations to this study indicated by the pretest difference between treatment and control groups may be more apparent than real. The important measure was improvement between pretest and posttest. Due to the nature of the goal-setting tests, a student who listed, say, 25 goals on the pretest would have to list 40 goals on the posttest to improve by 15 goals whereas a student who listed only five goals on the pretest would show equal improvement generating only 20 goals on the posttest. Thus, the higher scores of the treatment group on the pretest may have made It harder for them to show improvement. The negative coefficient of the pretest covariate (Table 2) indicates that this is likely. CONCLUSIONS Two major conclusions emerge from this study. 1. Participation in a management simulation better enhances the goal-setting abilities of business policy students than does the case method of instruction without gaming. 2. Rigorous research methods can be used in evaluating the effectiveness of gaming in business policy instruction. One caveat should be noted in relation to the first conclusion. The authors do not suggest that this study proves that the use of simulations is more effective than the case analysis method in all aspects of business policy instruction. The results of this study suggest that participation in a simulation does a better job of enhancing the goal-setting skills of business policy students. There are, of course, many other areas of business policy instruction that must be addressed. The authors personally believe that the business policy educational framework developed by Lang and Dittrick [17] combining simulations and case analysis is probably the best approach. In addition to replicating this goal-setting study, future research efforts should operationalize other performance variables such as scenario building and environmental analysis. Emphasis should be upon comparative studies of different instructional methods to provide a broader base of good research from which to argue the efficacy of gaming in business policy pedagogy. REFERENCES [1] Alexander, L.D., O’Neil, H., Snyder, N., & Townsend, J., “How Academy Members Teach the Business Policy Course,” Proceedings, 44th Annual Meeting of the Academy of Management, 1984. [2] Boseman, F.G., & Schellenberger, R.F., “Business Gaming, an Empirical Approach,” Simulation and Games, 1974, 5, 383-402. [3] Boyd, F.G. & Summers, I., “Business Policy Course Content and the Distinct Disciple Issue: A Survey of Professors,” Proceedings of the Southern Management Association Meeting, 1983, 1-3. [4] Butler, R.J., Markulis, P.11., & Strang, D.R., “Learning Theory and Research Design: How Has ABSEL Fared?” Proceedings, Twelfth ABSEL Conference, 1985, 54-55. [5] Cosenza, R.M., Boone, L.E., & Kurtz, D.L., “Integrating Microcomputers Through the Use of Marketing COMPUPROBS,” Proceedings, Twelfth ABSEL Conference, 1985, 54-55. [6] Dennis, T.L, “Painless Computer Simulation,” Proceedings, Twelfth ABSEL Conference, 1985, 157- 158. [7] Eldredge, D.L. & Galloway, R.F., “Study of the Undergraduate Business Policy Course at AACSB- Accredited Universities,” Strategic Management Journal, 1983, 4, 85-90. [8] Estes, J.R., “Research on the Effectiveness of Using a Computerized Simulation in the Basic Management Course,” Proceedings, Sixth ABSEL Conference, 1979, 225-228. [9] Fritzsche, D.J. & Cotter, R.V., “A Guide to Writing Microcomputer Simulations,” Proceedings, Twelfth ABSEL Conference, 1985, 69-73. [10] Gooding, C., “Decision Making Under Environmental Constraints: A Management Simulation Game,” (Unpublished Doctoral Dissertation, University of Georgia, 1976). [11] Goosen, K.R., “From the Editor,” ABSEL News and Views, 1985, 5-7. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 25 [12] Hargraves, D. & Bolton, P.K., “Selected Creativity Tests For Use in Research,” British Journal of Psychology, 1972, 63, 451-562. [13] Hershey M. & Kearnes, P., “The Effect of Guided Fantasy on the Creative Thinking and Writing Ability of Gifted Students,” The Gifted Child Quarterly, 1979, 23 71-77. [14] Keefe, M.J. & Cozan, C.J., “Popular Management Simulations: Why Some Simulations Succeed and Others Fail,” ABSEL News and Views, 1985, 5, 6-7. [15] Khatana, J., “Major Directions in Creativity Research,” The Gifted Child Quarterly, 1976, 20, 216-228. [16] Klein, R.D., “Can Business Games Effectively Teach Business Concepts?” Proceedings, Seventh ABSEL Conference, 1980, 128-131. [17] Lang, J.R. & Dittrich, J.E., “Information Skill Building, and the Development of Competence: An Educational Framework for Teaching Business Policy,” Academy of Management Review, 1982, 7, 269-2 79. [18] Markulis, P.M. & Strang, D.R., “Techniques to Enhance the Learning of Students Participating in Computerized Simulations,” Proceedings, Twelfth ABSEL Conference, 1985, 30-34. [19] Neuhauser, J.J., “Business Games Have Failed,” Academy of Management Review, 1976, 1, 124-219. [20] Newgren, K.E., Stan, R.M., & Kuehn, R.R., “Decision Efficiency and Effectiveness in a Business Simulation,” Proceedings, Eighth ABSEL Conference, 1981, 263-265. [21] Norris, D.R., “Management Gaming: A Longitudinal Analysis of Two Decades of Use in Collegiate Schools of Businesses,” Proceedings, 45th Annual Meeting of the Academy of Management, 1985, 378. [22] Norris, D.R. & Snyder, C.A., “External Validation: An Experimental Approach to Determining the Worth of Simulation Games,” Proceedings, Eighth ABSEL Conference, 1981, 247-250. [23] Parnes, S. & Meadows, A., “Effects of Brainstorming,” Journal of Educational Psychology, 1959, 50, 171-176. [24] Raia, A.P., “A Study of the Educational Value of Management Games,” Journal of Business, 1966, 39, 339-352. [25] Rosenthal, R., “Biasing Effects of Experimenters,” Et Cetera, 1977, 253-264. [26] Rosnow, R.L. & Davis, D.J., “Demand Characteristics and the Psychological Experiment,” Et Cetera, 1977, 361-373. [27] Rowland, K. & Gardner, D.M., “The Uses of Business Gaming in Education and Laboratory Research,” Decision Sciences, 1973, 4, 268-283. [28] Snow, C.C., “A Comment on Business Policy Teaching Research,” Academy of Management Review, 1976, 1, 133-315. [29] Steiner, G.A., Miner, J.B., & Gray, E.R., Management Policy and Strategy. New York: MacMillan & Company, 1982. [30] Thompson, K.R. & Keon, T.L., “Manipulating Environments of a Management Simulation: A Pedagogy and Test,” Proceedings, Eighth ABSEL Conference, 1981, 86-90. [31] Twelker, P.A., “Some Reflections on Instructional Simulation and Gaming,” Simulation and Games, 1972, 3, 147-153. [32] Wheatley, W.J., “Enhancing Strategic Planning Through the Use of Guided Imagery,” (Unpublished Doctoral Dissertation, The Florida State University, 1985). [33] Wheelen, D.L. & Hunger, J.D., Strategic Management and Business Policy. Reading, Mass.: Addison- Wesley, 1983. [34] Wolfe, J., “Effective Performance Behavior in a Simulated Policy and Decision-Making Environment,” Management Science, 1975, 21, 872-882. [35] Wolfe, J., “The Effects and Effectiveness of Simulation in Business Policy Teaching Applications,” Academy of Management Review, 1976, 1, 47-56. [36] Wolfe, J. & Guth, G.R., “The Case Approach vs. Gaming in the Teaching of Business Policy,” Journal of Business, 1975, 48, 349-364. Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 26 Developments in Business Simulation & Experiential Exercises, Volume 13, 1986 27 Table of Contents Volume 13, 1986 How to Use Microcomputers in Human Resource Management Courses What's My Line? An Exercise in Job Analysis, Description and Classification Relationships Between Team Cohesion Dimensions and Business Game Performance Four Factors Affecting Group Performance in Business Policy Simulations Enhancing Strategic Management Goal-Setting Skills in the Business Policy Course Using a Business Simulation in the Principles of Management Course - Learning Outcomes and Perceptions Personal Computers: Drexel's Experience Using Psychological Type to Enhance Negotiation Skills The Effects of Conflict Handling Styles on Negotiation Behavior: An Analysis of an Experiential Exercise An Experiential Test of Bargaining Decisions and Representation Self-Grading of Examination by Students, A Viable Alternative to Traditional Procedures Perceived Instructor Enthusiasm and Student Achievement An Experiential Approach to Teaching Subordinate-Oriented Communication Assessing the Effects of a Computerized Study Guide in Macroeconomic Principles: A Statistical Analysis A Sales Management Simulation for the PC an Integrative Tool for the Sales Management Course Teaching Re-evaluation of Salespersons Through the Use of a Simulation Game A Test of Student Performance and Attitudes and Performance Under Varying Game Conditions Developing and Testing Airways: A Marketing Simulation Experiential Learning Revisited: Some Thoughts on Designing More Adaptive Management Education Programs Developing the Competencies of Creativity and Accurate Self-Assessment Self-View and Norms: Some Longitudinal Laboratory Results Strategic Planning with an Experiential Case The ABSEL Research Consortium: Preliminary Steps and Position Papers An Analysis of ABSEL Conference papers (1974-1985) The Creation and Operation of a Data-base System for ABSEL Research Funding for a Research Consortium on Simulation and Experiential Learning Simulation - Indoctrination or Learning The Dilemma In Evaluating Classroom Innovations Personality Variables on Group Cohesion, Team Participation and Total Learning Justifications for and Problems in Developing and Using Computerized Experiential Activities Management Competency Models and the Life-Long Learning Project: What Role for ABSEL External Validity of Business Games Restructuring the University - An Experiential Exercise The Surrogate Class Exercise Images of Effectiveness: A Classroom Exercise The Algebran Industry: A Tutorial Framework Another Chance: A Multi-Leveled, Macro Business Game for Micro-Computers A Water Quality Management Simulation Game Acquisitions? Divestitures? Progress Report on a Conglomerate Game Enterprise: A Multi-Purpose Management Development Simulation Marketer: A Microcomputer Simulation in a High Tech Industry A Pedagogical Analysis of Alternative Tax Elections for Exploration Costs in the Mining Industry Using of Multiple Microcomputers Application Programs to Teach Fundamental Business Concepts and Practices Demonstrating the Role of Simulation in Strategic Operations Planning: A Case Study in Bank Check Processing Location Analysis The Subjective Side of the Decision Support System: A Pitfall for the Panacea A Group-Based Procedure for Revealing Judgmental Heuristics and Biases An Experiential Exercise in Bayesian Decision-Making Values for Selected Parameters in Physical Distribution Simulation and Games Computerized Business Management Simulations for Tyros The Management Decision Laboratory at New York University An Example: The Use of Management Games on Microcomputers by Computer Novices Enhancing Mainframe Simulation Via Microcomputers: Designing Decision Support Systems Integrating Personal Computers Into a Course as a Decision Support Tool A Decision Support System for Capital Funds Forecasting Performance and Attitudinal Affects of a Decision Support Package in a Business Game Using the Real World: Interviews with Woman Mangers The Cooperatively-Authored Student Handbook International Buying: An Experiential Exercise The Research/Teaching Interface: Turning a Pretest into an Experiential Exercise Building Microcomputer Business Simulations Simulation with Discrete and Continuous Mathematical Modeling An Interpolation Approach to Developing mathematical Functions for Business Simulations The Design of Simulation Models Using Thought Organizers The Assessment Center as a Teaching/Learning Device An Approach to Meeting the AACSB Guidelines for Introducing Skills and Personal Characteristics (SACs) Using an Interclass Simulating and Role-Playing Pedagogy Experiential Learning about the World of Work: A Program for Primary and Secondary School Educators Assessment of Relevant Personal Characteristics of Potential Entrepreneurs