AN INVESTIGATION OF THE REAL WORLD USEFULNESS OF A STRATEGY AND POLICY COURSE USING A BUSINESS SIMULATION FRAMEWORK Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 122 AN INVESTIGATION OF THE REAL WORLD USEFULNESS OF A STRATEGY AND POLICY COURSE USING A BUSINESS SIMULATION FRAMEWORK Theodore F. Gautschi, Bryant College ABSTRACT This is an empirical study that questioned Bryant College MBA alumni who graduated during the years 1982 - 1987 regarding their current evaluation of the usefulness of the required Strategy & Policy course that they took prior to graduation. The course is designed around a business simulation framework. It uses a mixture of teaching techniques which include: simulation, experiential learning, case studies and lectures. The results provide useful insights to anyone using or planning to use, a business simulation with a Strategy & Policy course. Three types of questions were asked: (1) Demographic data (2) Two specific questions to be answered on a 5 point scale, and (3) Four open-ended questions. The response rate was good (about 33%, 163 out of 500). The graduates perceive the course to be quite useful to them. Analysis of the open-ended questions identify several valuable highlights, lowlights, benefits and limitations. Cross tabulation between the demographic data and the responses to the questions indicate some interesting trends. This paper is presented in five sections: (1) description of the concept and questionnaire, (2) analysis of the data (3) concluding remarks (4) data presentation--Appendix A (5) BML description--Appendix B THE CONCEPT AND QUESTIONNAIRE Since 1982 Bryant College has been using a mixture of techniques--simulation, experiential learning, case studies and lectures--primarily within a business simulation framework for teaching the MBA capstone Strategy and Policy course, The same format has been used throughout; however, there has been some “fine tuning" resulting from student critique, but nothing basic has been changed. The purpose of this investigation is to determine, by using a mailed questionnaire, how useful this MBA capstone course is perceived to be by the graduates who took it. The questionnaire was mailed to 500 MBA graduates (years 1982-1987) and 163 usable responses were received. The questions are in three categories: (1) Demographic Data Year of graduation, area of educational concentration (accounting, management, etc.) industry currently working in, current job title, current salary, sex, and instructor (during this time period two were used). (2) Two specific questions to be answered on a 1-5 scale ( 1= not useful at all, 5 = very useful) “Overall evaluation concerning the usefulness of this course to you.” “Your evaluation of the usefulness of this course in integrating the other courses of study. ‘ (3) Four Open-ended questions “Course highlights and lowlights. As you think back on this course what things come to mind?” “List what you consider to be the major limitations of this course. How could this course be improved?” “Other comments.” Responses to the demographic, and the two specific questions were tabulated without any special interpretation. After all of the responses to each of the open-ended questions were reviewed, they were segregated into categories and tabulated. This involved some interpretation, but it is believed that the final listings are representative of the input responses. Appendix A presents the data from this study. The Course This course is designed around a business simulation framework which is called the Bryant Management Laboratory--BML. It also uses a number of cases and lectures which are intended to expand the students’ learning regarding strategic management concepts and provide exposure to industries other than the micro-computer industry. The cases and lectures are handled in the traditional way. The BML application is somewhat unique and is described in Appendix B. Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 123 ANALYSIS Nearly one-third of the graduates who were mailed questionnaires responded. Apparently the course made enough of an impact that even several years after graduation students were willing and able to recall and evaluate the experience in some detail. According to the demographic data collected: Most of the respondents are in management positions (Table 4) and in late 1987 their average annual salary was $45,800 (Table 5). This was expected since about 95% of the graduate students hold full time jobs while they work on their MBA degrees to enhance their promotional opportunities. About 90% of the responses are equally distributed between the graduates covering the period 1984-1987. The other 10% are equally distributed between the 1982 and 1983 graduates (Table 1, all tables are in Appendix A.) over two-thirds of the graduates are Management majors and 16% are Accounting majors (Table 2) The graduates work in four major types of business: 1) manufacturing, engineering, software, computers and system design (34.4%); 2) banking insurance and other financial services (19.6%); 3) government activity (16.6%); and 4) health care (9.8%) (See Table 3). Of the respondents, 31.9% are female and 68.1% are male (Table 6). Dr. G and Dr. L conducted their sections in a like manner, using the same graduate assistants, the same simulation parameters, the same cases, the same simulation handbook and the same syllabus. 36.2% had taken the course from Dr. G and 54% from Dr. L. (During the period 1982-87, due to scheduling, Dr. L had about 50% more students than Dr. G.) As can be seen in Tables 8 and 9, the overall evaluations in terms of usefulness )f this course to the individual student, and its usefulness in integrating other courses is perceived to be high. (3.75 and 3.80 on a 1-5 scale, where 1=not useful at all and 5=very useful). About two-thirds of all respondents gave an evaluation of 4 r 5. There is a relationship between years after graduation and the perceived usefulness of the course, with those who graduated earlier perceiving the course to be more useful than those who graduated more recently. Perhaps this indicates that as students mature, they gain a greater appreciation for the course. The open-ended questions (Tables 10-14) were asked to get some idea of how the course is perceived to be useful by the participants some time after they had graduated. All of the responses were volunteered by the respondents without any kind of list or prompting. After reviewing the responses, nine categories for each question were established. All of the responses to each question were then coded into one of these nine categories and tabulated. The percentages shown are in relation to the total responding population of 163. Using this process any category with 10% or more of the responses would seem to be important. According to Table 10, by far the most important highlight for the course (44.8%) is, “having the opportunity to work and interact with others over a relatively long period of time on important activities.” A major benefit from this course is “the opportunity to learn about, and experience, group dynamics” (Table 12 32.5%). This Is consistent with the above stated highlight. Another important highlight is, “the challenge and opportunity to run a business in a somewhat real world environment involving risk, change and competition” (Table 10, 11.7%). The dominant benefit (Table 12, 55.8%) is, ‘the opportunity to integrate the various management tasks (planning, organizing, leading and controlling) as well as the organizational functions (finance, marketing, production and human resource management) into one strategically managed organization which establishes goals and strategies and makes the decisions to implement them. Other important benefits (see Table 12) are “establishing strategy and its execution: strategic management making decisions,” “hands-on experience relating to planning, marketing, and production” as well as “the opportunity to develop or improve presentation skills.” There is only one lowlight of significance (See Table 11): 23.9% believe that, “the course is too time consuming for just three credits," and suggest that it be presented as a two semester course. The one major limitation supports the above lowlight (Table 13, 27.6%) and also suggests that the course be presented in two semesters. This time-related lowlight may be a comment on the course or on student expectations which are conditioned by previous course workloads, or poor personal work and study habits. One of the features of the course design is that it challenges the participants to work effectively and efficiently in terms of using good interactive skills and good time management in order to be successful. There is no obvious completion point until the end of the semester. Just as in the real world, more can always be done in terms of analysis, developing decision-support systems, and improving the quality of the decisions. To be successful, the participants must quickly learn to work as a team, and equally share the workload; they Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 124 must gain an early understanding of the simulation, and not simply go through the motions; and they must exercise good time management as they make their decisions and prepare their reports and case analysis. There is a relationship between years after graduation and the volunteered lowlight, “too time consuming”, with more recent graduates perceiving it to be more important. However, the limitation, ‘too much time required" is approximately the same for all years. This seems to indicate that those who graduated earlier do not see this element as a lowlight feature, but simply as a limitation on their performance. A major question regarding simulation is often that of verisimilitude (the degree to which the players feel that the simulated situation is real.)1 Under lowlights 5.5% volunteered that “simulation rules are not always real world,” and under major limitations 9.2% volunteered that the ‘simulation is somewhat unrealistic in terms of parameters.” This response leads one to believe that in this situation verisimilitude is not a major concern. Over one-third of the respondents listed no limitations. (34.4%) The volunteered comments are compatible with the above- discussed highlights, lowlights, benefits and limitations. Nearly 20% of all respondents stated something like the following: - “An excellent course, don’t change anything.” - “My favorite course.” - ‘Most valuable course in the MBA program. - “Wish I could take it again.” - “I highly recommend this course.” Another 11.7% stated that it is, “a very good course’. Of the respondents, 57.7% did not reply to the comments question. There is a relationship between the student’s major and his/her perception of the usefulness of the course. In descending order they are: 1. Entrepreneur 2. Management 3. Accounting and CIS 4. Health Care 5. Finance All majors except Finance perceive this course as being quite useful. On the other hand, 25% of the Finance majors volunteered under Comments that this is an “excellent course”. The Finance majors also volunteered the highlight, “group dynamics more than any other major. There Is a relationship between the student’s major and his/her selecting, ‘too time consuming’ as a lowlight. Health 1 External Validity of Business Games by Dwight R. Norris 1986 ABSEL Proceedings. Care majors volunteered more often than any other major the lowlight “too time consuming” and the limitation, "too much time is required”. This may indicate that Health Care majors have insufficient preparation for/or interest In, the capstone course as it is being offered. In descending order they are: 1. Health Care 2. Finance 3. CIS 4 Management 5. Accounting 6. Entrepreneurs The Accounting and Management majors tend to recognize the strategic implications of this course more than the other majors. There is significant difference between male and female responses in two situations: a. More males volunteered the highlight, “working as a team”. 50% vs. 35%. b. More females volunteered the benefit, “group dynamics” 46% vs. 26%. Perhaps this indicates a greater sensitivity to group dynamics on the part of females, and a greater male awareness of the team process. The responses were significantly different for the two instructors in only three situations: a. Dr. L’s students volunteered the course lowlight, “it takes too much time,” more than Dr. G’s students 47% vs 15%. b. Dr. L’s students volunteered the benefit, “group dynamics”, less that Dr. Gs students: 26% vs 42%. c. Dr. L’s students volunteered the comment, “course is excellent”, less than Dr. G’s students 16% vs 20%. There are possible explanations for this: a. Dr. L is on the part-time faculty; whereas, Dr. G is on the full-time f ac u 1 ty. b. Dr. L’s classes tended to be larger than Dr. G’s: 40 to 50 students vs 25 to 30 students for Dr. G. Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 125 The data indicates other possible trends, but the sample sizes in those instances are too small. CONCLUSIONS Although this paper is based upon a specific course, it provides useful insights to anyone using or planning to use a business simulation with a strategy and policy course. 1. Past MBA graduates can, and will, respond to a mailed questionnaire asking them to evaluate how useful a capstone Strategy & Policy course taken before graduation Is to them now. This technique can be used to authenticate the value of a course design and its implementation. 2. This study demonstrates that a mixture of teaching techniques2 can be used within a business simulation framework and result in an MBA capstone Strategy & Policy course that is evaluated to be quite useful by former participants long after the experience has been completed. 3. The following are perceived to be important aspects of such a course:3 1. ‘The opportunity to work and interact with others over a long period of time on important activities involving change, risk, and challenge.” 2. “The opportunity to integrate the various management tasks and organizational functions.’ 3. “The opportunity to gain hands-on experience relating to planning, finance, marketing and production. 4. The data indicates a positive relationship between years after graduation and the perceived usefulness of the course. This may indicate that as students gain working experience, they also gain a greater appreciation for the course experience. This relationship raises some question about the validity of current student evaluations of a Strategy & Policy course and indicates that evaluations should also e sought from alumni after they have had a few years experience. 5. There is a relationship between the students’ major and their perception regarding: 2 The Role of Experiential Learning & Simulation in Teaching Management Skills” by Richard Teach & Gita Vocahi, 1988 ABSEL Proceedings. 3 These compare favorably with the results reported in, “A Comparison of Student Perceptions with Accepted Expectations for Business Simulation by McLaughlin & Bryant 1987 ABSEL Proceedings. (1) the usefulness of the course (2) the course being too time consuming In general, the Entrepreneur, Management, Accounting, and CIS majors perceive the course as being more useful and not too time consuming. Whereas, Health Care and Finance majors perceive the course as not being so useful and that it is too time consuming. More effort should be made to assure that all majors have an experience which is perceived to be more useful by them. Appendix A DATA PRESENTATION The responses for each of the demographic factors, the answers to the two specific questions, and the categories responses which were volunteered for the open-ended questions; are shown as a percent of those completing the questionnaire. The mean responses for the two specific questions were cross-tabulated vs. each demographic factor and MANOVA (Multivarient Analysis of Variance) was used to determine which of the factors are significant at the .05 level. In addition, the categorized responses to the open-ended questions as a percent of those completing the questionnaire were cross-tabulated vs each demographic factor and a chi- square test was used to determine which relationships were significant at the .05 level. APPENDIX B THE BML The BML uses a competitive, flexible, sophisticated and challenging, state-of-the-art total-enterprise business simulation that is an expanded version of, “THE IMAGINIT MANAGEMENT GAME.”4 It is used in a dynamic case format, with some 120 parameters that can be changed at any time by the administrator to establish or modify the business environment and operational rules. The MBA capstone course uses BML parameters that depict the microcomputer industry. This industry was chosen because of its rapid growth and volatility. To encourage strategic thinking and planning, and provide the opportunity for job rotation, the simulation is run for sixteen quarters following four practice quarters. This, plus other aspects of the course design, are compatible with many of the recommendations 4 The Imaginit Management Game by Richard F. Barton; Lubbock, Texas (Active Learning 1978) Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 126 set forth by Jauch, et al.5 Specifically, it is designed to 1) “provide an integrative experience in which the student can put into practical decision-making terms all of the skills and knowledge acquired in the business program.” 2) require the development and use of "decision--support systems (DSS) to support integrated decision-making.” 3) ‘Require numerous activities beyond mere decision-making.” 4) Require firms to prepare and present long range plans and annual reports to the instructor, and often others, acting as a Board of Director Manuals describing the simulation in detail, including the par meters and their initial values, the processing that the computer performs, and descriptions of how all printout figures are developed are made available to each student along with the firm’s initial Year 0 Quarter 4 position. The students in each section of this course are self-divided into firms of four or five persons each. Where possible, those who are interested in the various functional areas-- such as accounting, finance, system analysis, production/engineering and management--are spread equally amongst the firms. As the simulation proceeds, everyone is required to perform a different operating function each four quarters, e.g., president, vp/marketing, vp/production; etc.) A maximum of seven firms can compete in any one industry. Each firm has the opportunity to make seven quarterly decisions for each of three products, plus seven company- wide quarterly decisions. These decisions are entered by the firm once or twice a week into the mainframe computer using a computer terminal6 according to a schedule established by the professor conducting the course. After the decisions for all of the firms in an industry are processed, each firm is provided with printouts which indicate the results of their interaction with the other firms, These include a financial statement, a balance sheet, the firm’s policy and environment, and an end of quarter report giving selected information about all of the firms in the industry. Once the students achieve a reasonable understanding of the simulation, as determined by a written quiz and the results of the practice simulation, class time is devoted to lecturing, case studies and the presentation of simulation annual reports by the various firms. Most participants claim they devote ten to twenty hours a week outside of the class meeting preparing decisions and complying with the other requirements of the course. In order to cope with the course load most firms develop decision-support systems which use a micro-computer to help them prepare their BML decisions and annual reports. 5 “Capstone Renaissance = Simulation + Interaction + DSS by Jauch, Snodgrass & Szewczak, see ABS L ‘88 Proceedings. 6 Curran Horn day in ‘A Hybrid Method of Executing a Management Simulation: Combining the Best of Mainframes and Microcomputers” ABSEL ‘88 Proceedings, indicates that “this is the fastest system of all.” All firms in an industry start from the same Year 0 Quarter 4 position. Success is measured by where each firm finishes each simulated year with respect to the other firms in the industry as modified by weighing factors assigned by the firm. The success factors are: average stock price for the year reported, return on equity over the past four quarters, average annual market share for each product line, and total cumulative earnings. The firms assign weights to these factors in accordance with their chosen objectives and strategy These weights can only t changed annually Each firm is required to prepare and submit: 1. An environmental scan based upon the simulation parameters. 2. A list of objectives. 3. A strategy to achieve the objectives. 4. A long-range plan to implement the objectives and strategy. 5. A listing of weights to be applied to the “success factors” and are in accord with 1-4 above. 6. Their organization chart with job descriptions and a job rotation plan. At the end of the first simulated year each firm must also prepare and submit a ‘hardcore analysis” which is intended to provide guidelines for policy and decision making. These are graded on a relative basis, and will directly affect the results of the simulation interaction. Course grades are based upon the firm’s success in the four simulated years, the quality of its annual reports , individual contribution to the simulation and tie case studies and the results of any exams. Developments in Business Simulation & Experiential Exercises, Volume 16, 1989 127 Table of Contents Volume 16, 1989 Quality Control Circles (QC™s): Towards a Computerized Simulation The Canadian Hospital Executive Simulation System (CHESS) The Impact of Using Group Performance Evaluation as an Experiential Exercise The Impact of Leader and Team Member Characteristics Upon Simulation Performance: A Start-Up Study Planning for Career Success: Is Where you are Going Where you Really Want to Be? The Production Frontier: Modeling Production in the Computerized business Simulation A Study of the Need for Valid Business Game Algorithms Modeling the Human Component of Business Simulations A Stimulating Simulation in International Business Business Ethics, Experiential Exercises and Simulation Games Collective Bargaining Simulation: Adding Reality Through Point Scoring The Use of Experiential Teaching Techniques: Creativity vs. Conformity Visualization and Guided Imagery in the Organization Behavior Class: An Experiential Exploratory Approach Arranging an Agenda: An Activity on Running Better Meetings Harried Harry: An Experiential Capstone for Students of Organizational Behavior Coping with Stress: An Experiential Exercise Fairness in the Classroom: An Empirical Extension of the Notion of Organizational Justice A Study of the Relationship Between Student Final Exam Performance and Simulation Game Participation Competency Based Development: A Management Development Exercise Simulation Performance Revisited: The Fit Between Instructor Style and Learning Style An Evaluation and Application of an Instrument for Measuring Pedagogical Effectiveness A Knowledge Based System to Support Reasoning by Analogy for Business Simulation Gaming using Forecasting Accuracy as a Measure of Success in Business Simulations The Development of Algorithmic Functional Business Games Strategy Design, Process and Implementation in an Unstable/Complex Environment: A Second Exploratory Study Simulation Integration Contrasts Between MBAs and Undergraduates in the Capstone Policy Course An Investigation of the Real World Usefulness of a Strategy and Policy Course Using a Business Simulation Framework Duel (sic) Views of Internships, as Experiential Learning The Impact of Decision Support Systems on the Effectiveness of Small Group Decisions - An Exploratory Study An Investigation of the Relationship Between Formal Planning and Simulation Team Performance Under Changing Environmental conditions Sensitivity Analysis with the Complete IFPS/Personal Student Analysis Package: A marketing Decision Support System A New Approach to Teaching Salesmanship using Persona, Microskills, and a Sales Process A Rational Case for Synthetic Experience as a Prime Ingredient in the Marketing Curriculum SalesHire: A Microcomputer-Based Salesperson Selection Exercise TRANSECON: An Interactive Program for Learning Transportation Economics Hypercard as a Construction Tool for Short Instructional Exercises A Game to Introduce Accounting Information Systems Students to Certain Internal Control Concepts "Commitments" - A Demonstration Proposal An Analysis of Popular Games as Experiential Models for Corporate and Collegiate Management Education An Exploratory Study of the Effects of Strategic Emphasis in Management Games on Attitudes, Interest, and learning in the Business Policy Course Predicting Individual Decision Making Performance in a Business Simulation: An Empirical Study Strategic Planning And Organizational Performance In A Business Simulation: An Empirical Study, PAM (Planning Action Management) Simulation of a District Sales Territory Lifelong Learning and ABSEL: An Inquiry on Definitions and Relationships A Review of Salient Trends in Proceedings: A Fifteen year (1974 - 1988) Review of ABSEL Contributorship