Coaching and Business Simulations: A Formula for Success? Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Coaching and Business Simulations: A Formula for Success? David George Vequist IV University of the Incarnate Word vequist@universe.uiwtx.edu ABSTRACT This research studies the affect of coaching on student’s success in a simple, off-the-shelf, business simulation. Data was collected and analyzed from an undergraduate Business Research class at a small, private, 4-year university. The data, most likely because of the small sample size, did not fully support the expected findings that students would benefit from coaching on strategy, which would result in higher levels of performance on an off-the-shelf business simulator. However, strong correlations were found between the self-stated familiarity with similar games and results in the simulation. The research and results that support these findings are presented and discussed. KEY WORDS: BUSINESS SIMULATION; COACHING; MANAGEMENT; TRAINING; HIGHER EDUCATION; TEACHING, BUSINESS SCHOOL INTRODUCTION This paper explores the relationship between coaching and business simulations to determine if students exposed to strategy coaching will be more successful in an off-the-shelf business simulation. There is much support in industry for a “blended” approach (using more than method) to teaching business principles. Every year, Training magazine highlights 100 best practices from industry by listing formal programs that have been deemed to be innovative and/or successful. In 2003, one of Training magazine’s 100 best practices included a best practice in First-Line Supervisor Development (Schettler, 2003). The winner was IBM, who in 2001, trained 5,431 new managers, located around the world, in their Basic Blue for Managers training program (it is a blended learning program for first-line supervisors). IBM’s approach is that training is an extended process rather than an event, and IBM immerses managers in yearlong program that is a combination of online self-study, simulations, competency assessments, management coaching and classroom experiences. The intent of IBM is to utilize a comprehensive training program that emphasizes ‘doing’ while learning (e.g., simulations) and continuous feedback during performance (e.g., coaching) to maximize the utility of the training. The goal is to teach first-line managers to become better people managers while aligning their work outcomes with IBM's strategic focus. IBM estimates that first-year business gains due to leadership skill improvements, purportedly from the training program, average $450,000 per employee (Schettler, 2003). In addition, Mihaliak and Reilly (2001) in an article for Best's Review state that “Online training courses, system-based coaching and business-simulation programs can save companies time and money.” They suggest that for modern businesses training is more than just a business expense, it is a creator of value. The article gives examples of organizations that have implemented new learning technologies (such as business simulations) that have reduced learning curves, increased speed to competency, implemented best practices, leveraged managers' time and reduced costs compared to traditional classroom teaching methods. In addition, Mihaliak and Reilly (2001) suggest that “Achieving results like these requires managers to blend appropriate learning technologies and strategies.” The three strategies they elaborate on are to shift expertise to intelligent mission-critical systems, use enhanced learning techniques and provide learning at point of need (Mihaliak and Reilly, 2001). Many industry practitioners recognize that a blended learning environment is the most effective method for changing the behavior of employees. One author Maise (1998), suggests that the following 4 methods be used to ensure this behavior change: coaching, assessment, simulation, and prescriptive learning. In addition, the suggestion is made to focus on the amount of change required and the methods used to change behavior, rather than just the information content of the course. The belief of this researcher, as well as many others (see Thompson, Purdy and Fandt, 1997) is that these strategies and learning technologies are highly effective in training undergraduate students in higher education settings as well as industry practitioners. As stated in a relevant article about business simulations in management training: Virtual business simulations are a form of combative training in which students pit their business skills against those of formidable opponents under the watchful eye of a training coach. This [author] advocates the inclusion of large-scale, virtual-reality simulations in the training of future managers. …they can help students develop an almost intuitive understanding of business, including a seamless perspective of its functional elements and a knowledge of how these elements can be coordinated to achieve a strong and profitable position in the market (Cadotte, 1995). The advantages of simulations, over traditional pedagogical methods, have been researched in many industry training and higher educational settings. Particularly in management training, “they offer a fairly realistic model of the interdependence of decisions that managers make in an organizational setting” (Thompson, Purdy and Fandt, 1997). A simulation is defined in this study as “an experiential learning 339 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 exercise in which students practice the design, implementation, and control of business strategies” (Cadotte, 1995). However, as mentioned in several of the articles above, a simulation utilized as a training tool by itself, is not as effective as it is in a blended learning environment. When you evaluate some of the skills that simulations are most commonly thought to develop, critical and strategic thinking skills, it is well documented that these skills need a great deal of practice, feedback from a subject-matter- expert (SME), and coaching to develop (Thompson, Purdy and Fandt, 1997). The idea of coaching, as used in this article is more similar to what exists in corporate environments than in a traditional educational setting. That is, an attempt by a SME to share expertise that is relevant to the tasks being performed, combined with other enhanced learning techniques and providing learning just-in-time (JIT), at exact at point of need of the learner. The role of a coach is described by Cadotte as a person to “help students develop critical thinking skills” and “[ask questions] during executive briefings [that] should stimulate students to consider additional dimensions in their strategic thinking” (1995). The suggestion of this author is the inclusion of coaching in a simulation-based training strategy may not only be helpful but necessary for superior learning to take place. This is supported by other researchers who state the following “[Simulations] change the focus of the class from teacher oriented to student oriented by shifting the emphasis from the teaching of the instructor to the learning of the student” (Thompson, Purdy and Fandt, 1997) and “[the coach’s role] comes at a highly relevant time and within a context that is germane for and unique to each student.” (Cadotte, 1995). FURTHER RESEARCH Researchers suggest that a “business school should strongly stimulate autonomous learning (i.e. a sideline or coaching role for the teacher), peer-to-peer learning, team based learning, experiential learning, and learning through the use of technology” (Moratis and van Baalen, 2002). These authors also point out that in order to promote this the business school should upgrade their management-learning environment so that they can run various technological applications such as: content resources, exploration- and experience-based learning (like simulations and games), community-based learning technologies, and applications that allow for contact between students and faculty as well as interaction among students. These are aspects of what the authors refer to as a networked (Moratis and van Baalen, 2002) or Synolic (from the Greek word "synolos", meaning complete or all together) business school (van Baalen and Leijnse, 1995). Typically, as mentioned above they will use a variety of interdisciplinary, action-oriented pedagogical approaches and new methodologies such as experiential or project-based teaching (Vequist, 2003). METHODS AND MATERIALS In this study, 4 students participated in an off-the-shelf business simulation for partial class credit (see Figure #1 below). The business simulation used was Big Biz Tycoon 2 (Released in 2003 by 4Head Studios and Activision), which was run on a standard PC (Intel Pentium III and Windows 2000 OS). In this simulation, the students hire/fire/train/manage employees (employees varied on their level of constitution, production, intelligence, creativity, charisma and loyalty, these can be increased through training and these levels also determine what tasks can be performed by the employee), purchase office equipment/fixtures to meet employee needs (if equipment is available or not available, it will affect the employee’s competencies), research new products (the students were instructed to focus on children’s toys), produce products (based on the competencies of the employees assigned to this task), advertise products to increase their price (again based on the competencies of the employees assigned to this task) and sell the products (can be set to automatic distribution). In addition, there are also options to invest profits to earn finance revenue, take out business loans for capital investments, buy additional properties and interact with competitors by headhunting, spying or sabotage (Note: these options were not utilized by the students due to the time allowed for the study). The game was played independently across players. 340 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Figure 1- Screen Print of Office Layout for Big Biz Tycoon 2 All students started with the same base setup- beginner level of difficulty, free game (New York, U.S. is corporate headquarters with the opportunity to expand to Hamburg, Germany or Tokyo, Japan), 1 employee, $50,000 start-up money and no furniture or products. All students were sent the game manual at least a day before the study and all students were given a 15 minute ‘training session’ on the navigational icons and general game play. All students then participated in game play for 45 minutes. The students were assigned to either the research or control group (2 in each group) based on their self- professed level of familiarity (2 familiar and 2 not familiar) with similar games (thus students were assigned to either the research and control group in matched sets). The research group differed from the control group in that the research group students were given coaching on strategy to ‘assist’ them during the performance of the simulation. Students that were assigned to the research group received Human Capital Management strategy coaching to assist them throughout the simulation. The student received individual coaching that was given to them as they participated in the game. The time spent coaching the individual student depended on the communication between the coach and the player and the observations of the coach. The coach did not suggest specific actions but suggested general ‘themes’ that revolved around maximizing the human capital of the business. The coaching strategy utilized was similar to the applied practice of “management by walking around” or MBWA. The types of advice that was given depended on the tact the student took and the situations the student found themselves in, however the coaching focused on the following areas: • Your employees are the core competency of your business • Hire/Assign your employees based on their competencies • Managing your employees is your most important responsibility • Treat your employees well and they will perform better for you • Continuing education of your employees will benefit the organization • Teamwork allows an organization to be more productive The students were undergraduate students of a small, private 4-year southwestern, university taking a Business Research class (with primarily 3rd or 4th year students). 4 members (2 females and 2 males) of the class voluntarily elected to perform in this study for partial class credit. Their performance on the simulation did not affect the credit they received in the class. Of the 4 students, 2 self-reported that they were very familiar with 341 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 similar simulation games (the 2 males) and 2 reported that they were unfamiliar with such games (the 2 females). The criteria by which student success in the simulator was measured was based on the outcomes reported by the simulator. They varied from measures of production, sales, advertising and organizational demographics. The criteria that were chosen to be evaluated, in this study, were similar to metrics that a practitioner (such as a small business owner) might use to determine the ‘success’ of their business. See the criteria in the table below: Table 1. Criteria for the Study Criterion # Criteria 1. Level of item researched (up to 6 levels of toys are available to be researched) 2. Items produced and maintained in inventory (these can be in different levels- see 1 above) 3. The price an items sells for (this is affected by advertising and these can be in different levels- see 1 above) 4. Salaries of the employees 5. Sum of all overhead costs 6. Ranking (ranking among the 6 competing firms- 5 are computer run) 7. Cash reserves (the player starts out with $50,000) 8. # of Employees 9. Sales (at the end of 2 weeks- all players did not fully finish 2 weeks in the simulation because of the time spent [45 minutes]) RESULTS The data that was reported was the student’s scores on the 9 criteria mentioned above. The author collected the data from the completed and saved games of the students. The data was collected after the students were asked to stop game play (45 minute period) and after all current game activities were stopped (e.g., many times the students had begun to research a new product or production of a current product was ongoing). Credit on the criteria, previously mentioned, was given for only completed items or tasks (e.g., if research on a new product was ongoing, that product was not counted in the data unless it was 100% complete by the end of game play). A Microsoft Excel (version 2000) spreadsheet was utilized to collect and analyze the data. For the first analysis, the scores of the first 3 criteria were reported and correlations and T-tests (one tailed, two sample-equal variance) to test the differences between the Control and Research groups. Table 2. Results of the Study (first 3 criteria) Subject ID Group Game Familiarity Level of Product Researched # of Product Level 1 in Inventory # of Product Level 2 in Inventory # of Product Level 3 in Inventory Price of Product- Level 1 Price of Product- Level 2 Price of Product- Level 3 C1 Control Low 1 0 0 0 83 NA NA C2 Control High 2 0 0 0 38 26 NA R1 Research Low 2 16 1 0 41 76 NA R2 Research High 3 0 0 0 41 64 137 Correlation w/Exp Grp 0.7071 0.5774 0.5774 NA -0.5225 0.9732 0.5774 T-Test w/Exp Grp 0.1464 0.2113 0.2113 NA 0.2387 <0.02* NA * Projected based on correlation without missing data, significant at the .05 level The correlation between level of product researched (there are 6 possible levels) and group membership did approach significance (with the Research having slightly higher levels of product researched). It can be argued that given a larger sample size this difference could have been significant. Only 1 significant correlation was found and that was the price of the 2nd level of product (in the simulator it was Toys: Soccer Ball). However, 1 of the Control group members was unable to achieve the 2nd level of product (the first was Toys: Teddy Bear) and this exaggerated the correlation. However, there does seem to be a trend of higher achievement (as mentioned in the first finding) and even possibly price on the 2nd level of products that correlates with being in the Research group. 342 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Table 3. Results of the Study (last 6 criteria) Subject ID Group Game Familiarity Total Salaries Sum of Overhead Costs Overall Ranking Cash reserves # of Employees Total Sales C1 Control Low 844 1408 4 41818 3 707 C2 Control High 2260 3433 2 33122 8 1921 R1 Research Low 1127 2584 3 38937 4 1916 R2 Research High 2058 2854 2 14261 7 6053 Correlation w/Exp Grp 0.0338 0.2024 -0.3015 -0.5066 0.0000 0.6589 T-Test w/Exp Grp 0.4831 0.3988 0.3492 0.2467 0.5000 0.1705 The expected results would have been supported by a finding of a significant correlation between Total Sales and Group membership. However, the correlation approached significance but fell short most likely because of sample size. There did seem to be a definite trend of the Research group having higher overall sales than the Control group but this could be explained by other phenomenon (such as the Hawthorne affect) than the coaching the students received. Another somewhat interesting finding is that the cash reserves of the Research group seem to be overall lower (the correlation is high [-.50] but not significant) than the Control group. Perhaps, the coaching strategy of ‘treating your employees right’ may have led to a increase in buying behavior to meet the ‘needs’ of the employees (more office furniture and fixtures). Based on the matched pairs sample that the author had utilized because of the presence of 2 students who were familiar with this type of simulation game, an analysis of the differences caused by simulation familiarity was undertaken. The following results were found: Table 4. Additional Results of the Study (first 3 criteria) Subject ID Group Game Familiarity Level of Product Researched # of Product Level 1 in Inventory # of Product Level 2 in Inventory # of Product Level 3 in Inventory Price of Product- Level 1 Price of Product- Level 2 Price of Product- Level 3 C1 Control Low 1 0 0 0 83 NA NA R1 Research Low 2 16 1 0 41 76 NA C2 Control High 2 0 0 0 38 26 NA R2 Research High 3 0 0 0 41 64 137 Correlation w/Fam Grp 0.7071 -0.5774 -0.5774 NA -0.6029 0.1157 0.5774 T-Test w/Fam Grp 0.1464 0.2113 0.2113 NA 0.1985 NA NA As shown by the results above, the student’s self-reported familiarity with simulation games did seem to have an impact on the level of product that they reached in the game (correlation is high [.70] but not significant). This would be expected because it would be assumed that the students would be able to respond quicker in the simulated environment and thus be more successful in reaching milestones faster. Table 5. Additional Results of the Study (last 6 criteria) Subject ID Group Game Familiarity Total Salaries Sum of Overhead Costs Overall Ranking Cash reserves # of Employees Total Sales C1 Control Low 844 1408 4 41818 3 707 R1 Research Low 1127 2584 3 38937 4 1916 C2 Control High 2260 3433 2 33122 8 1921 R2 Research High 2058 2854 2 14261 7 6053 Correlation w/Fam Grp 0.9788 0.7779 -0.9045 -0.7776 0.9701 0.6601 T-Test w/Fam Grp 0.0106* 0.1110 0.0477* 0.1112 0.0149* 0.1699 * Significant at the .05 level 343 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 There were 3 correlations that resulted in significance when the familiarity with simulation games was analyzed. Two of these correlations were highly interlinked- Total Salaries and # of Employees. Both of these correlations (.9788 and .9701 respectively) were significant at the .05 level. Again, like the previous finding, it is suggested that student’s who are more familiar with this modality are more likely to navigate the game quicker and thus achieve ‘better’ results. The other correlation that was significant was the overall ranking (-.90, p<.05), this was the correlation that would have supported the expected findings if it had been a correlation with Group membership. Instead however, it appears that familiarity with this genre of games is more important to success in this game that the coaching that was received. DISCUSSION Overall, it would seem that the students in the Research group who received business strategy coaching did not seem to perform at a significantly higher level than the students in the Control group who did not receive any coaching. There did seem to be some trends that indicated there could be significant findings if the sample size was larger. Findings similar to what was suggested by IBM (5,431 new managers trained and average savings of $450,000 per employee) are far beyond the reach of this simple analysis (Schettler, 2003). It is disappointing that the addition of coaching did not have a stronger effect on the student’s performance but the suggestion of this author is to replicate this study again with a larger sample size to test for a significant effect. It still seems clear to researchers like Cadotte, that the coaching role an instructor takes in a simulator exercise is of fundamental importance to the student’s success (1995). In addition, it is very important that even more research be performed that analyzes simulators in applied environments and particularly the differences between skilled (or super-users) and novices utilizing this type of methodology. The impact of experience using similar types of games/tools appeared (at least in this study) to affect the performance of the students more than any other factor. One additional area that should be explored by future studies is whether students exposed to a simulation and coaching assignment would enjoy the assignment more than a traditional assignment because it is more germane to the expected educational outcomes the student wants (e.g., be able to apply new skills, etc.). Muir (1996), in a study of experiential teaching of business communication skills, stated that the “traditional approach to teaching business communication often presents a simplistic process model that fails to address the political and power relationships that exist in the workplace”. In a related study, when students were asked if they were more satisfied with an experiential or project-based class, a couple of studies (Vequist, 2003 & Romney and Cherrington, 1993) found that students did have favorable opinions of these types of experiences. Therefore, similar feedback is expected from the students that would be surveyed regarding their satisfaction with a business simulator assignment. ACKNOWLEDGMENTS I would like to thank the Dean of the H-E-B School of Business & Administration, Dr. Robert Ryan, and also the students in my classes this last year that have made my transition to a teaching position such a rewarding experience. REFERENCES Cadotte, E. (1995) “Business simulations: The next step in management training.” Selections, Autumn, 8 Maise, E. (1998) “New thinking needed to alter learner’s behavior.” Computer Reseller News, August, Issue 803, 47 Mihaliak, C. and Reilly, M. (2001) “More efficient, less costly training.” Best’s Review, June, Volume 102, Issue2, 111 Moratis, L. and Van Baalen, P. (2002) “The radicalization of the multiversity: The case of the networked business school.” The International Journal of Educational Management, Volume 16, Issue 4/5, 160 Muir, C. (1996) “Using consulting projects to teach critical- thinking skills in business communication.” Business Communication Quarterly, Dec. Volume 59, Number 4, p.77-88 Romney, M. B. and Cherrington, J. O. (1993) “Implementing a graduate management consulting course.” Journal of Information Systems, Spring 1993, Volume 7, Issue 1, 48-62 Schettler, J. (2003). “Training Top 100: Top 100: Best Practices.” Training, Mar., Volume 40, Issue 3, 58 Thompson, T., Purdy, J. and Fandt, P. (1997) “Building a strong foundation: Using a computer simulation in an introductory management course.” Journal of Management Education, Volume 21, Issue 3 van Baalen, P. and Leijnse, F. (1995) “Beyond the discipline: inserting interdisciplinary in business and management education.” Challenges for the Business Schools, Eburon, Delft, 7-25. Vequist IV, D.G. (2003) “Project-based consulting groups for applied classes.” Presentation- ACBSP Annual Conference, Fort Lauderdale, Florida Author Note David George Vequist IV, Ph.D., Asst. Professor of Business; H-E-B School of Business & Administration, University of the Incarnate Word; San Antonio, Texas. Correspondence concerning this article should be addressed to: David G. Vequist IV, Ph.D. Asst. Professor of Business H-E-B School of Business & Administration University of the Incarnate Word; 4301 Broadway, San Antonio, Texas, 78209 Phone #: (210) 805-5825; E-mail: vequist@universe.uiwtx.edu 344 Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes