A Study of the Acquisition of Competitive Information in Business Simulations Page 158 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 ABSTRACT This is the initial report from a research study centered upon both the use and non-use on various forms of student-centered educational experiences. The full study is to attempt to discover some of the reasons these methodologies are not used by many business-school faculty members. The data from this study was collected by a survey of faculty teaching in business school programs throughout the United States. It collected information from both current users, defined as faculty members who have used at least one of these methodologies and faculty members who did not use these methodologies. The student-centered learning actives studied included computer-run business simulations, manual business simulations, 3-D, video-style business games, board games, in-class experiential learning exercises or an undefined learning experience. If a subject reported that he/she had not used any of the methodologies, he or she was directed on to that part of the questionnaire that provided insight on why they did not use any of these methodologies. The median time that respondents took to complete the survey was between 10 and 12 minutes, as the respondents were requested to select only one of the above types of exercises and base their responses on a single course in which the selected exercise was used. THE RATIONALE There have been many published studies on the use of simulations in business-schools within the US and in countries outside the US. With a few notable exceptions, most of these studies have been limited in their analysis of data collected by the author(s) in their own universities and from their students. There seems to be no baseline data as to those who use and those who do not use the general class of these exercises defined as student-centered experiential learning exercises. These exercises include both manual and computer-run simulations, board games, which often require 8 to 12 or more rounds and last for almost the entire academic term or 3-D, video style games. In addition, there are a large number of games and exercises that do not require so much time. These may be role-playing games or simulations that can be completed in a single class period or two and maybe require as little as 15 minutes. This study is attempting to discover the extent to which simulation and gaming exercises are being used. The assessment of the learning that takes place while students are playing these exercises are beginning to be addressed, however, most of these assessment studies have been based upon small samples that are very restricted and cannot be generalizes. While a multitude of student satisfaction studies have been published, there have been very few that address instructor satisfaction. Do instructors each use only one exercise during a single academic term or do they use numerous exercises? Since these methodologies require a substantial amount of both instructor and classroom time, do instructors, who teach more than one class per term use the same exercise in every class or do they use several simulations or games in their various courses? Historically, simulation and game development was a cottage industry. That is, an instructor or two designed an exercise and then used it in their class or classes and if the exercise was successful, they, at first, sent it to other instructors free of charge. Later, many of these instructor/developers began charging for their software. As technology and development became more costly and sophisticated, for-profit businesses created business simulations and charged fees for licensing their software. The important question now is, “Who pays these licensing fees?” Historically, business-school faculty were highly enterprising in game development or, at least a few of the leaders had been enthusiastic in creating simulations, games and experiential exercises. Has the appearance of for-profit software -development firms resulted in a decline in the creative inputs from current faculty? This study cannot answer this question, but it can establish a base line in which future studies will be able to answer for this question. From the authors’ observations on “Linked-In,” there seem to be a lot of faculty design inputs for the short simulations and games, especially for ones that can be played within a single class period. WHY NOW? At the 2014 ABSEL Conference, Dr. Maling Ebrahimpour made a keynote presentation on the recent inclusion of student- centered education in the AACSB accreditation standards. This presentation was the impetus for this research study. Teach and Govahi (1988) presented a paper at that appears in ABSEL conference proceedings. A highly revised version has been published in Simulation and Gaming (Teach and Govahi (1993). That paper explored what had been learned by students who had played business simulations, sat through lectures, participated in experiential learning exercises and read The Use and Non-Use of Business Simulations, Games and “In-Class” Experiential Learning Exercises: The Initial Report Richard Teach Georgia Institute of Technology richard.teach@scheller.gatech.edu Gita Govahi University of Southern California ggovahi@marshall.usc.edu mailto:ggovahi@marshall.usc.edu Page 159 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 and made comments on classical business cases. We surveyed business-school graduates, three to five years after they had completed their degrees and asked them about the nature of their jobs. Then we asked these now working college graduates to describe their jobs and to tell us how they learned the skills they were using. While we had a relative small sample BACKGROUND RESEARCH Every ABSEL conference has had papers devoted to these methodologies and the learning that takes place while students are participating in these learner-centered educational activities. Stanley Vance (1974) in the lead paper of the very first ABSEL proceeding publication wrote: “Be sure it (a business simulation or game) is purposeful. While class-room stimulation and having fun can and should be integral to game-playing, it must have pedagogical and hopefully, research value." That paper clearly stated that there must be learning involved in a business game and inferred that the student learning that takes place while playing the game needs to be assessable.” Quote from page 1, in the abstract. In the 2014 ABSEL proceeding publication, the lead paper, Jimmy Chang et al stated that in Hong Kong: The focus is on students' perception of the usefulness of using computer simulation as a learning tool in a strategic management course.” Quote from page 1 in the Abstract. (The bolding is the authors attempt to draw attention to the aspect of student learning.) If learning is taking place, then there must be some methodologies available to measure it. The authors’ survey is recording the teachers’ attempts to assess and enhance the learning that is acquired by those experiencing student-centered educational activities. The questions of “Do the winners of business simulations and game competitions learn the most has largely been answered in ABSEL presentations. The question of participant leaning by playing simulations and games and experiencing experiential learning exercises has been a constant ABSEL concern. See Eugene Byrne and Douglas Wolfe (1974), Stanley Vance (1974), Leo Parrish, Jr (1975), Gerald Smith (1981), , Hans Thorelli (1997), Richard Teach and Vishal Patel (2007), Richard Teach and Elizabeth Murff (2008), Steven Gold (2008), Joseph Wolfe, et al (2013), Jimmy Chang et al (2014) and many more are available from the ABSEL website. Measuring or assessing the learning acquired by those participating in business simulations and games has been addressed at ABSEL conferences more recently. Tom Basuray and Abraham Shani (1986), Richard Teach and Gita Govahi (1988 and 1993), Paul McDevitt (1997), Gregory Patton et al (1998), Jerry Gosen et al, (1999), Duane Hoover (2008), Maja Zelihic and Lora Reed (2014), John Burch et al (2014). Cannon et al (2014) presented a research paper that reported: “...simulations as a method of preparing students for organizational competence. However, it will link the principles of individual learning and the associated methodology of experiential learning to organizational learning and absorptive capacity, suggesting how to see these principles as a guide for simulation design.” Quote from page 333. THE SAMPLING PROCEDURE The initial sampling procedure consisted of sending an invitation to participate to; 1.) Authors of papers at the 2014 ABSEL conference; 2.) Recent users of the Organizational Behavior Teaching Society (OBTS) list-serve; 3.) Recent users of an Entrepreneurship list-serve in addition to; 4.) Business- school faculty within the state of Georgia. A few of the OBTS and Entrepreneur list-serve members were from non-US institutions. Only those who returned the invitation to complete the survey were sent our survey. The sampling procedure for the complete study will include the all the faculty member in US accredited business schools. QUESTIONNAIRE DEVELOPMENT The first question of the survey was “Have you used a simulation, game or an “in-class” experiential learning exercise in any course that you have taught? The respondents who answered yes were redirected to the section defined for users of simulations, games and “in-class” experiential exercises. Those who responded “No,” were redirected to the section for non- users. The Users User respondents identified the year they first used simulations, games and/or “in-class” experiential learning exercises, and the most-recent year they had used simulations, games and/or an “in-class” experiential exercises. They reported the topic of a single course they had taught when using simulations, games or “in-class” experiential learning exercises from a pre-defined list of 15 typically course topics, plus a course topic delineated as “other.” If they checked “Other,” they were to key in the topic of that specific course. Next the subjects recorded the importance of simulations, games and/or “in-class” experiential exercises when teaching the content of the selected class. They used a five-point scale ranging from ‘These techniques are not important” to “I would find it impossible to teach my course without these techniques.” They also reported the nature of their class: i.e. 1) a non-credit or a for-credit undergraduate course; 2) a Pre-MBA course; 3) a “for-credit” MBA or graduate course; 4) a course taught to PhD students; 5) an executive MBA course; 6) a course taught for continuing education; 7) a course taught for an out-of-the-university organization; 8) a non-related to business course. The respondents described the nature of the classroom experience, such as: 1) it was taught “only in a typical classroom;” 2) it was taught with some of the students located in a regular classroom, while other students were “on-line;” 3) it was taught only “on-line;” 4) They could detail any other condition not previously listed. Afterward that, they recorded the class size utilizing a specified set of groupings and indicated whether-or-not the course was team taught. Next they recorded the type of experiential activity that they used has used, such as: 1) a computer-run business simulation; or 2) a manual business simulation or a; Page 160 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 3) a 3-D, video-style game or, 4) an “in-class" experiential exercise or, 5) a “different type of experiential activity not covered by the ones above.” After that question, they were directed to a section of the survey that represented the selected experiential activity. They recorded the name of the exercise and the number of terms, they had used the named activity. They also reported if the exercise was supplied by a commercial organization and if the answer was “yes;” they responded to the question; “Who paid the fee.” In addition, they indicated if they or a fellow faculty member had designed or helped design the exercise. They then responded to a set of ten Likert-scaled statements using a six-point scale Strongly Disagree to Strongly Agree. The statements were: 1) The exercise resulted in a greater student involvement 2) The active involvement with the exercise facilitated student learning 3) The active involvement with the exercise helped students to gain a more in-depth understanding of the course content 4) The active involvement in the discussion after the exercise helped students to think of ways of applying their learning to real-life business situations 5) The discussion after the learning exercise helped students to reflect on their personal learning and plan effective strategies 6) Repeated experiences and participation in the experiential activities helped improve student skills 7) I spent a significant amount of time debriefing the experiential activity after the exercise had been completed 8) I did not spend a significant amount of time debriefing the experiential activity 9) I used specific assessment tools to measure the knowledge/skills that were acquired by students after having participated in this experiential activity 10) I designed a specific exam questions that tested student learning from this experiential activity The respondents who had used experiential activities then reported on the degree to which adoption of this particular intervention had been instrumental in their achieving their stated course objectives. The respondents who had used experiential activities further reported on the number of course sections or offerings that they had taught using the same experiential activity and during the same academic year. They stated the number of similar experiential activities that they used during that academic year. Subsequently the subjects expressed their plans for their next use of experiential learning activities. They answered this question by selecting one of the following possible answers: 1) I will use the same experiential learning activity; 2) I will use an experiential learning activity, but I will not use the same experiential learning activity; 3) I will not use the same category of experiential learning activities, but I will use a different category of educational learning activities; 4) I will not use any of the experiential learning activities. The Non-users The non-user respondents indicated their awareness or lack of awareness of computer-run business simulations, manual simulations, 3-D, video style business games, business board, and “in-class” experiential exercises. In addition they reported their awareness or lack of awareness in relation to the vendors of these experiential learning activities. These non-user faculty members responded to the following questions using a six-point Likert scale to indicate their level of agreement or disagreement with the following statements: 1) “In-class” experiential learning activities do not increase student involvement. 2) Active involvement in the in-class experiential learning actives does not facilitate knowledge acquisition. 3) Active involvement in the “In-class” experiential learning activities do not help students to gain a more in-depth understanding of the concepts presented in a course. 4) Active involvement in the “in-class” experiential learning activities do not help students to be able to better apply the concepts they learn to “real life” situations. 5) Student involvement in the “in-class” experiential learning activities do not help students learn better. 6) Repeated experiences and participation in the “in- class” experiential learning activities do not help build students’ skills. 7) Most students using “in-class” experiential learning activities are unable to learn the lessons that were intended to be taught by participating in these exercises. 8) There are no current methods to accurately assess the learning outcomes on “in-class” experiential learning activities. 9) Existing “in-class” experiential learning activities are too costly for my students. 10) The use of “in-class” experiential learning activities require too much of my limited class time. 11) The use of “in-class” experiential learning activities require too much of my time. 12) The use of “in-class” experiential learning activities too much effort on the part of my students. 13) The use of “in-class” experiential learning activities require too much of my students’ time. 14) The learning that takes place by participating in the “in -class” experiential learning activities are too subtle to be effectively measured. 15) I know how to coordinate “in-class” experiential learning activities with my teaching subject matter. 16) “In-class” experiential learning activities detract students from their learning objectives. Both Users and Non-users Have you used a simulation, game or an “in-class” experiential learning activity in any course that you have taught? 1) Yes 2) I do not know 3) No Page 161 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 All respondents reported their university’s calendar type as one of the following: 1) a semester system; 2) a quarter system; 3) a trimester system; 4) as a system different to those mentioned above. They replied to the question, “Does your business program have any dedicated faculty or other personnel for conducting experiential exercises?” 1) Yes 2) I do not know 3) No Responding to the question; “Does your business program have a dedicated center for experiential learning where students learn by engaging and interacting activities? 1) Yes 2) I do not know 3) No The subjects looked into the future to forecast the development of “in-class” experiential learning activities by conveying their opinions on the use of these methods 5 to 10 years hence. 1) less than currently exists 2) about the same as currently exists 3) a little more than currently exists 4) a lot more than currently exists All respondents conveyed their opinions to the statement: Do you think that most students who learn “in-class” experiences will be able to transfer their learning to their post collegiate careers better than students who have not been exposed to “in- class” experiential learning activities? 1) I strongly believe this is true 2) I believe this is true 3) q I believe this may be true 4) I do not believe this is true 5) I believe this idea is false 6) I strongly believe this idea is false They reported whether-or-not their business school has specially built classrooms to facilitate “in-class” experiential learning activities using 1) Yes 2) I do not know 3) No Each recorded their current position title followed by the number of students enrolled in their business school using the abbreviated scale 1) Less than 500 2) 501 to 1,000 3) 1,001 to 1,500 4) Over 1,500 What type of "assurance of learning" measures have been incorporated in the student-centered experiential learning methods that you use? Open ended question. Does your program have a dedicated center for experiential learning where students learn by engaging and interacting activities? 1) Yes 2) I do not know 3) No “Does your program supply trainers to conduct experiential exercise sessions?” Trainers are defined as full-time staff/ faculty who help design/inaugurate and debrief experiential activities at the Experiential Learning Centers while faculty teaching the class addresses the content issues relating to the experiential activity. 1) Yes 2) I do not know 3) No Does your program provide instructional designers to assist faculty members in developing educational objectives? 1) Yes 2) I do not know 3) No Does your program provide instructional designers in making your educational objectives more learner centered?” 1) Yes 2) I do not know 3) No Are there joint ventures with other departments within your institution for developing experiential learning activities?” 1) Yes 2) I do not know 3) No What kind of support do (or did) you receive from your institution for incorporating experiential learning into your curriculum? Open ended Does your institution make experiential learning a required component of certain courses? 1) Yes 2) I do not know 3) No SOME INITIAL DATA FROM EARLY RESPONDERS Question: Have you used a simulation, game or an “in-class” experiential learning activity in any course that you have taught? Yes 114 88% No ….. 16 12% No answer 4 Page 162 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 As a result, the current data set is quite sparse and statistical significance testing will not be attempted. Question: What was the first year that you used a simulation, game or an “in-class” experiential exercises? The respondents reported actual years. We have grouped their responses into decades 1960 1969. 1 1970 1979 6 1980 1989 9 1990 1900 27 2000 2009 39 2010 2014 21 No Response 25 Question: What was the last year that you used a simulation, game or an “in-class” experiential exercises? The respondents reported actual years. We have grouped their responses into decades 1980 1989 2 1990 1999 1 2000 2009 4 2010 2015 102 No Response 25 The data above shows that a great majority of the respondents started using these methodologies couple of decades ago and that many are still using them even if they have managed to switch specific experiential activities. Question: How important is it to teach your subject matter using experiential learning activities? (Only reported by those responding that they taught using simulations, games, or “in-class” experiential exercises) Question: What are your primary teaching subjects? Accounting 9 Business Law 3 Economics 5 Entrepreneurship 17 Ethics 5 Finance 8 Info Systems 7 Info Technology 1 Org Behavior 23 Management 31 Mgt. Science 6 Marketing 20 HR 9 Statistics 2 Strategy 23 Others 11 Others included Business Communications, CRS, Fluid Mechanics, Hospitality Mgt. Information Security, International Business, Negotiation, Mediations, Negotiations, Operations, Public Administration and Risk Management, all received one mentation, No Response; Question: In the most-recent year that you used a simulation, game or an “in-class” experiential learning exercise, what was the topic of the course that you taught? Accounting 6 Business Law 3 Economics 4 Entrepreneurship 8 Ethics 1 Finance 5 Info Systems 6 Info Technology 0 Org Behavior 16 Management 15 Mgt. Science 5 Marketing 16 HR 3 Statistics 1 Strategy 19 Other 16 Others included: Intro to Mgt. 2 Operations Mgt. 3 Negotiations 2 Along with Communications, Basic Business (MBA), Intro to Business, Decision Making, Hospitality Mgt. Info. Security, International Business. And Public Administration which received one mention each. No response 15 The breath of use of experiential learning activities was very broad. Far more extensive than the authors had originally thought. When there are larger numbers of responses of both users and non-users, interpreting the differences in these two sets of answers may provide some insights into the course-by- course coverage of experiential learning methodologies. Question: What is a reasonable estimate of the number of students who were registered in that class? 10 or less 2 Between 11 and 25 3 Between 26 and 50 45. Between 51 and 100 14. Between 101 and 200 …. 9 Question: How Important is it to teach you subject matter using student-centered experiential techniques? These techniques are not im- portant at all. 1 1% These techniques are fun, but not particularly important. 4 3% These techniques have some importance. 21 18% These techniques have great importance. 59 42% I would find it almost impossi- ble to teach my class without them. 32 27% No Response 17 These techniques are not important at all 1 1% These techniques are fun but not particularly important 4 4% These techniques have some importance 21 19% These techniques have great importance 65 50% I would find it almost impossible to teach my course without these technique 29 26% Page 163 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 Question: The course you selected to report on this survey was Question: Was this course taught on Line? Question: What is a reasonable estimate of the number of registered students in your class? Question: What specific type of student-centered experiential exercise did you use in that one specific course? The largest group that responded was the “computer-run” business simulation users, and the close second was the “in- class” experiential learning exercises group. However, those indicated that they use “a different king of exercise” is also significant and worthy of further exploration Question: What is the name of the computer run business simulation that you use? (This question was directed only to computer-run Business Simulation users) This clearly is a fragmented market. Question: How many terms have you used the current computer-run business simulation? This is an important finding. Over 25% of computer-run respondents have used simulations only once or at most twice. This finding may be indicative of a need for ABSEL to offer breakout sessions in its annual meeting to teach new users how to incorporate simulations into business curriculum. Question: Was this simulation a commercial simulation? Question: Who pays the licensing fees? No .102 89% In a classroom simultaneously with on-line students 11 10% Yes 2 2% No response 19 1% 10 or less 2 2% Between 11 and 25 35 30% Between 25 and 50 54 46% Between 51 and 100 16 14% Between 101 and 200 10 9% No Response 17 A computer-run simulation 48 41% A manual simulation 7 6% A board game 1 1% A 3-D, video style game 1 1% An “in-class” experiential exercise 41 35% A different kind of an exercise 20 17% No Response 16 BUSINESS STRATEGY GAME 5 CAPSTONE 5 THE BEER GAME 4 HBS Games 4 CLO-BUS 3 GEO 2 LINKS 2 Other games with only a single mention. 15 No response 8 A for-credit undergraduate course. 80 68% A for-credit MBA level course. 33 28% A course taught for an out-of-the university 3 3% An executive MBA course. 1 1% A continuing education course. 1 1% This is the first term. 4 9% This is second term. 9 19% I have used the same simulation 3 to 5 terms 12 26% I have used the same simulation 6 or more terms 22 47% No response 1 Yes 41 87% No 6 13% The students paid the Fee 28 68% The fee was shared between the student and the B school or the university 2 5% The B school or the university paid 11 27% No response 7 Page 164 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 Question: Did you or some other faculty member help creating this simulation? Developing computer-run business simulation is no longer a cottage industry. Question: The next time I teach this course I will: This indicates some dissatisfaction with the faculty’s current simulation. Fifteen percent will change their current simulation to a different one and 11% will not use a computer run simulation and 4% will not use any experiential learning activities at all the next time they teach the same course. Based on the information obtained in this initial test run, it would not be infeasible to conduct any statistical analysis on this data due to sparseness of the outcome. The authors are continuing to collect data for this project, and expect to collect substantial data to analyze and report on at the following ABSEL meeting in 2017. When this survey reaches you, please encourage all the faculty members in your business school to complete it ;As it will help us all to get a much clearer picture of the use/none use of Experiential learning methodologies in institutions of higher education. The authors will send an executive summary to each faculty who has responded to the survey and has provided an email for future correspondence. Basuray, M. Tom and Abraham B. Shani (1986), Experiential learning revisited: Some thoughts on designing more adaptive management education programs,” Developments in Business Simulation & Experiential Exercises, Vol. 13, pp. 80 – 85. Biggs, William D. and David J. Fritzsche (2010), “Using accumulative profits to assess simulation performance,” Developments in Business Simulation and Experiential Learning, Vol. 38 pp. `183 - 1189 Burch, Gerald F., John H. Batchelor, Nathan A. Heller, Joanna Shaw, Walter Kendall and Beverly Turner (2014), “Experiential learning: What do we know? A meta- analysis of 40 years of research,” Developments in Business Simulation and Experiential Learning, Vol. 41 pp. 279 – 281. Byme, Eugene T, and Douglas E. Wolfe (1974), “The design, conduct and evaluation of a computerized management game as a form of experiential learning,” Simulations, Gaming and Experiential Learning Techniques, Vol. 1, pp 22 – 30. Cannon, Hugh M., Andrew H. Feinstein, Daniel P. Friesen, and Attila Yaprak (2014), “The role of simulations in organizational learning: Building individual absorptive capacity,” Developments in Business Simulation and Experiential Learning, Vol. 41 pp. 333 – 343. Chang, Jimmy, Karen Ka-Leung Moon, Mei-Mei Lau, Man Chong Wong (2014), “Student views on the use of business gaming in Hong Kong: An update,” Developments in Business Simulation and Experiential Learning, Vol 41, pp. 8 – 13. Ebrahimpour, Maling (2014), Keynote address give at the opening of the ABSEL 2014 conference. Unfortunately, no transcript was made available for the ABSEL proceedings. Gosen, Jerry, John B. Washbush, Alan L. Patz, Timothy W. Scott, Joseph Wolfe, and Richard V. Cotter. (1999), "A test -bank for measuring total enterprise simulation learning." Developments in Business Simulation and Experiential Learning Vol. 26, pp 82 - 86. Gold, Steven (2008), “Designing and demonstration of an on- line managerial, economics game with automatic coaching for learning and graded exercises for assessment,” Developments in Business Simulation & Experiential Learning, Vol. 24, pp. 336 – 339. Hoover, J. Duane (2008), “Experiential learning is not just experiential teaching: Measurement of student acquisition via assessment centers,” Developments in Business Simulation and Experiential Learning, Vol.35, pp. 78 – 84. McDevitt, Paul K. (1997), “Measuring student learning using business simulations: A theory based perspective,” Developments in Business Simulation & Experiential Learning, Vol. 24, pp. 247 – 252. Parrish, Jr., Leo G., (1975), “Business simulations: Competition or learning,” Business Games and Experiential Learning in Action, Vol. 2, pp. 285 – 288. Patton, Gregory H. Daniel Cochece Davis, and Gita Govahi (1998), “Predictive models of learning: Participant satisfaction of experiential exercises in business education,” Developments in Business Simulation and Experiential Learning, Vol. 25, pp. 69 – 75. Smith, Jerald R., (1981), “Business game design, from theory to practice,” Developments in Business Simulation & Experiential Exercises, Vol. 8, pp. 90 – 94. Teach, Richard and Gita Govahi (1988), “The role of experiential learning and simulation in teaching management skills,” Developments in Business Simulation and Experiential Learning Vol. 15, pp 65 REFERENCES Yes 3 7% No 42 93% No response 3 Use the same computer-run business simulation 32 70% Use a different computer-run business simulation 7 15% Not use a computer-run Business simulation 5 11% Not use any type of experiential learning activity 2 4% No response 2 Page 165 - Developments in Business Simulation and Experiential Learning, Volume 43, 2016 Teach, Richard and Gita Govahi (1993), “The role of classroom techniques in teaching management skills,” Simulation and Gaming Vol. 24, No 4, pp 429 – 445. Teach, Richard and Elizabeth Murff (2008), “Are the business simulations we play too complex?” Developments in Business Simulation and Experiential Learning, Volume 35, p.p 205-211 Thorelli, Hans B. (1997), “Musings on business game performance evaluations, Developments In Business Simulation & Experiential Learning, Volume 24, pp 12 - 18 Vance, Stanley (1974), “Long live your business game,” Simulations, Games and Experiential Learning Techniques, Vol.1, page 7. Wolfe, Joseph, Bill Biggs and Steven Gold (2013) “Early- Determined Business Endgame Standings: A Replication and Expansion of Teach and Patel’s Findings,” Simulation & Gaming 1046878113477056, first published online March 4, 2013 Zelihic, Maja and Lora Reed (2014), “Teaching, learning and using technology in the on-line classroom,” Developments in Business Simulation and Experiential Learning, Vol. 41, pp. 313 - 320.