You Have Troubles My Friend: That Starts with T and that Rhymes with B and that Stands for Business Simulations Page 199 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT Faculty members that use business simulations in their classes tend to consider that simulations are effective and enjoyable ways to learn business shills. But is this true? This paper, by the use of an electronic survey of students just after they had completed a business simulation a set of questions about how they felt about the business simulation. His survey was presented to students from a variety of US Universities in order to remove the bias of gathering information from only one university or a single professor or those who used only a particular simulation. The survey results show that as many as one quarter or 25 percent of the students reported that the initial assumption of “It is fun and educational” is not true for them. This are important finds and they need to be addressed. INTRODUCTION The research project in which this paper is only a small part, will hopefully shed light on this type of learning. As a result of this project, there have been several research papers published using data collected from students at multiple US universities authored by multiple authors. This is just one of the papers produced with more to come. Existing papers have been authored by Teach and Szot (2018a), Chasteen, Szot & Teach (2018), Teach and Szot (2018b) and Nugent (2018). The reason for this research program is an attempt to measure the learning that takes place when students participate in a business simulation. Some of the theory leading up to measuring participant skills learning was developed by Teach (2018). It is general assumed that business simulations are a teaching process that every or almost every student enjoys and they are able to turn the experience into a learning experience. Business simulations are used by the vast majority of business schools. One USA study (Faria 1998) showed that over 62 per cent of large business firms use simulation games in their training programs and about 97.5 per cent of AACSB member schools use simulations in their programs. Participant learning from a business simulation is critical if this methodology is to continue to be so universally adopted. However, the measurement of the learning that takes place when participants play business simulations continues to be elusive. There have been many attempts at measuring the learning that student obtain when playing business simulations and most have failed. See Nation, Reed and Swank (2014), Gredler (2004), Paich and Sterman (1993), Galareau (2005), Vos (2015) and many others. Do students need to like or enjoy their simulation experiences to learn from it? And if they do not like or enjoy this form of active learning, do they still learn from it? This paper attempts to answer some of these perplexing questions. Why use business simulations in classrooms Faculty members who use simulations in their classrooms believe in Activity Theory whether or not they know the term. What simulations need in order to have this theory to work is student involvement. Student involvement refers to the amount of physical and psychological energy that the student devotes to the academic experience. Thus, a highly involved student is one who, for example, devotes considerable energy to studying, spends much time on campus, participates actively in student organizations, and interacts frequently with faculty members and other students. (Austin, 1999). (This article was produced prior to the rise of on- line education.) Active learning is where the participant is active in the learning process, and it occurs when the objective includes pragmatism and skill application. (Gopinath and Sawyer, 1999). What kind of learning results occur when a student participates in a business simulation? Business simulation is a sub-set of the rubric of experiential exercises and “experiential learning proposes a constructivist theory of learning whereby social knowledge is created and recreated in the personal knowledge of the learner.” (Kolb and Kolb. 2005) The term used in learning theory is called Activity Theory. It claims that learning can occur only within meaningful activities. (Bodker, 1991.) Activity Theory’s primary assumption is that the activity and learning occur simultaneously. (Kaptelinin, 1996). Thus, the participation in a simulation is that meaningful activity. In addition, the use of business simulations results in Cooperative Learning. “Cooperative learning constitutes a special case of active learning, for it involves the use of small groups of students who work together with the goal of maximizing both their own learning and that of members of their group” (Johnson, Johnson, & Smith, 1991). The theory of cooperative learning is built on six propositions that are shared by the above three scholars. 1. Learning is best conceived as a process, not in terms of outcomes. You Have Troubles My Friend: That Starts with T and that Rhymes with B and that Stands for Business Simulations Richard Teach, Professor Emeritus Georgia Tech richard.teach@scheller.gatech.edu https://www.emeraldinsight.com/author/Gopinath%2C+C https://www.emeraldinsight.com/author/Sawyer%2C+John+E mailto:richard.teach@scheller.gatech.edu Page 200 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 2. All learning is relearning 3. Learning requires the resolution of conflicts between dialectically opposed modes of adaptation to the world. Conflict, differences, and disagreement are what drive the learning process. 4. Learning is a holistic process of adaptation to the world. 5. Learning results from synergetic transactions between the person and the environment. 6. Learning is the process of creating knowledge. Some do not enjoy playing business simulations There is a general assumption that students enjoy the experiential activity that a business simulation provides and these activities are better able to learn from this activity. James Schreier (1975) wrote: “In feedback evaluation reports, this exercise went from one which received several comments on how much fun it was and how enjoyable it was to an exercise that was both fun and thought provoking.” (Quote from page 110.) He went on and wrote: “The students who are experiencing, in many cases, their first simulation of a business world environment, find the focus on content delightfully refreshing, ultimately challenging, and if the results of their decisions are in any way what they expected, a very rewarding and enjoyable process.” (Quote from page 110.) But, is this always a “rewarding and enjoyable process” and what happens if it not? In 1981, HA Waggener wrote “Almost 78% of the respondents stated that the experience was enjoyable. They used expressions, such as, simulation was fun’, “very enjoyable, “made learning more interesting", and “a ‘wow experience’. (Quote from page 142.) Note the limitation of” almost 78 percent.” This means that more than twenty percent of the class admitted that the experience was “less than enjoyable.” In addition, this was an in-class result. How many, even if they were anonymous, did not want to suggest to their professor that the experience was not an enjoyable one. Thus, when the use of business simulations was in its infantry, it was recognized that not everyone considered business simulations as an enjoyable way of learning. The phenomenon of asking students about issues about the use of simulation in a single class is always suspect. Do the students want to confront the faculty member or to praise the faculty member? The data collected by the current study was both anonymous and across many universities, many different business simulations, many different faculty members and by researchers who were not know by the students. THE DATA As the data was collected from students at numerous universities, the survey required approval by the Institutional Review Board (IRB) at Georgia Institute of Technology and several other universities also ran the survey through their own IRB process. The Georgia Tech IRB committee required the following statement to be appended to the beginning of the survey: • You are being asked to volunteer for this research survey. This is an international survey about the business simulation that you have just participated in while in a college or university class. This research is being conducted by Dr. Richard Teach (Georgia Tech) and Dr. James Szot (UT Dallas). The purpose of this research is to better understand how students” participate in business simulations and the learning that takes place when students participate in a business simulation. This is critical information and we need your cooperation. • All survey participants will remain anonymous, meaning your survey responses are not linked to you. Your instructor has forwarded the link to this questionnaire to you. Most students complete this survey in 10 to 15 minutes. • Please note that there will be no compensation or costs for your participation and there are no anticipated risks or benefits to you by completing this form. • At the end of this survey, you may choose to send your anonymous responses to your course instructor. You do not waive any of your legal rights by participating in this study. • If you have any questions about your rights as a research participant, you may contact Melanie Clark, Georgia Institute of Technology, Office of Research Integrity Assurance, at (404) 894-6942. As a result, some university professors had only a small percentage of their class willing to complete or even partially complete the survey. While other faculty had almost every student complete the form Contacted faculty members at many US universities were asked to distribute the surveys and those that had responses are reported in the acknowledgement section of this paper. The questionnaire was constructed using the Qualtrics survey design software. The data were collected and stored by Qualtrics, Inc. The data were analyzed using SPSS. A copy of the survey was emailed to faculty members for review and if they approved of the survey they were asked to assist in the research by forwarding the address of the anonymous survey to their students after the last round of the business simulation they were using in their classes. This process assured the students of the legitimacy of the survey. The actual number of students that received the survey is unknown. If a student began the survey by responding to the first question: “Do you wish to participate in this international study of learning by the use of a business simulation? 1 = Yes, 2 = No.” If the student answered this question with a “Yes,” a record of the student’s participation was recorded. All the data were received without any personal identification being recorded. Table 1 shows the breakdown of student participants and the number of acceptable surveys processed in the analysis for this paper. After the the sample, the illogical data needed to be culled. What is illogical data? When a survey respondent gets bored, ges tired or tuns to michevius behavior, they leave tell-tail signs. One sign is to record the same value for a string of responses (666666). Another sign is when a pattern appears in their coded responses; and example would be 63636363 or 12345678. These Page 201 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 responces may in some sections of the survey and not in others. As a resdult, the individul responses were tested for these anomolies. In addition, some students leave some questions unanswered. The removal of affected data and unanswered questiions results on unequal cell sizes in a variety of analyses. THE ANALYSIS The result of the statistical analysis is shown in Table 2. The more significant values are shown in bold. Note that when analyzing The most important point shown in Table 2 is that approximaty one fourth of students participating in a business simulation did not like it! This dislike response was not statistically significant when considering the students grade point average. The “p” TABLE 1 THE NUMBER OF STUDENT RESPONSES The total student responses`received 847 The number that decided not to participate after starting the survey 142 ==== Number avaliable for analysis 701 Note that students had the freedom to respond or not to respond to most questions on the survey. The mean time to complete the survey was 9.9 minutes and 85% of the students completed it within15 minutes. TABLE 2. FREQUENCY COUNTS OF STUDENTS’ ATTITUDES ABOUT THE BUSINESS SIMULATIONS THEY USED Question # that Disagreed % that Disagree d # that Agreed % that Agreed N Sig. with GPA Sig. with My Team Finished The Simulation was both fun and educational. This question was omited from one survey administration. Thus a low N. 36 24.8% 109 75.1% 145 0.381 0.006 The learning experience of the business simulation will lead to a better job. 122 17.3% 401 76.7% 523 0.075 0.141 Playing the business simulation was a worthwhile. 138 26.4% 385 73.6% 523 0.625 0.493 What I learned from the business simulation was valuable. 101 18.1% 423 8170% 523 0.746 0.335 I was well prepared to participate in the business simulation 114 27.8% 296 72.2% 410 0.994 0.047 The business simulation was unrealistic. 281 68.4% 130 31.6% 411 0.167 9,947 The business simulation took entirely too much time. 294 71.6% 119 28.4% 410 0.003 0.522 The business simulation took entirley too much effort. 296 72.0% 115 28.0% 411 0.067 0.061 I spent more time on this coursae than I spent on any other course that I had this semester. 190 46.3% 220 53.7% 410 0.027 0.068 The simulation represented the “Real World.” 132 32.3% 277 67.7 409 0.119 0.417 Ther Simulation was unrealistic 281 68.4% 130 31.6% 411 0.167 0.121 Corralation between team placement abnd the student’s GPA The original data was collected using a 6 point Likert scale where 1 = Disagree Strongly and 6 = Agree Strongly The freuentcy counts combined the three Disagree responses and the three Agree responses. The significant levels were calculated using an Analysis of Varience of the means of the original six point scale. Page 202 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 value was 0.318. However, it is highly significant with the team’s final placent in the competition with other teams, with a “p” value of 0.006. This may be an anomily of the timing of the data collection. The data were collected after the last round of play. Thus, in the short run, winners learned more and had more fun if their team placed higher. No surprize here. There was a low correlation between the students’ self-reported GPA and the students’ self-reported ranking of his or her team after the last round of the simulation. The correlation coeficient was 0.108, but that correlation had a “p” value of 0.009 and an N = 586. Thus, while highly singnificant it explained very little varance. The students’ responses to “The business simulation took entirely too much time,”was statisticly significant with the students Grade Point Average. The GPA was student estimated on a 4.0 basis using two decimal places. That “p” value was 0.003. Students with high grade point averages tended to not agree with the statement and students with lower GPAs tended to agree with the statement. When it came to how well the student’s team placed in the competition, the feeling that the simulation took too much time was inmaterial. When the question was “The simulation took entirely too much effort” the difference between higher GPA and lower GPAs had a “p” value of 0.067. (When using data where the cost of being wrong is small, I tend to loosen-up the traditional 0.05 significance level.) Thus, it appears that students with higher grades tend to be more tolorant of greated effort than did students with lower grades. Whem comparing these responses to the student team’s competitive placemen the “p” value was 0.061. Students whose team placed higher in the competition were more tolorant than students whose teams place lower in the competition, A second variable; “I was well prepared to participate in the business simulation”with a “p” value of 0.076 could be considered as having an important relationship with the students GPA. Students with lower grades may feel unprepared. Recall that a student’s firm is a group project and no the result of a single student’s efforts. When it came to how well the students’s team place in the simulation competition, one variable: “I spent more time time on this course than I spent on any other couirse that I had this semester” had a “p” vale of 0.038. Thus, those who spentan extra- oridinary amount of timet finished higher tham teams that did not or could not spent the amount of time needed to “win.” As a side note, I was chastized by a fellow faculaty member in a faculty meeting that my students were spending too much time on the simulation I was using and not enough time on his course. I responded, “I guess thoe students understand the concept of Return on Invester Time.” Most, but not all faculty members responded with laughter. The other variable that clearly had an affect upon the student’s team’s competive position was; “I was well prepared to participate in the business simulation”with a “p” value of 0.094 Preperation from other tcourses taken before undertaking a business simulation is criticually imortant and before we allow students to “play” a business game, the prerequests When the question was, “The learning experience of the business simulation will lead to a better job,” the data shows that students that have better grades tent to think this is true while students with lower GPAs tend not to think this is true. When it come to the simulation’s realism and the students’ GPA and team placenent after the last round, the relationships had a “p” vale of 0.167 and 0.121. Clearly these relationships are not traditionally significant, but these relationships still may be important.Facuty must remember that over 0ne-fifth or more of the students do not think the simulation is fun and educational. IMPLICATIONS Facuty members usin a business simulation in theit coure(s) must remember that over one-fifth or more of the students do not think the simulation is fun and educational. In addition, students with commitments to other courses or non-curricullar activities may not be able or willing to spend the necessary time to a couse using a business asimulation. During the introduction, instructors must be clear about the simulations’ time and effort requireents must be included. I would suggest that the simulations’ similarity to reality not be stressed. It is clear from only a currsively review, that most simulations are not very realistic. The one outstanding example is that, at the start all competitors have the same assets, liabilities and the same parameters; one would never find this codition in the “real world.” ACKNOWLEDGEMENTS This publication was the result of a multi-university study of the learning that takes place while students participate in a business simulation in US universities. The author gratefully acknowledges the following professors and universities for their assistance in forwarding the electronic surveys to their students. Name University Name University Dr. Eric Kinnamon Alabama A&M Dr. Raghu Kurthakoti Arcadia University Dr. Stuart Graham Georgia Institute of Technology Dr. Kathleen Gruben Georgia Southern University Dr Steven Gold Rochester Institute of Technology Dr. Can Usley Rutgers University Dr. Michael Nugent SUNY Stony Brook Dr. Al Lovvorn The Citadel Dr. Mick Fekula University of South Carolina at Aiken Dr. Frances Fabian University of Memphis Dr. Shawn Carraher University of Texas at Dallas Dr. Larry Chasteen University of Texas at Dallas Dr. James Szot University of Texas at Dallas Dr. Blaine Lawlor University of West Florida Dr. Mihail Motzev Walla Walla University Page 203 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Austin, Alexander (1999), “Student Involvement: A Developmental Theory for Higher Education.” Journal of College Student Development, September/ October. Vol.40, No 5, pp 518-529. Bodker, S. (1991). Activity theory as a challenge to systems design. IN H.E. Nissen, H.K. Klein, & R. Hirschheim (Eds.), Information systems research: Contemporary approaches and emergent traditions. Amsterdam: Elsevier. Chasteen, Larry, James Szot and Richard Teach (2018), Simulations for strategy courses: Difficulty vs. realism, preliminary findings,” Developments in Business Simulation and Experiential Learning, Vol. 45, 2018, pp 64 – 68. Faria, A.J. (1998), “Business simulation games: current usage levels – an update”, Simulation and Gaming, Vol. 29, pp. 295‐30 Gopinath, C., and John E. Sawyer, (1999) "Exploring the learning from an enterprise simulation", Journal of Management Development, Vol. 18 Issue: 5, pp.477- 489, https://doi.org/10.1108/02621719910273596 Gredler. Margret 2004, “Games and simulations and their relationships to learning,” Handbook of research on educational communications, coulthard.com, pp 571- 583 Galareau, Lisa (2005), “Authentic Learning Experiences Through Play: Games, Simulations and the Construction of Knowledge,” Proceedings of DiGRA Conference: Changing Views – Worlds in Play. Website: http://summit.sfu.ca 5d5e4b90649934735982a7916cee.doc Gopinath, C. and John E Sawyer, (1999), “Exploring the learning from an enterprise simulation,” Journal of Management Development, Vol. 18, No 5, pp.477-489, https://doi.org/10.1108/02621719910273596 Johnson, DW, RT Johnson and K. Smith (1991), “Cooperative learning: Increasing college faculty instructional productivity, ASHE-ERIC Higher Education Report No.4. Washington, DC: Graduate School of Education and Human Development, George Washington University. ISBN-1-878380-09-5 Kapetlinin, V. (1996). Activity theory: Implications for human- computer interaction. In B.A. Nardi (Ed.), Context and consciousness: Activity theory and human-computer interaction. Cambridge, MA: MIT Press. Koistinen, K., & Kangasoja, J. (1997). Kolb, Alice Y. and David A. Kolb (2005), “Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education, Academy of Management Learning & Education. Vol, 4, No 2, pp 193 – 21. Nation, Daniel, Lora Reed and Alan Swank (2014); ‘‘Innovations and Future Directions for Experiential Learning for a Large Online Business Degree Program, (2014): Developments in Business Simulation and Experiential Learning, Vol 41, pp Nugent, Michael G. (2018), “Business simulation performance after completing a reflective observations module,” Developments in Business Simulation and Experiential Learning, Vol. 45, pp 31 – 41. Paich, Mark, John D. Sterman, (1993) “Boom, Bust, and Failures to Learn in Experimental Markets,” Published Online: Management Science (December 1993), https://doi.org/10.1287/mnsc.39.12.1439 Schreier, James W., (1979), “Issues in the organizational application of simulation and experiential material, Insights into experiential pedagogy, Volume 6, pp 107 – 112. Schreier, James W. (1975), “RAISE II, A persona; simulation, Simulation Games and Experiential Learning in Action, Volume 2, pp 181 – 188. Teach, Richard (2018), “Why is learning so difficult to measure,” Developments in Business Simulation and Experiential Learning, Vol. 45, 2018, pp 55 – 63. Teach, Ricard and James Szot (2018a), “How students play business simulations and what they learn: The preliminary report,” Developments in Business Simulation and Experiential Learning, Vol. 45, 2018, pp 89 – 92. Teach, Ricard and James Szot (2018b), “What business simulations teach: The effect of debriefing.” Presented at ISAGA 2018 and currently in-Press. Vos, Lynn (2015), “Simulation games in business and marketing education: How educators assess student learning from simulations,” The International Journal of Management Education, Vol 13, No 1, pp 57-74 Waggener, HA, (1981), “Students’ perception of learning by simulations.” Developments in Business Simulation & Experiential Exercises, Volume 8, 1981, pp 142 – 144. REFERENCES https://www.emeraldinsight.com/author/Gopinath%2C+C https://www.emeraldinsight.com/author/Sawyer%2C+John+E https://doi.org/10.1108/02621719910273596 http://summit.sfu.ca https://www.emeraldinsight.com/author/Gopinath%2C+C https://www.emeraldinsight.com/author/Sawyer%2C+John+E https://www.emeraldinsight.com/loi/jmd https://www.emeraldinsight.com/loi/jmd https://www.emeraldinsight.com/toc/jmd/18/5 https://doi.org/10.1108/02621719910273596 https://journals.tdl.org/absel/index.php/absel/issue/view/41 https://journals.tdl.org/absel/index.php/absel/issue/view/41 https://journals.tdl.org/absel/index.php/absel/issue/view/41 https://pubsonline.informs.org/author/Paich%2C+Mark https://pubsonline.informs.org/author/Sterman%2C+John+D https://pubsonline.informs.org/doi/10.1287/mnsc.39.12.1439 https://pubsonline.informs.org/doi/10.1287/mnsc.39.12.1439 https://pubsonline.informs.org/journal/mnsc https://doi.org/10.1287/mnsc.39.12.1439 https://www.sciencedirect.com/science/article/pii/S1472811715000026#! https://www.sciencedirect.com/science/journal/14728117 https://www.sciencedirect.com/science/journal/14728117