ERPsim Games in Management Higher Education Page 264 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT The ERPsim simulations have been used in the past decade mainly to help students discover real-life skills in the classroom environment within higher education and to promote better decision-making skills. To date, these simulations have aided numerous instructors in teaching business processes in Information Systems courses. This paper reports on utilizing the games in a new context within senior level, undergraduate, managerial decision-making classes in universities. The main objectives in introducing this simulation to the students was to give them exposure to the use of business information in tactical and operational decision-making.. By implementing simulation business games within the classroom curriculum and modify the way in which the simulation was introduced over subsequent interactions of the course , the instructors were able to increase student satisfaction.. NOTE: This is an extended abstract NOT a full paper INTRODUCTION Undergraduate students typically bring little business or work experience with them into the classroom. Some of the concepts and topics introduced in the classroom, particularly in the realm of management education are difficult to understand without such valuable experience. These simulations, which are aimed at replicating the business environment, can positively enhance the students’ training and complement the classroom experience. By offering students a more practical understanding of concepts in this way, the students are able to start building their management decision-making skills. The students then get the opportunity to enhance their understanding by gaining a realistic view of how these real-life decisions are being made and what their consequences could be as well. Simulations that appear well suited to the management classroom and have already been extensively tested in the Information Systems classroom are those developed in the HEC Montreal ERPsimLab. They provide ERP related simulations using SAP software at varying levels of complexity (Leger, 2006; Leger et al, 2007). To the best of the authors’ knowledge, the series of “experiments’ reported in this and an earlier paper [reference to be included after the review process] are the first time the ERPsim simulations have been used in a management decision-making class, although some work has been done in using the games to developing cognitive learning (Cronan et al 2012). Regardless, the games have been widely used in teaching business processes (Leger et al 2010; Seethamraju 2011, Leger et al 2011; Dick and Szymanski, 2013). In the study reported within this paper, the focus of the authors was to understand how to improve the introduction and use of the simulation to prime the students for its use to help them get the most out of it. By performing this study, the authors hope to deermine a “model of best practice” for utilizing the game in the undergraduate classroom setting. However, it was necessary to first determine whether or not the simulation aids in student understanding of the managerial decision-making concepts introduced in the course at all. If it was deemed that it indeed was a helpful tool, the authors could then move to better understanding what factors contribute to that understanding of the students. Furthermore, the use of simulations in the classroom setting could potentially have positive implications for business institutions around the world. The use of the simulation seems to match the most recent AACSB guidelines as modified in 2017, regarding a new knowledge area in undergraduate programs: “Evidence-based decision making that integrates current and emerging business statistical techniques, data management, data analytics and Information technology in the curriculum. Student experiences integrate real-world business strategies, privacy and security concerns, ethical issues, data management, data analytics, technology-driven changes in the work environment, and the complexities of decision making.” (AACSB 2017). BACKGROUND Simulation Games The idea of using simulation games in the classroom is not new (Bredemeier and Greenblat, 1981). Playing simulation games that can give an idea of real-life workplace environments gives students the opportunity to experience practice and theory together and often will provide the opportunity to acquire skills that are normally acquired through practice after graduating. However, this type of learning and experiment must be done in a reduced time frame due to scheduling of terms and semesters for ERPsim Games in Management Higher Education Giulia Paulet Northern Arizona University gip8@nau.edu Geoffrey Dick Northern Arizona University gfdick@aol.com mailto:gip8@nau.edu mailto:gfdick@aol.com Page 265 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 universities. By choosing a simulation that can provide rapid feedback in a cohesive manner alongside the instruction, the students can better understand and be led to strategic adjustments. These adjustments can then be evaluated and discussed within the classroom, resulting in a complex and realistic learning environment of what truly happens within an organization or business- setting. Traditionally, simulations are often able to provide a simplified version of a business environment, but in a risk-free way for the student. The game will usually be simple to operate and learn, inexpensive, and sometimes provide some opportunity to work at a preferred pace. Finally, the more engaging the simulation, the more interesting, motivating and enjoyable it will become for the students. The game can be introduced by making involvement with the simulation competitive, providing regular feedback and promoting the atmosphere of a competitive and entertaining game rather than a conventional classroom exercise or worksheet. Moreover, this is a chance to establish teamwork tendencies and skills in the classroom as well when creating teams. The simulation used here consisted of ordering and distributing various bottled water products into 3 regions of a European country. Each product was defined uniquely with a material number and each team sold the same 6 products – therefore, initially the playing field was level and no one team had any advantage over another. Students were able to make decisions about pricing (per product) and how much to spend on marketing (product per region). Initially, all teams were provided with the same inventory of each product, so they could simply begin by selling that stock. They competed against other teams in class as to which team could make the most profit. Learning Outcomes and Student Satisfaction The theoretical underpinning for this study is the model known as the Unified Theory of Acceptance and Use of Technology - UTAUT (Venkatesh et al 2003). This model has been widely used in areas as diverse as customer behavior, mobile learning and online banking adoption. As such it is well suited to the use of technology in the classroom. The paper suggests that attitude is a central component in how (in this case) perceptions of the achievement of learning outcomes and their satisfaction with the simulation as a mechanism to enhance their learning will be affected. These concepts have been studied before in examining success or otherwise of technology in class room – see in particular Eon, Wen and Ashill (2006) and Alshare and Lane (2011). The degree to which students achieve the learning outcomes for a course is an indication of the degree of knowledge they have acquired from it. In this study, the SAP, ERP driven game was intended to provide the students with a base, underscored by some examples to help them understand the concepts of the course and to see those concepts in practice. The more salient concepts expected to be experienced in the simulation game included anchoring, bounded awareness, change blindness, framing and over confidence. Student satisfaction can be considered as the perceptions held by students of the quality of learning, supplemented by enjoyment and whether they would recommend the course (in this case the activity) to others. OVERVIEW OF CLASS AND METHODOLOGY The class into which the game was introduced was a senior year undergraduate class titled Management Decision Making. The learning objectives of the course, inter alia, were to understand and improve decision making processes, demonstrate the use of descriptive, normative and prescriptive approaches to managerial decision making, frame decisions by effectively describing and analyzing the problems, integrating internal and external business analysis, and generating and analyzing alternatives. Due to the concepts mentioned not being severely complex, and instead being based on psychological experiments or relevant literature and concepts, it can be assumed that students will gravitate towards wanting to learn more within the classroom. However, it is important to note that most students will largely lack any meaningful work experience prior being exposed to these topics within the classroom, so they might not be able to fully relate or connect to the implications of this subject-matter in the workforce. As a result, the course uses psychologically-based behavior and invites the students to consider the implications of this behavior in the workplace. The class presentation material is supplemented by instructor experience, anecdote and story, but there remains a considerable gap between the classroom, the exercise, and the “real world” in which they will be gathering and using information for managerial decision-making activities. For practical experience in managerial decision making, we turned to the ERPsim games. The data reported here was collected over three iterations of a course for seniors in Management Decision Making at one institution. A survey was used to collect the data – the survey remained unchanged over the three times the course was offered – Spring ’17, Fall ’17 and Spring ’18. The scales used to evaluate the constructs of Learning Objectives and Student Satisfaction were adapted from those developed and tested by Alshare and Land (2011). The constructs and measurement scale items, along with Cronbach alpha scores are given in Appendix A. As the students completed the survey in class all students completed the surveys, all of which were useable. The number of students involved in each group was Spring ’17: n = 77; Fall ’17: n = 36; and Spring ’18: n = 57. After completion of the surveys, the data was entered into Excel (and randomly checked for accuracy) and then transferred to SPSS which was used for analysis. For the most part, the survey employed a 5-point Likert scale with respondents replying Strongly Disagree through Strongly Agree. Common method bias was avoided by having respondents choose a point on a scale between extremes such as “Overall this game was Enjoyable…..…Dull” for some statements. Reliability analysis using Cronbach’s alpha showed all constructs measuring above the recommended 0.70. For part of the analysis the individual items were combined into one variable for each of the constructs measured. PRELIMINARY RESULTS In order to present a broad view of the results the individual items were combined into one variable for each of the three constructs under evaluation: Learning Objective 1 (how the students saw the simulation as demonstrating decision-making), Learning Objective 2 (the simulation covered the course material) and Student Satisfaction (essentially whether the students saw the Page 266 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 simulation as a quality addition to the course). The instructor (the same instructor taught all three iterations of the course and both sections offered each time) made slight changes to the way course was presented to the students and the classroom presentations. The results of the first iteration of the course were reported in [reference to be provided after the review process]. In that paper, as further work to be conducted, the authors identified several potential areas that could be changed to enhance perceived achievement of the learning objectives and student satisfaction. These included: • Using the simulation activities in prior classroom presentations to prepare the students for the game. As an example, anchoring could be explained as using the default prices as a starting point for pricing decisions, bounded awareness could be described in terms of the reports available, etc. • Students had mentioned confusion and the fast pace of the game in things they did not like about the exercise – slowing the game down or pausing it could aid in reducing these issues. • It is possible that a low grade based on performance was seen as a surrogate for not getting much out of the game in terms of learning objectives achieved and satisfaction – it would be possible to remove the assessment component. • The students were asked to read the material before playing the game, although many did not. Perhaps it would be beneficial to spend more time in this training area. For the subsequent use of the simulation, the instructor identified a number of the above areas for possible improvement. In particular the classroom presentation material was significantly amended to use examples from the simulation whenever possible. Slightly more emphasis was placed on training and pre-reading in order to reduce the confusion, the game was paused briefly to allow students to catch up and ask questions. Finally, the nexus between performance in the simulation and course assessment was removed. There was a statistically significant difference between groups as determined by a one-way ANOVA. A Tukey post hoc test revealed that for each of the variables of Learning Objectives and Student Satisfaction the students felt statistically significantly more positive in regard to the Fall 2017 course and course than they did for the Spring 2017 course (LO1, p = .000; LO2, p = .031; and SS, p =.000) and the Spring 2018 course (LO1 p = .001; LO2, p =.000; and SS, p =.000). There was no statistically significant difference between the Fall 2017 course and the Spring 2018 course groups (LO1 p = .615; LO2 p=.264; and SS p=.991 ). These changes in student perceptions of Learning Objectives and Student Satisfaction are represented graphically in Figure 1 below along Page 267 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 AACSB 2017. Initial Self Evaluation Report (Business) Outline and Guidelines. Available at www.aacsb.edu/-/media/aacsb/docs/accreditation/guides/ business-iser-template.ashx Alshare, K. A. & Lane, P. L. (2011). Predicting Student- Perceived Learning Outcomes and Satisfaction in ERP Courses: An Empirical Investigation. Communications of the Association for Information Systems: Vol. 28, Article 34. Bredemeier M & Greenblat C. (1981). The Educational Effectiveness of Simulation Games - A Synthesis of Findings. Simulation and Games 12(3), pp. 307-332 Cronan, T.P., Léger, P.-M., Robert, J., Babin, G. et Charland, P. (2012). Comparing Objective Measures and Perceptions of Cognitive Learning in an ERP Simulation Game: A Research Note. Simulation & Gaming, Vol. 43, pp. 461-480 Dick, G. N. and Syzmanski, R. (2013) “Integration of an SAP Simulation Game into an IS Course” Conference of the Southern AIS Savannah GA http://aisel.aisnet.org/ sais2013/7/ Eom, S., H. W, & N. Ashill. (2006). The Determinants of Students’ Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation. Decision Sciences Journal of Innovation Education (4)2, pp. 215–235. Léger, P.-M. (2006). “Using a Simulation Game Approach to Teach Enterprise Resource Planning Concepts” Journal of Information Systems Education, Vol. 17, pp. 441-448 Léger, P.-M., Robert, J., Babin, G., Pellerin, R. and Wagner, B. (2007). ERPsim, ERPsim Lab, HEC Montréal, Montréal, QC Léger, P.-M., Robert, J., Babin, G., Lyle, D., Cronan, P. and Charland, P. (2010). “ERP Simulation Game: A Distribution Game to Teach the Value of Integrated Systems”, ABSEL 2010 Conference (Association for Business Simulation and Experiential Learning), Little Rock (AK), 24-26 mars. CD-ROM Léger, P.-M., Charland, P., Feldstein, H.D., Robert, J., Babin, G. and Lyle, D. (2011). “Business Simulation Training: Guidelines for New Approaches in IT Training”, Journal of Information Technology Education, Vol 10, pp. 37-51 Seethamraju, R. (2011). Enhancing Student Learning of Enterprise Integration and Business Process Orientation through an ERP Business Simulation Game. Journal of Information Systems Education, Vol. 22(1), pp. 19-29 Venkatesh, V. et al. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly (27)3, 425–478 REFERENCES with the timing of the changes. In addition, a series of independent sample t-tests on the original construct items as shown in Appendix A (i.e. not the combined variables) indicated that there was a statistically significant difference between the Spring 2017 course and the Spring 2018 one for every one of the construct items measured. These differences were significant at least at the p =.01 level. Most items were significantly different when comparing the Spring 2017 and Fall 2017 courses except for the item “The ERP SIM game provided a good demonstration of many theories introduced in the course”. This item was significantly different (p = .001) from Fall 2017 to Spring 2018. This suggests the use of simulation examples in class lead to this improvement. DISCUSSION It seems clear that the simulation game is of benefit in teaching management decision-making. It also seems clear that the changes made to the course in which the simulation is used have resulted in at least a perception of a greater achievement of learning objectives and in student satisfaction, as demonstrated clearly by the use of simulation examples in the prior to simulation class presentation material. Interestingly, the removal of performance in the simulation from the course assessment does not seem to have made any difference to the perceptions. Further work would seem beneficial. Other aspects that the authors are considering for future iterations of the simulation include emphasizing more strongly the course learning objectives in the leadup to the simulation and putting more weight on the importance of the training material. In addition, the practice the authors have followed to date is running the simulation in consecutive class periods. Many students have reported confusion especially in the early stages. It may be beneficial to increase the time between the rounds rather than pause the simulation briefly. With the objective of developing a model of best practice for the use of this (and other simulations) in management education the authors are conducting further work relating to the causality of the findings for Learning Outcomes and Student Satisfaction and plan on providing a set of updated results to the conference. There is ample evidence that the simulations are beneficial in teaching business processes (see Dick and Syzmanski, 2013) This research indicates that they will very useful in the wider business education environment too. Specifically too, it seems that they may help business schools comply with the new AACSB directives and guidelines. The model of best practice that the authors intend to develop will include an evaluation of these simulations as a learning tool, with an identification of strengths and weaknesses, the steps that it is recommended to be undertaken by the relevant instructor in order to ensure that the students get the most out of the simulation, an identification of the ideal environment for its running and how best to structure feedback and reflection activities following its use. In summary, after comparing the data from the first use of the simulation in class to the most recent, it can be observed that a significant difference in perception of achieved learning objectives and student satisfaction occurred. In view of the higher degree of approval towards the game, this research indicates strong support in recommending it being introduced in other classrooms aiding in learning about management decision-making, obtaining a higher quality learning experience, and getting students to collaborate in learning objectives. This research suggests an additional study with the purpose of evaluation further modification of the approach used here could be beneficial as we move to a “model of best practice” for the simulation games being introduced to teach management courses in higher education. Page 268 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Construct / Item Cronbach’s Alpha .846 .742 .934 Learning Outcomes 1 The ERP SIM game demonstrated tactical decision making The ERP SIM game was a good practical example of decision making The ERP SIM game demonstrated operational decision making Learning Outcomes 2 I saw many of the concepts we covered in the course in the ERP SIM game The ERP SIM game provided a good demonstration of many theories introduced in the course I performed well in the ERP SIM game I expect a good grade for this part of the course Student Satisfaction I would recommend the ERP SIM to other students to learn about making decisions I am satisfied with the quality of the learning experience of the ERP SIM game I enjoyed the ERP SIM game The ERP SIM game is a good addition to the course APPENDIX A CONSTRUCT VALIDITY