Improving Student Cognition in Management Classes What role for Simulations? Page 269 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT In an attempt to improve understanding of managerial practices involved in decision-making, the authors of this paper turned to the ERP Simulation games developed by HEC Montreal. The past use of these games has primarily been limited to teaching business processes or SAP systems – on the face of it they also lend themselves to aiding understanding of management practices and providing an experience of decision-making. This paper reports on the first use of these games in this context and compares the results with the use of the games in another institution using them in their more traditional role of teaching business processes. The findings indicate that the simulation did not work as well in the management context as the business processes but provided indications of several areas where changes could be made to improve their efficacy. Keywords: Simulation, ERPSIM, decision-making, business processes NOTE: This is an extended abstract NOT a full paper BACKGROUND AND PURPOSE Video-game playing, a pastime previously possessing negative connotations, has seen an increase in use over the years for the development of student cognition. This general acceptance of somewhat unconventional teaching methods has led to universities implementing gamification. One of the ways to introduce simulations into the classroom is through the ERPsim games, developed in HEC Montreal (Leger, 2006, Leger et al 2007). Playing games that simulate the workplace environment gives students the opportunity to experience practice and theory together and provides an opportunity to acquire skills that are normally acquired through practice, all in a reduced time frame. The rapid feedback provided by the simulation and the professor can lead to strategic or tactical adjustments, which in turn can be evaluated and discussed in the classroom, resulting in a complex and realistic learning environment. The idea of using simulation games in the classroom is not new (Bredemeier and Greenblat, 1981) but has taken on a wider use with the advent of technology. In addition, recent changes to the AACSB guidelines has made it imperative for business schools desiring accreditation to ensure that their students are provided with some experience in evidence-based decision-making (AACSB 2017) The outcome of the decisions the students take in the simulation is provided as regular feedback to the participants, allowing them to assess the efficacy of their decisions and take appropriate remedial or additional action. On the face of it at least then, the use of the ERPSIM games would seem to meet the AACSB requirement. Prior research has examined the relationship between IS students’ gaming behavior and academic performance, how ERPsim enhances student learning outcomes in IS courses, and how ERPsim prepares IS students for a global environment. What becomes evident upon examination of the existing literature is that it largely substantiates gamification as an educational tool to teach business processes in IS courses (Leger et al 2010; Seethamraju 2011, Leger et al 2011; Dick and Szymanski, 2013), and widely disregards the use of ERPsim as a means to develop student understanding in other specialties. The practice of such gamification among other specialties could provide similar benefits, such as teaching the strategic application of business processes. Therefore, the immediate objective of this ongoing study is to determine whether or not the ERPsim, a simulation tool developed for IS courses, aids in student understanding of the managerial decision-making concepts in a similar way to prior use in teaching business processes and if not, to identify the differences with the aim of modifying the simulation in order to achieve better results. Improving Student Cognition in Management Classes What role for Simulations? Corey Frantz Northern Arizona University csf62@nau.edu Giulia Paulet Northern Arizona University gip8@nau.edu Geoffrey Dick Northern Arizona University gfdick@aol.com Michael Cuellar Georgia Southern University mcuellar@georgiasouthern.edu mailto:csf62@nau.edu mailto:gip8@nau.edu mailto:gfdick@aol.com mailto:mcuellar@georgiasouthern.edu Page 270 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 The ERPsim game used for this study provides a practical application of using information for decision making. To the best of the authors’ knowledge, the series of “experiments” reported in this and an earlier paper [reference to be added after the blind review] are the first time the ERPsim games have been used in a management decision-making class, although some work has been done in using the games to develop cognitive learning (Cronan et al 2012). In this paper, we will explore the possible correlation between the perceived benefits of ERPsim games by both IS students and students enrolled in a managerial decision-making course. The widely used theoretical model ATAUT (Unified Theory of Acceptance and Use of Technology) has been used to underpin this research (Venkatesh et al 2003). This model suggests that behavior is influenced by attitude which in turn is governed by a number of external components – see further work by Eon et al (2006) and Alshare and Lane (2011). Further work (an overview to be provided at the conference) will examine causality in the event of differences in student perceptions. Our initial hypothesis is that the two groups will see similar benefits from using the simulation. RESEARCH METHOD A survey method was used to collect the data reported here. The scales used to evaluate the constructs of Learning Objectives and Student Satisfaction were adapted from those developed and tested by Alshare and Land (2011). The constructs and measurement scale items, along with Cronbach alpha scores are given in Appendix A. The data was collected from students at two institutions – Group 1 were undertaking an MBA where the objective of using the simulation was to acquaint the students with business processes, Group 2 were senior students experiencing the simulation in a management decision-making class (the objective being to use the simulation to demonstrate practical decision-making). Minor amendments were made to the items in the survey to accommodate the objectives of each course. As the students completed the survey in class all students completed the surveys, all of which were useable. The n for each group is also given in Appendix A. That Appendix also provides details of the specific learning outcomes that students were expected to achieve by the sue of the simulation. After completion of the surveys the data was entered into Excel (and randomly checked for accuracy) and then transferred to SPSS which was used for analysis. For the most part, the survey employed a 5-point Likert scale with respondents replying Strongly Disagree through Strongly Agree. Common method bias was avoided by having respondents choose a point on a scale between extremes such as “Overall this game was Enjoyable…..…Dull” for some statements. Reliability analysis using Cronbach’s alpha showed all constructs measuring above the recommended 0.70. For the analysis the individual items were combined into one variable for each of the constructs measured. PRELIMINARY RESULTS Appendix B provides a series of histogram charts for the frequency distribution by group for each of the two Learning Outcome variables and for Student Satisfaction. A review of this data indicates that on the face of it, the students in the two groups feel differently about the efficacy of the simulation as a tool for enhancing course concepts. While students in both groups are generally satisfied, those in the Decision-Making class have a wider distribution, indicating that, for some, the simulation was not achieving its intentions. This is particularly evident in the Student Satisfaction variable where around 25 – 30% of the class found it less than a satisfactory addition to the course. This was not the case with the Business Processes group. A series of Mann-Whitney U tests were run to compare the means of the two groups, the null hypothesis being that there would be no significant difference between the groups reflecting the research question as to whether the simulation would perform equally well in both environments – in other words, the simulation would achieve the same degree of perception of achievement of learning objectives and student satisfaction in the management decision-making class as it did in the class teaching business processes. This non-parametric test was used due to the non-normal distribution of the data and the small sample size of Group 1. The results are given in Table 1 and 2 below Ranks Group N Mean Rank Sum of Ranks Av LO-3 1 26 43.58 1133.00 2 44 30.73 1352.00 Total 70 Av LO4-7 1 26 42.10 1094.50 2 44 31.60 1390.50 Total 70 Av SS8-11 1 26 45.75 1189.50 2 44 29.44 1295.50 Total 70 TABLE 1 Page 271 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 AACSB 2017. Initial Self Evaluation Report (Business) Outline and Guidelines. Available at www.aacsb.edu/-/media/ aacsb/docs/accreditation/guides/business-iser- template.ashx Alshare, K. A. & Lane, P. L. (2011). Predicting Student- Perceived Learning Outcomes and Satisfaction in ERP Courses: An Empirical Investigation. Communications of the Association for Information Systems: Vol. 28, Article 34. Bredemeier M & Greenblat C. (1981). The Educational Effectiveness of Simulation Games - A Synthesis of Findings. Simulation and Games 12(3), pp. 307-332 Cronan, T.P., Léger, P.-M., Robert, J., Babin, G. et Charland, P. (2012). Comparing Objective Measures and Perceptions of Cognitive Learning in an ERP Simulation Game: A Research Note. Simulation & Gaming, Vol. 43, pp. 461-480 Dick, G. N. and Syzmanski, R. (2013) “Integration of an SAP Simulation Game into an IS Course” Conference of the Southern AIS Savannah GA http://aisel.aisnet.org/ sais2013/7/ Eom, S., H. W, & N. Ashill. (2006). The Determinants of Students’ Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation. Decision Sciences Journal of Innovation Education (4)2, pp. 215–235 Kenny 2017 Léger, P.-M. (2006). “Using a Simulation Game Approach to Teach Enterprise Resource Planning Concepts” Journal of Information Systems Education, Vol. 17, pp. 441-448 Léger, P.-M., Robert, J., Babin, G., Pellerin, R. and Wagner, B. (2007). ERPsim, ERPsim Lab, HEC Montréal, Montréal, QC REFERENCES From this data, it can be concluded that both the Learning Outcomes and the level of Student Satisfaction were statistically significantly higher for the Business Process group than the Management decision-making group (Learning Outcome 1, U = 362, p = .009; Learning Outcome 2, U = 400, p = .036; Student Satisfaction, U = 305, p = .001). DISCUSSION AND CONCLUSION While the game was well received and appreciated by most students in the decision-making class, it is concerning that the exercise was not appreciated by around 1/4 to 1/3 of the class. With the objective of using the game in the best way possible to enhance learning in management education, and specifically in the decision-making class, it needs to be improved to a point where satisfaction and perceived achievement of learning outcomes emulates the results of using the game in its more traditional practice of teaching business processes. For example, a potential improvement could be around the course material which contained many decision-making terms and concepts that would have been unfamiliar to most undergraduate students. Examples are anchoring, bounded awareness, inattentional and change blindness, satisficing, confirmation bias, risk aversion and risk seeking action, competitive irrationality and impression management. All of these are represented (to varying degrees) in the game but many students may not have seen them. In the next offering of this course the presentation material will contain examples drawn from information used for decisions more closely resembling the information in the game. As an example, anchoring could be explained as using the default prices as a starting point for pricing decisions, bounded awareness could be described in terms of the reports available, etc. The authors are conducting further work relating to the causality of the findings for Learning Outcomes and Student Satisfaction and plan on providing a set of updated results to the conference. The simulation has continued to be used in subsequent sessions of course delivery and it is proposed to continue refining the way in which the game is introduced to the students and compare subsequent results with the results reported here. The simulation has been used successfully in business process classes for some time (see Dick and Syzmanski, 2013). The use of this simulation in management classes is still in introductory stages, but it would seem the simulation has the fundamentals in place for success. It is hoped that the results as reported by some students will be able to be perceived by most, if not all, and that these widely available simulations can enhance the achievement of course and program objectives and will enable business schools to use simulations such as these to improve the student curriculum and meet AACSB objectives. Test Statistics a Learning Outcome 1 Learning Outcome 2 Student Satisfaction Mann-Whitney U 362.000 400.500 305.500 Wilcoxon W 1352.000 1390.500 1295.500 Z -2.598 -2.098 -3.252 Asymp. Sig. (2-tailed) .009 .036 .001 a. Grouping Variable: Group TABLE 2 http://www.aacsb.edu/-/media/aacsb/docs/accreditation/guides/business-iser-template.ashx http://www.aacsb.edu/-/media/aacsb/docs/accreditation/guides/business-iser-template.ashx http://www.aacsb.edu/-/media/aacsb/docs/accreditation/guides/business-iser-template.ashx http://aisel.aisnet.org/sais2013/7/ http://aisel.aisnet.org/sais2013/7/ Page 272 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Léger, P.-M., Robert, J., Babin, G., Lyle, D., Cronan, P. and Charland, P. (2010). “ERP Simulation Game: A Distribution Game to Teach the Value of Integrated Systems”, ABSEL 2010 Conference (Association for Business Simulation and Experiential Learning), Little Rock (AK), 24-26 mars. CD-ROM Léger, P.-M., Charland, P., Feldstein, H.D., Robert, J., Babin, G. and Lyle, D. (2011). “Business Simulation Training: Guidelines for New Approaches in IT Training”, Journal of Information Technology Education, Vol 10, pp. 37-51 Seethamraju, R. (2011). Enhancing Student Learning of Enterprise Integration and Business Process Orientation through an ERP Business Simulation Game. Journal of Information Systems Education, Vol. 22(1), pp. 19-29 Venkatesh, V. et al. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly (27)3, 425–478 Appendix A Construct Validity Note: the items in italics are the items used in the Group 1 survey – the students using the simulation to under- stand business processes. Construct / Item Cronbach’s Alpha Group 1 (n = 26) Business processes Group 2 (n = 44) Decision-making Learning Outcomes 1 The ERP SIM game demonstrated tactical decision making The ERP SIM game demonstrated business processes The ERP SIM game was a good practical example of decision mak- ing The ERP SIM game was a good practical example of business pro- cesses The ERP SIM game demonstrated operational decision making The ERP SIM game demonstrated operational processes Learning Outcomes 2 I saw many of the concepts we covered in the course in the ERP SIM game The ERP SIM game provided a good demonstration of many theo- ries introduced in the course I performed well in the ERP SIM game I expect a good grade for this part of the course .930 .861 .838 .724 Student Satisfaction I would recommend the ERP SIM to other students to learn about making decisions I am satisfied with the quality of the learning experience of the ERP SIM game I enjoyed the ERP SIM game The ERP SIM game is a good addition to the course .949 .914 Page 273 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 APPENDIX B FREQUENCY DISTRIBUTION BY GROUP Group 1 Business Process Group 2 Decision-Making