Business Simulation Performance after Completing A Reflective Observations Module Page 31 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 ABSTRACT Many factors can contribute to a student’s successful learning experience when participating in a business simulation as part of their undergraduate curriculum. It’s important for faculty to understand the best methods to deploy simulation assignments to students. Is there a benefit to having students answer a set of reflective observation questions? Reflective observation questions ask students to think about their past actions, and to describe the effects their decisions have on their simulation performance. The main objective of this research: using comparative data from two classes to define the effectiveness of student performance within a business simulation after completing sets of reflective observation questions after each round of the simulation. INTRODUCTION The utilization of business simulation programs in undergraduate education has grown increasingly popular over the last 40 years. “The number of AACSB member schools using business simulation games in their degree programs remains high. Usage has now reached 97.5%.” (Faria 2008). The majority of these simulations help students gain an understanding through a virtual experience in operating a corporate firm within a competitive marketplace. Recently simulations have expanded their pedagogical approaches to better meet faculty’s need for the assessment of course learning goals. (Markulis Nugent Strang 2014). This research paper examines changes in individual student simulation performance after students answer a series of reflective observation questions. Reflective observations are a set of intriguing questions which incorporates student decisions to encourage the self-recognition of problems and guide students to create effective strategies to solve problems and improve performance. The goal of these questions is to provide learning opportunities for students and to create a richer learning experience. Reflective observation questions provide the potential for students to experience that 'ah-hah' moment where a clear understanding of key learning goals occurs. Most business based simulations lack any written components, the simulation used in this research has incorporated a set of reflective observation questions for students to complete at the end of each round of play, six rounds in total. These questions are related to the student’s decisions and performance within an experiential learning exercise. The questions help students think about their actions leading to realizations and understanding of course material. Reflective observation questions with in a business simulation seek to guide students to discover their own understanding of key concepts, learning goals and the integration of business acumen. After the completion of each major decision point within the simulation students are asked a series of questions that will have them consider the impact of their past and current decisions. Students often struggling with these questions, which ask them to describe their past performance issues and explain their current strategy to solve these problems and improve performance. LITERATURE REVIEW Simulations and reflective observations are two educational tools which enhance learning by integrating theoretical and practical experiences to create an interactive learning environment. Business simulations with reflective observations components give students an opportunity to be actively engaged in a learning environment. The goal of creating such an environment is to use the students’ immediate experiences and performance to reinforce the course learning goals and course objectives. “Experiential learning emphasizes learning in which the learner is directly in touch with the material being studied, rather than just watching it or reading, or hearing about it.” (Kolb 1984) & (Kohonen 2001). According to Woolfe’s paper experiential learning in workshops, experiential learning consists of the following four components. “One: Students are aware of the learning processes taking place, and which are empowering them to learn. Two: students are involved in a reflective experience which enables them to relate current learning to past, present and future. Three: the experience and content are personally significant: what is being learned and how it is being learned. Four: there is an involvement of the whole self: body, thoughts, feelings and actions, not just of the mind; in other words, the student is engaged as a whole person.” (Woolfe 1992) Two studies argue that traditional business schools lectures and homework problems do not prepare students for the real world, and do little to help them understand and deal with the uncertainties in real world business. (Lee 2002) & (Tesch 2008). Past studies have shown that traditional classroom activities fail to motivate the student as much as active experiential learning activities such as simulations. (Rosemary 2013). A paper by Martin was able to establish that students will learn abstract concepts and skills to Business Simulation Performance after Completing A Reflective Observations Module Michael Nugent SUNY Stony Brook Michael.Nugent@Stonybrook.Edu Page 32 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 solve real-world problems faster with experiential learning based simulations than traditional lecture based learning. (Martin 2000) Additionally the student that develops formal decision-making models and integrated them with information systems while playing a business based simulation are more likely to gains skills in effective data and information handling skills. (Zvi 2010) Research has suggested that business simulations are useful for developing problem solving skills. Xu’s paper hypothesized that students who complete simulations develop high-order knowledge and problem-solving skills by synthesizing diverse perspectives. (Xu 2010) Nadkarni’s research showed a connect to business simulations and students developing more complex models which enable students to more accurately identify key concepts and link these concepts in solving problems (Nadkarni 2003). This however does not show that students are retaining core business acumen while playing simulations because they are not asked to express what they have learned. David Kolb developed a theoretical model of experiential learning that included reflective observations as one of the four orientations to learning “Reflective observation, learning by perception, focuses on understanding the ideas, and situations by careful observation. The learner is concerned with how things happen by attempting to see them from different perspectives and relying on one’s thoughts feelings and judgement.” (Kolb 1984) “The literature on the relationship of teaching and learning suggests that students will comprehend better, retain longer, and become more interested in the material when active learning techniques are used.” (Poling & Hupp, 2009) Traditional college level business simulations are often criticized for being “a numbers game” it is not always clear to faculty what participants are learning when students are only entering numbers into boxes. How do educators know what students have learned from playing the business simulation and will students be able to translate these experiences to working for or running a real business. Boud, Keogh and Walker looked at this problem and concluded that faculty failed to close the loop by not asking students to think and reflect on their actions, strategies and results. The authors argue that students should be asked to think about how they played the game; students should be asked to explain how their decisions helped to solve problems and improve their performance. The authors continue to argue that the activity of reflection is often overlooked during simulations and that students are not granted an opportunity for reflection within the simulation experience. In 1984 research author Kolb referred to reflection observations as the most important part of the learning experience. Unfortunately most of today’s business simulations don’t include any reflective approaches to learning or reinforcing the basic business ideas their business simulations were designed to teach. Kolb suggested the importance of reflection in learning; he suggested that without reflection on our actions and outcomes we could not learn efficiently from new experiences. Kolb insisted that students were destined to repeat their mistakes if they did not adequately reflect on their experiences. Kolb’s learning model referred to reflection as a key element to experiential learning, he stated that reflection is the activity that connects action to theory and practice. (Kolb 1984) Research continued in this area of simulations and reflection, in 1992 Cookall discussed the importance of incorporating a reflection element to business simulations. He noted that this aspect was neglected in all of the current business simulation available on the market. He discovered that most business instructors don’t fully understand the importance of including reflective exercises into simulation assignments. (Crookall 1992) More recently Frederik Anseel used dual-process models to develop and test the effects of reflection strategy and feedback on learning. The results of his research showed using feedback and reflection activity enhanced performance improvement on a web- based work simulation better than feedback alone. He wrote: “Reflection without feedback did not lead to performance improvement. Further analyses indicated that the proposed reflection strategy was less effective for individuals low in learning goal orientation, low in need for cognition, and low in personal importance as they engaged less in reflection. Together these findings provide a theoretical basis for the future study of reflection in organizations and suggest a practical and cost-effective strategy for facilitating employee development after feedback in organizations.” (Anseel 2009) Feedback is an important consideration, in research by Chak Fu Lam conducted in 2011 showed that different levels of feedback have different effects on a student performance. The zoom simulation provides automated feedback; all students received the same automated feedback on their results. This automated feedback feature allows the effects of feedback to be constant for each student. One aspect of the reflective observation questions asked by the Zoom simulation is that they are mostly open ended, allowing students to reflect on their failures as well as successes. Student often will reflect their actions that were successful and speculate on what key decisions lead to enhanced performance. Research from Shmuel Ellis in 2014 “Showed that through systematic reflection, people can learn from both their successes and their failures.” His research helped establish the idea that reflection can be focused on success as well as failure to help student learn and retain key concepts. In 2001 Prensky’s book “Digital Game Based Learning”, discussed the lack of reflective exercises with in business simulations and concluded that the students are none-reflective and non-analytical about their performance and learning while participating in the current business simulations. Prensky also noted that students working with business simulations are very involved in the actions that game play requires, and become immersed in the experience; however they are not provided an opportunity to reflect on their experiences and learn how to connect key business concepts to their actions within the simulation. Students become good at solving small puzzles but lose sight of the larger goal. (Prensky 2001) Page 33 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 One reason the majority of today’s business simulations don’t include reflective elements is because business simulations are focused on simulating decisions made by a typical manufacturing business, not reinforcing core learning goals of a typical business course. The creators of simulations focus on developing “sets of strategic and tactical decisions that are logically and harmoniously related through algorithms and mathematical equations for the purpose of revealing the consequences of decision interaction in the form of financial statements and other reports.” (Goosen 1999) In 1995 Anderson and Lawton found that instructors who use business simulations in the classroom are often unaware or underestimate the value of reflection in the learning process. The business simulations of the time failed to provide opportunities for students to reflect on their actions and results. (Anderson & Lawton 1992) These findings are reinforced by Marshall in 2009 who discovered that faculty were frustrated in their attempts to connect students actions within the simulation to the learning goals and knowledge base of the course. He wrote “Students were quick to accept rules and make assumptions on how the simulation worked and rarely changed their game playing strategies to the core concepts and traditional approaches to effective decision making learned in the classroom.“ (Marshall 2009) Students, who develop their own understanding rather than absorbing professor’s points of view, will require learners to ask questions, generate and explore their own ideas, and build representations that organize their experiences (Fosnot 1996) Student learning experiences developed while playing a business simulation can elevate the student from knowing to thinking and applying in finance and accounting courses. (Springer 2004) Simulations can begin the process of moving students from talking about business knowledge to applying their own understandings of the subject matter. Springer hypothesized that learning at a higher level with simulations will be retained for a longer periods of time than traditional lectures that make up the majority of class activities of introductory business courses. (Springer 2004) Some Studies have tried to link GPA as a predictor of students’ perceptions of the improvement of their higher order cognitive skills; Bradley was able to show that “students with high GPAs are more inclined to demonstrate higher order cognitive skills than students with low GPAs.” (Bradley 2007) Lynch has conducted research on predicting performance in simulation gaming to factors likely to affect simulation performance. Factors that he suggests could be linked to simulation performance included academic ability, personality, motivation, and cohesiveness. Lynch found five studies which purported to support the conclusion that academic ability does predict successful performance in simulations, but he also identified five studies which disagree with that conclusion. (Lynch 1989) HYPOTHESIZES This research paper will investigate the following: Hypothesis (1): Students who complete the simulation and answer a series of reflective observation questions, at the end of each round, will have higher simulation scores, than students who only engage in the simulation. Hypothesis (2): Students who submit lengthier responses to reflective observation questions will perform significantly higher on the simulation. Hypothesis (3): The individual simulation scores will be a predictor of class grade. Hypothesis (4): Students who submit lengthier responses to reflective observation questions will have a higher course grade. Hypothesis (5): Students who complete the simulation reflective observation questions will perform significantly better on homework and exams, than students who only engage in the simulation. . SIMULATION For this study students worked with the “Zoom Business Simulation” from Jupiter Interactive. This simulation is comparable to: Capsim’s “Capstone simulation”, Marketplace’s “Business simulator”, and Smartsim’s “Mikes Bikes simulation”. All four of these commonly used simulations have a similar design; students make a series of top management decisions in various business areas, such as: sales, marketing, operations, finance and human resources. Students compete directly against each other for market share. After each round these simulations provide feedback and reports on performance in various measures. In the Zoom Business Simulation students research and design, forecast, market, produce and manage four vehicle class products: Economy, Sedan, Truck, and Luxury automobiles. METHOD This study was based on data collected from two sections of Finance taught at Stony Brook University’s College of Business. The first section of 208 students occurred during the fall semester of 2011 and the second section of 158 students occurred during the spring semester of 2012. In both course sections students competed individually against each other in six rounds of the Zoom simulated business environment. The spring 2012 section completed the reflective observations module; this module was not available to the fall students. Regression analysis and two sample T-tests were conducted on the data to search for significance and relevance between Page 34 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 independent and depended variables. It should be noted that I was the instructor for both sections. This research paper was conceived two years after the last day of the spring 2012 section, May 19th 2012. I collected the data with the help of Jupiter Interactive and Stony Brook University in September of 2014. Both sections were the same subject: Finance, all course materials were identical, chapters covered, textbook, homework, in class assignments, and exams. STUDENT COMPOSITION Academic Standing Fall 2011 Academic Standing Spring 2012 Year Count Percent Spring 2011 Count Percent Freshman 24 11.54% Freshman 28 17.72% Junior 48 23.08% Junior 59 37.34% Senior 58 27.88% Senior 36 22.78% Sophomore 78 37.50% Sophomore 35 22.15% Grand Total 208 100% Grand Total 158 100% EXHIBIT 1 ACADEMIC STANDINGS EXHIBIT 2 GENDER Gender Fall 2011 Gender Spring 2012 Fall 2010 Count Percent Spring 2011 Count Percent Female 97 46.63% Female 76 48.10% Male 111 53.37% Male 82 51.90% Grand Total 208 Grand Total 158 EXHIBIT 3 FULL/PART TIME Full/Part Time Fall 2011 Full/Part Time Spring 2012 Fall 2010 Count Percent Spring 2011 Count Percent Full Time 159 76.44% Full Time 112 70.89% Part Time 49 23.56% Part Time 45 29.11% Grand Total 208 Grand Total 158 Both sets of students are comparable, no major differences were evident. RESULTS Hypothesis (1): Students who complete the simulation and answer a series of reflective observation questions, at the end of each round, will have higher simulation points, than students who only engage in the simulation. Simulation points are a measure of performance within the Zoom simulation, based on the financial performance of each simulated firm. Page 35 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 Two-sample t test: Hypothesis test: Simulation Points by Reflective Observations: Comparing whether the average difference between two groups (Fall 2011 - No Reflective Observations) students & (Spring 2012 - Yes Reflective Observations) students, is significant or due instead to random chance. This test is using a 95% confidence interval and a significance level of 5%. Null Hypothesis 1 (H0): the difference between the (No Reflective Observations) and (Yes Reflective Observations) simulation scores equals zero. Alternative Hypothesis 1 (Ha): the difference between the observed mean of the (No Reflective Observations) and (Yes Reflective Observations) simulation scores is not equal to zero. Based on the results: Ha: diff < 0, Pr(T < t) = 0.001, the null hypothesis is rejected. The difference between the two groups is significant. Additionally the average simulation points for the class that completed the reflective observations increased by 380 points or 14.69% and the standard deviation decreased by 131, a 25% reduction. \ Hypothesis (2): Students who submit lengthier responses to reflective observation questions will perform significantly higher on the simulation than students who submit shorter responses. Characters were used to measure length of response; each character in every word written would count as one. For example the word “work” would count as four characters. A Linear regression model using simulation points as the dependent variable and characters as the independent variable shows significance between the two sets of data (R2=.219, P=.001) The two are positively correlated. POINTS BY REFLECTIVE OBSERVATIONS STATISTICS Variable N Mean SD SE No Reflective Observations 208 2587 530 37 Yes Reflective Observations 158 2967 399 32 Difference 380 POINTS AND REFLECTIVE OBSERVATIONS BOX PLOT REGRESSION RESULTS POINTS BY CHARACTERS Variable Coef. Std Err P>t R-squared F Prob>F Char .038 .005 .001 .219 43.79 .001 Page 36 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 Running a multivariate regression analysis to include grade did not enhance the R-square in a significant way. . Hypothesis (3): The individual simulation score will be a predictor of class grade. A Linear regression model with simulation points as the dependent variable and course grade as the independent variable shows no significance between the two sets of data (R2=.001, P=.983). Course grade does not include the simulation grade. Hypothesis (4): Students who submit lengthier responses to reflective observation questions will have a higher course grade. SCATTER PLOT POINTS AND CHARACTERS MULTIVARIATE REGRESSION RESULTS POINTS BY CHARACTERS, COURSE GRADE Variable Coef. Std Err P>t R-squared F Prob>F Char .029 .0057 .0001 .221 26.51 .0001 REGRESSION RESULTS POINTS COURSE GRADE Variable Coef. Std Err P>t R-squared F Prob>F Grade .075 3.45 .983 .001 0.001 .9826 SCATTER PLOT POINTS AND COURSE GRADE Page 37 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 A Linear regression model with Class Grade as the dependent variable and characters as the independent variable shows no sig- nificance between the two sets of data (R2=.003, P=.476) Hypothesis (5): Students who complete the simulation reflective observation questions will perform significantly better on homework and exams, than students who only engage in the simulation. Two-sample t test: Hypothesis test: Course Grade by Reflective Observations: Comparing whether the average difference between two groups (Fall 2011 - No Reflective Observations) students & (Spring 2012 - Yes Reflective Observations) students, is significant or due instead to random chance. The influence of simulation assignment grade was backed out of the course grade. This test is using a 95% confidence interval and a significance level of 5%. Null Hypothesis 1 (H0): the difference between the (No Re- flective Observations) and (Yes Reflective Observations) simulation scores equals zero. Alternative Hypothesis 1 (Ha): the differ- ence between the observed mean of the (No Reflective Observations) and (Yes Reflective Observations) simulation scores is not equal to zero. Based on the results: Ha: diff < 0, Pr(T < t) = 0.001, the null hypothesis is rejected. The difference between the two groups is significant. After backing out the influence of grade, for the simulation assignment, for both sections, the average grade for (Yes Reflective Observations) was 81.91 compared to an average grade for (No Reflective Observations) of 78.02. Based on this statistical test the difference of 3.89 is significant. The previous eight sections of this course with me as the instructor had an average grade of 78.1 with a standard deviation of 1.013 REGRESSION RESULTS COURSE GRADE & CHARACTERS Variable Coef. Std Err P>t R-squared F Prob>F Char .0001 .0001 .476 .003 .51 .4761 SCATTER PLOT COURSE GRADE & CHARACTERS Page 38 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 DISCUSSION OF RESULTS The analysis of Hypothesis (1) did show that students who completed the reflective observation questions performed significantly better on the simulation, than students who did not answer the reflective observations questions. The Two Sample T test produced a P=.001, showing significance. The mean increase in scores for students who completed the reflective observation questions was 380 points higher than students who did not complete the questions. In addition the students who completed the reflective observations questions had a tighter dispersion of simulation points based on a smaller standard deviation of 399 vs. 529. These results suggest that a student who completes the reflective observation questions have a significant advantage and score higher on the simulation. The linear regression model for Hypothesis (2) compared the simulation points earned and the number of characters each student had written. Each letter in a word equals one character, from example the word (home) would be considered 4 characters. This model produced a significant (F=43.79, P=.001) and positive coefficient of .038, for every additional character written a .038 point increase in score can be expected. The R2 of .219 for this model demonstrated as strong influence between these two variables. This model analyzes data only from the group of students who completed the reflective observations questions; the POINTS AND REFLECTIVE OBSERVATIONS BOX PLOT DESCRIPTIVE STATISTICS Variable Mean Median Mode Standard Deviation Minimum Maximum N Dependent Variable Points 2751 2837 3026 513 1225 3895 366 Independent Variables Reflective Observations .43 0 0 .50 0 1 366 Characters 12328 0 0 14522 0 36645 366 Course Grade 79.70 81 90 10.54 55 94 366 Stock Price 210 177 156 110.53 53 641 366 Page 39 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 positive coefficient of .038 has a significant impact on points when you consider the average number of characters written per student is 28557. This implies that students who had written longer responses to express their reflective thoughts more completely had an advantage over students who used fewer characters. The linear regression model failed to support Hypothesis (3.) The linear regression resulted in a (R2=.001, P=.983) no significance was found to support the idea that, the grade a student receives in the course is a predictor the amount of simulation points. Course grade does not include the simulation grade. This regression model included all 366 students from both sections. Some might argue that students who were “better students” as measured by course grade, would naturally have higher simulation scores. Similarly Hypothesis (4) failed to show significance; the linear regression model explored the possibility that the amount of characters students wrote would be a predictor of class grade. This model only used data from the students who completed the reflective observations module; additionally course grade does not include the simulation grade. The regression produced a (R2=.003, P=.476) clearly the length of students reflective observation responses is not a good predictor of class grade. This is an important result, refuting the idea that students who wrote more characters on the reflective observations were doing so because they were better students as demonstrated by their overall course grade and would naturally have a higher simulation scores as well. The failure to show significance here strengthens the findings in hypothesis (2) The results of Hypothesis (5) shows support for the concept that student who are challenged by reflective observation questions will develop a better understanding of learning goals and will perform significantly better on homework and exams. When comparing, the average difference in course grade between students who completed the reflective observations module to those who did not, we find an average increase in course grade of 3.89 points or an improvement of 5%. The influence of the grade for the simulation is backed out of the course grades used in this model. It should be noted that the average course grade for the past eight sections of this course with me as the instructor was 78.1 with a standard deviation of 1.01, after backing out the effect of the simulation on the course grade. A multivariate regress model was considered to review the relationship between the independent and dependent variables; however this model failed to show any significant increase in R2. In summary the research suggests students who complete the reflective observations questions at the end of each round of a six round business simulation will earn significantly higher simulations points, as well as higher homework and exam scores. In addition students who write lengthier reflective observations as measured by characters will have higher simulation points, but not higher course grades. This finding is important for those who would suggest that students who write a higher volume of characters are naturally better students in their class room efforts and would also score higher on a simulation assignment. It’s interesting to discover that no clear relationship was found using individual simulation points as a predictor of class grade for all students. However course grades do improve significantly when you look at only the students who completed the reflective observations module. The relationship between course grade and the simulation points is important because it demonstrates how a business simulation that includes reflective observations can help students to better their understanding of the course material. The results suggest improvement in the comprehension and retention of the course concepts contained within a business simulation. CONCLUSION As a pedagogy tool simulations are an innovative and active learning experience. Students apply business concepts within a simulated real world environment. Simulations have a tremendous potential to enhance a student’s comprehension of the fundamentals of business. Research on improving the effectiveness of student performance and the best practices of deploying simulations in the classroom can have a significant impact on student learning. The results of this research will be valuable to both faculty and simulation designers. Gaining new knowledge in this area and examining how students can improve simulation performance and enhance knowledge retention after completing the business simulation is critical to the success of future students. The findings of this research, which demonstrated that answering reflective observation questions at the end of each round of a business simulation can enhance student performance in the business simulation, as measured by simulation points, as well as homework and exams. These results will be beneficial to institutions and faculty who incorporate reflective observation modules in future classroom implementations of business simulations. LIMITATIONS • This study only looked at one simulation game; different simulations may have different results between student who answer reflective observation questions and those who do not. • The study looked at only two sections of finance, taught at different points in time, by the same instructor. However the students in each class were comparable; they did not have any major differences. • Results were not compared to other groups of students, at different schools. • The reflective observations were not separated in the categories of learning from successful and unsuccessful decisions. • Homogeneity comparison was not conducted for student grade level, as part of the student comparison section. Page 40 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 RECOMMENDATIONS FOR FUTURE STUDY • I suggest repeating this study using other popular simulation games to see if the results can be confirmed across platforms. • Measuring the progression of improvement of simulation points by round, to see if reflective observations contribute more to an increasing of score as the rounds progress. • Replicating this study on larger scale by including other schools and instructors, to further confirm the results of this research. • Future studies comparing the effectiveness of reflective observations with other academic assignments. APPENDIX A: REFLECTIVE OBSERVATION QUESTIONS FROM THE SIMULATION • Describe your new strategy for redesigning your economy class vehicle. How will this help your company's performance? • What factors influenced your calculation of the sales forecast for the economy class vehicle? Was your design and forecast successful last round? • Describe your new strategy for redesigning your sedan class vehicle. How will this help your company's performance? • What factors influenced your calculation of the sales forecast for the sedan class vehicle? Was your design and forecast successful last round? • Describe your new strategy for redesigning your truck class vehicle. How will this help your company's performance? • What factors influenced your calculation of the sales forecast for the truck class vehicle? Was your design and forecast successful last round? • Describe your new strategy for redesigning your luxury class vehicle. How will this help your company's performance? • What factors influenced your calculation of the sales forecast for the luxury class vehicle? Was your design and forecast successful last round? • What is your overall demographic and advertising strategy for each vehicle class? • What strategies did you use when allocating your advertising investment dollars per vehicle class? Was your allocation successful last round? • What is the capacity utilization percentage for each vehicle class (production units/total unit capacity) this round? • Why is this important? • What is your new strategy for operational investments for each vehicle class? • How will your current operational investment affect your company's performance? • What is your strategy for additional funds this round? • What is your strategy for retiring funds this round? • Are the Pro Forma financial statements reflective of your strategic plans? • Has your company's strategic decisions improved your sales, cash surplus, net profits and profit margins this round? Page 41 - Developments in Business Simulation and Experiential Learning, Volume 45, 2018 Anderson, P. H., & Lawton, L. (2003). The optimal timing for introducing business simulations. Developments in Business Simulation and Experiential Learning: Vol. 20, 1-4. Anseel, F., Lievens, F., & Schollaert, E. (2009). Reflection as a strategy to enhance task performance after feedback. Organizational Behavior And Human Decision Processes, 11023-35. doi:10.1016/j.obhdp.2009.05.003 Boud, D. Cohen, R. and Walker, D. (1985) Using Experience for Learning, Open University Press, Buckingham. Boud, D. Koegh, R. and Walker, D. (1985) Reflection, truing experiences into learning. London Kogan Page; New York; Bradley, R. V., Sankar, C. S., Clayton, H. R., Mbarika, V. W., & Raju, P. K. (2007)., A Study on the Impact of GPA on Perceived Improvement of Higher-Order Cognitive Skills. Decision Sciences Journal Of Innovative Education 5(1), 151-168. Ellis, S., Carette, B., Anseel, F., & Lievens, F. (2014). Systematic Reflection: Implications for Learning From Failures and Successes. Current Directions in Psychological Science, 23(1), 67–72. Faria, A. J. (2008) Business Simulation Games: Current Usage Levels--An Update Simulation Gaming, Sep 2008; vol. 29: pp. 295-308 Faria, A. J. (2000). The changing nature of simulation Developments in Business Simulation and Experiential Learning, Volume 34, 2007 160 research: A brief ABSEL history. Developments in Business Simulation and Experiential Learning, Vol. 27, 84-90. Fosnot, C. T. 1996. Constructivism: A psychological theory of learning. In Constructivism: Theory, Perspectives, and Practice, edited by C. T. Fosnot. New York, NY: Columbia University Goosen, Kenneth R., Jensen, Ron, Wells, Robert (2001) Purpose and Learning Benefits of Simulations: A Design and Development Perspective. Simulation Gaming, Mar 2001; vol. 32: pp. 21-39 Kolb, D. 1984. Experiential learning. Englewood Cliffs, NJ: Prentice Hall. Kohonen, V. 2001. Towards experiential foreign language education. In Kohonen, V.R. Jaatinen, P. Kaikkonen and J. Lehtovaara, Experiential learning in foreign language education. London: Pearson Education, 8-60. Lam, C. F., DeRue, D. S., Karam, E. P., & Hollenbeck, J. R. (2011). The impact of feedback frequency on learning and task performance: Challenging the “more is better” assumption. Organizational Behavior and Human Decision Processes, 116(2), 217–228 Lee, S., Koh, S., Yen, D., & Tang, H. -. (2002). Perception gaps between IS academics and IS practitioners: an explanatory study. Information & Management, (1). 51. Lynch, R., & Michael, T. (1989). Predicting individual decision making performance in a business simulation. Developments in Business Simulation & Experiential Exercises, 16, 182-187. Martin A., (2000) The design and evolution of a simulation/ game for teaching information systems development, Simulation & Gaming: An Interdisciplinary Journal 31:4 445–463 Marshall, M. (2009) Simulation in Management Education, Simulation Education 11(4) 265-273 Nadkarni, S. (2003). Instructional methods and mental models of students: An empirical investigation. Academy of Management Learning and Education, 2, 335–351. Poling, D. A., & Hupp, J. M. (2009). Active learning through role playing. College Teaching, 57(4), 221–226. Prensky, M. (2001). Digital Game Based Learning. New York, McGraw-Hill. Rosemary, G., Robert, A., & James E., D. (2013). Games, motivation and learning: A research and practice model. Simulation & Gaming 33(4. Springer Carol and Borthick A. (2004) Business Simulation to Stage Critical Thinking in Introductory Accounting: Rationale, Design, and Implementation. Issues in Accounting Education: August 2004, Vol. 19, No. 3, pp. 277-303. Tesch, Braun, Crable, (2008) An examination of employers' perceptions and expectations of is entry-level personal and interpersonal skills, Information Systems Education Journal 6:1 1–16. Walker , D. (1985). Writing and Reflection. Reflectoin: Turing experience into learning. New York: Nichols Publishing Company. Wolfe, Joseph (1997) The Effectiveness of Business Games in Strategic Management Course Work Simulation Gaming, Dec 1997; vol. 28: pp. 360-376 Woolfe, R. 1992. Experiential learning in workshops. In Hobbs, T. (ed.) 1992. Experiential training: practical guidelines. London: Tavistock/ Roudledge, 1-13. Xu, Y., & Yang, Y. (2010). Student learning in business simulation: an empirical investigation. Journal of Education for Business, (4). 223. Zvi-Ben, T. (2010). The efficacy of business simulation games in creating Decision Support Systems: An experimental investigation. Decision Support Systems, 4961-69. doi:10.1016/j.dss.2010.01.002 REFERENCES