SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? Developments in Business Simulations and Experiential Learning, Volume 32, 2005 SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? Peter M. Markulis SUNY Geneseo markulis@geneseo.edu Daniel R. Strang SUNY Geneseo strang@geneseo.edu ABSTRACT This paper examines the relationship between predictor and performance variables in a typical computerized simulation. Most simulations require the users to make a set of predictions on various performance variables. The question is whether there is any relationship between what users predict and how they ultimately perform. The authors collected data on several predictor variables and compared them to a performance variable for one semester of simulation play for two groups of undergraduate business majors at a mid-sized college. The authors also surveyed a group of business faculty who typically use simulations in their classes to ascertain their views on the relationship between predictor variables and performance variables. The authors found that as simulation play progressed, those student teams which had less variance between their predicted sales and actual sales tended to have better simulation performance than those teams which had a greater variance between their sales predications and actual sales results. In terms of the survey of instructors who use computerized simulations, one of the more interesting results was that while most respondents stated that they asked their students to establish predictor variables; few indicated what purpose this served or why they asked students to establish predictor variables. More research is needed to both understand why simulation instructors ask users to formally establish predictions and what role predictor variable have in helping students with performance. INTRODUCTION & LITERATURE REVIEW Simulations and computerized management games have now been investigated thoroughly in terms of their ability to enhance learning (Anderson & Lawton, 1992; Gopinath & Sawyer, 1999; Goosen, 2002; Hornaday & Curran, 1996; Vaidyanathan & Rochford, 1998; and Wolfe & Luethge, 2003). Meta and comprehensive studies are beginning to appear suggesting that the field is reaching pedagogical maturity. Despite this body of research, the authors are not aware of any study which looked at the relationship between student’s simulation performance and their ability to make good predictions. For example, in their study of the relationship between formal planning and simulation performance in student teams, Hornaday and Curran state, “No attempt was made to determine whether the student plans accurately predicted what actually occurred during the course of the simulation. Lastly, organizational performance was the standard of success. The study did not investigate methods used by student teams to implement their plans,” (1996, 210). A paper presented in 1985 by Markulis and Strang proposed the use of a decision support system (DSS) to aid student teams in decision making for simulations. One of the ways in which a DSS can be helpful is by analyzing the relationship between predictor variables and performance. Yet the authors do not know if any student players are using a DSS or similar mode of analysis to enhance predictor success. (By predictor success, we mean the ability to continually shorten the variance between what a team predicts on a set of variables and what they actual receive after succeeding rounds of play). This led the authors to raise the following questions: • Do students who predict well, perform well? • What predictor variables serve as “better” predictors of performance? • How do students determine values for their predictor variables? • Do students “use” (and if so, how) the information they gain from performance to make decisions on future predictor variables? • Do students “learn” anything from the predictor/performance relationship? While this paper cannot adequately address all of the questions raised above, it does attempt to address the first 208 mailto:markulis@geneseo.edu mailto:strang@geneseo.edu Developments in Business Simulations and Experiential Learning, Volume 32, 2005 two. Section 2 of the paper discusses questions 1 & 2 above, while section 3 reports on the results of a survey of business instructors who use simulations. The authors were interested in what simulation instructors did—if anything— with the predictor/performance relationship. SECTION 1 The authors collected data in two upper level undergraduate classes of business strategy. Each class functioned as separate industry using the business game; DECIDE (Pray, 1980). There were 8 teams in industry 1 and 7 teams in industry 2. Each team consisted of three students, except for one team in industry 2 in which there were four students on the team. Students were randomly selected for team membership to help control for selection bias. Each team made a set of 26 decisions for their respective firm for a period of 6 weeks. Students made predictions on several variables. As part of their decision making process, student teams made predictions on several variables, including sales volume, cash balance, etc. The authors collected the predictions for each team throughout the semester and compared each team’s results to their own predictions. One could argue that students who predicted well would perform well. DATA MEASURES There are obviously several ways in which to compare prediction and performance. The authors devised a calculation procedure as follows. For each period of play each team made a set of predictions. The authors chose to investigate three salient predictions for analysis, namely, sales in dollars, income after tax (profit), and cash balance. The authors took the predictions on the three salient variables and compared them to the actual values for these variables that occurred during each round of simulation play. The authors then measured the difference between actual and predicted and gave the team a score. The team which was closest in matching their prediction with their actual results received a rank of 1, and the other teams were ranked accordingly, with 7 or 8 (depending on the number of teams in the industry) being the team with the greatest discrepancy between prediction and result. This measure was termed the variance ranking. This variance ranking was then compared to the team’s simulation performance score. Like many other simulations, DECIDE ranks team performance in terms of the team’s stock market value. Tables 1 & 2 show the results for the variance rankings between sales predictions and performance for both industries. TABLE 1: Performance on “Ranked Sales Variance” and Overall Ranking for Industry 1 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng Firm 1 4 5 Firm 1 1 4 Firm 1 2 1 Firm 1 1 1 Firm 1 1 1 Firm 2 7 1 Firm 2 6 1 Firm 2 8 2 Firm 2 8 5 Firm 2 7 7 Firm 3 3 2 Firm 3 5 2 Firm 3 5 5 Firm 3 3 2 Firm 3 5 2 Firm 4 8 8 Firm 4 1 8 Firm 4 1 7 Firm 4 2 6 Firm 4 1 5 Firm 5 5 4 Firm 5 3 5 Firm 5 6 4 Firm 5 5 3 Firm 5 3 3 Firm 6 7 7 Firm 6 7 6 Firm 6 3 6 Firm 6 4 7 Firm 6 4 6 Firm 7 1 6 Firm 7 8 7 Firm 7 7 8 Firm 7 7 8 Firm 7 8 8 Firm 8 1 3 Firm 8 4 3 Firm 8 4 3 Firm 8 6 4 Firm 8 6 4 Spearman’s rho = .325 Spearman’s rho = .646 In the instance of ties the two firms are given the same ranking. 209 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 TABLE 2: Performance on “Ranked Sales Variance” and Overall Ranking for Industry 2 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of S al es O ve ra ll R an ki ng Firm 1 3 2 Firm 1 2 4 Firm 1 5 4 Firm 1 5 4 Firm 1 2 5 Firm 2 2 4 Firm 2 3 5 Firm 2 1 3 Firm 2 1 2 Firm 2 1 2 Firm 3 6 1 Firm 3 4 2 Firm 3 6 2 Firm 3 3 3 Firm 3 5 3 Firm 4 1 3 Firm 4 5 1 Firm 4 2 1 Firm 4 2 1 Firm 4 3 1 Firm 5 5 5 Firm 5 6 3 Firm 5 7 5 Firm 5 4 5 Firm 5 6 6 Firm 6 3 6 Firm 6 1 6 Firm 6 4 7 Firm 6 7 7 Firm 6 7 7 Firm 7 7 7 Firm 7 7 7 Firm 7 3 6 Firm 7 6 6 Firm 7 4 4 Spearman’s rho = .252 Spearman’s rho = .678 The authors computed a Spearman’s rho to measure the degree of concordance of the rankings between ranked sales variance and simulation performance period 2 and for period 6. Period 1 was not used as the authors believed this period served more as a “trial” period than an actual competitive play period. As can be seen, as the play progressed from period 2 to period 6, teams who performed better had less variance between their predictions on sales (in dollars) and their actual sales (in dollars) results. This is particularly true for Industry 2 where the initial Spearman’s rho was .252 for period 2 and the value of Spearman’s rho rose to .678. for period 6. The authors then investigated relationship between predicting net profits and actual performance. Tables 3 & 4 contain these results for Industries 1 & 2 respectively. Table 3 shows that there was virtually no relationship between the variable of ranking for Income After Tax Variance and the team’s ranking based upon simulation performance for period 2. However, for period 6, there was considerably less difference between those teams which accurately predicted their After Tax Income and those that performed well, despite the fact that the overall Spearman’s rho was still only .595. For teams in industry 2 the degree of concordance was also small initially, but also rose by period 6. TABLE 3: Performance on “Ranked Income After Tax Variance” and Overall Ranking for Industry 1 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng Firm 1 6 5 Firm 1 2 4 Firm 1 2 1 Firm 1 2 1 Firm 1 3 1 Firm 2 7 1 Firm 2 6 1 Firm 2 4 2 Firm 2 8 5 Firm 2 7 7 Firm 3 4 2 Firm 3 3 2 Firm 3 6 5 Firm 3 3 2 Firm 3 5 2 Firm 4 8 8 Firm 4 1 8 Firm 4 1 7 Firm 4 1 6 Firm 4 2 5 Firm 5 3 4 Firm 5 5 5 Firm 5 7 4 Firm 5 5 3 Firm 5 1 3 Firm 6 5 7 Firm 6 7 6 Firm 6 3 6 Firm 6 4 7 Firm 6 4 6 Firm 7 1 6 Firm 7 8 7 Firm 7 8 8 Firm 7 7 8 Firm 7 8 8 Firm 8 1 3 Firm 8 4 3 Firm 8 5 3 Firm 8 6 4 Firm 8 6 4 Spearman’s rho = .193 Spearman’s rho = .595 In the instance of ties the two firms are given the same ranking. 210 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 TABLE 4: Performance on “Ranked Income After Tax Variance” and Overall Ranking for Industry 2 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e In co m e A fte r T ax O ve ra ll R an ki ng Firm 1 3 2 Firm 1 1 4 Firm 1 5 4 Firm 1 5 4 Firm 1 4 5 Firm 2 2 4 Firm 2 2 5 Firm 2 1 3 Firm 2 1 2 Firm 2 2 2 Firm 3 6 1 Firm 3 4 2 Firm 3 7 2 Firm 3 4 3 Firm 3 5 3 Firm 4 1 3 Firm 4 5 1 Firm 4 3 1 Firm 4 2 1 Firm 4 3 1 Firm 5 5 5 Firm 5 7 3 Firm 5 6 5 Firm 5 6 5 Firm 5 7 6 Firm 6 4 6 Firm 6 3 6 Firm 6 4 7 Firm 6 7 7 Firm 6 6 7 Firm 7 7 7 Firm 7 6 7 Firm 7 2 6 Firm 7 3 6 Firm 7 1 4 Spearman’s rho = .321 Spearman’s rho = .642 The third prediction variance that the authors measured and evaluated focused on the firm’s cash balance. Tables 5 and 6 show the results for this measure for industries 1 and 2. For both industries it is apparent that the increases in the Spearman’s rho values were less pronounced for the predictor variable cash balance than for the other two predictor variables that were analyzed. Specifically, for industry 1 the Spearman’s rho values changed from .395 to .452 for periods 2 and 6. Similarly, the Spearman’s rho values for industry 2 changed from .464 to .678 for periods 2 and 6. TABLE 5: Performance on “Ranked Cash Balance Variance” and Overall Ranking for Industry 1 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng Firm 1 4 5 Firm 1 1 4 Firm 1 3 1 Firm 1 1 1 Firm 1 2 1 Firm 2 6 1 Firm 2 5 1 Firm 2 2 2 Firm 2 8 5 Firm 2 5 7 Firm 3 3 2 Firm 3 2 2 Firm 3 5 5 Firm 3 4 2 Firm 3 7 2 Firm 4 8 8 Firm 4 6 8 Firm 4 1 7 Firm 4 2 6 Firm 4 3 5 Firm 5 5 4 Firm 5 3 5 Firm 5 7 4 Firm 5 5 3 Firm 5 1 3 Firm 6 7 7 Firm 6 7 6 Firm 6 4 6 Firm 6 3 7 Firm 6 4 6 Firm 7 1 6 Firm 7 8 7 Firm 7 8 8 Firm 7 7 8 Firm 7 8 8 Firm 8 1 3 Firm 8 4 3 Firm 8 6 3 Firm 8 6 4 Firm 8 6 4 Spearman’s rho = .395 Spearman’s rho = .452 In the instance of ties the two firms are given the same ranking. 211 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 TABLE 6: Performance on “Ranked Cash Balance Variance” and Overall Ranking for Industry 2 Period 2 Period 3 Period 4 Period 5 Period 6 R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng R an ki ng B as ed o n V ar ia nc e of C as h O ve ra ll R an ki ng Firm 1 3 2 Firm 1 7 4 Firm 1 5 4 Firm 1 3 4 Firm 1 2 5 Firm 2 2 4 Firm 2 1 5 Firm 2 1 3 Firm 2 1 2 Firm 2 1 2 Firm 3 5 1 Firm 3 2 2 Firm 3 6 2 Firm 3 5 3 Firm 3 5 3 Firm 4 1 3 Firm 4 3 1 Firm 4 3 1 Firm 4 2 1 Firm 4 3 1 Firm 5 6 5 Firm 5 4 3 Firm 5 7 5 Firm 5 6 5 Firm 5 6 6 Firm 6 4 6 Firm 6 5 6 Firm 6 4 7 Firm 6 7 7 Firm 6 7 7 Firm 7 7 7 Firm 7 6 7 Firm 7 2 6 Firm 7 4 6 Firm 7 4 4 Spearman’s rho = .464 Spearman’s rho = .676 SECTION 2. SURVEY OF SIMULATION INSTRUCTORS The authors decided to ascertain the view of instructors regarding the predictor/performance relationship. Since the authors were not able to come to a consensus and since a review of the literature did not provide a definitive answer, the authors decided to poll the experts. The authors obtained the most accurate and up-to-date list of ABSEL members that was available and emailed a request for each member to respond to a survey that was made available on a website. It was hoped that the responses from the survey would provide some assistance in establishing and prioritizing hypotheses. The ABSEL mailing list presumably contains the names of instructors who would more likely than not to be simulation users. The list contained 168 email addresses. The authors developed a simple 5 question web-based survey and asked the members of ABSEL who were on the mailing list to respond to the survey. There were 29 usable responses to the survey. Although, this number might initially seem to be a low number, it may not be an unworkable sample size given that the total population is relatively low and given that the authors’ intended use is to determine opinions and attitudes that might be generally held. Table 7 lists the questions and results for the survey. DISCUSSION Most of those who responded clearly were simulation users, and 62% indicated yes to the question about having participants submit or designate predictor variables. This relatively large percentage was higher than the authors had expected based on their experience with simulation users. Perhaps, the respondents interpreted the key word, submit, more liberally than was intended. The intention was to determine if students make predictions and give a record of the projections to the game administrator. Question 3 addressed the issue more directly—Do you (the instructor) collect any predictor variables? Fifty-two percent indicated yes. This number was also higher than the authors expected given their experience with simulations. Perhaps, the most interesting question was 4, where they respondents were asked, “Are the students asked to compare and/or explain the differences between what they predicted and what they actually receive on their variables?” To this question, 52% indicated yes. Although the authors didn’t formally establish a priori expectations, this number was lower than anticipated. It was assumed that most instructors using a simulation demand some ex post analysis which of course can mean a simple comparison of differences. So, a number approaching 100% would have been imaginable. Perhaps, the word, explain, caused some respondents to respond negatively. If ex post analysis is performed, but no TABLE 7: Affirmative Responses to Open-Ended Questions Q # QUESTION % Affirmative Response 1 Do you use a general type of computerized simulation?- 90% 2 Are the participants asked to submit predictor variables (e.g., 62% 3 Does the instructor collect information about the predictor variables? 52% 4 Are the “players” asked to compare and/or explain differences between their predictions and results? 52% 212 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 formal explanation process is established, then a respondent may have answered no to this question. A final question asked respondents to rank-order which variables they felt were most important in terms of being better predictor variables than others. The respondents could chose from the list provided and/or enter their own. Table 8 presents the results to this question. Unfortunately, a number of respondents didn’t respond within the strict constraints of the question. Although their responses were sincere and potentially informative, they didn’t facilitate convenient tabular analysis. For example, one respondent said, “The best predictor of student performance, however you choose to measure it, is student performance.” Another respondent said, “I ask the students to select their objectives and evaluate them based upon how well they meet their objectives.” A third respondent volunteered, “Congruence of predictor variable with proposed plans.” Additionally, some respondents went off the list and added their own items such as market share and costs of production. In each case, the responses were interesting, but precluded easy inclusion into formatted quantitative results. TABLE 8: Ranked Responses Respondent First choice Second choice Third choice Fourth choice Fifth choice 1 stock price 2 stock price net profits unit sales dollar sales cash balance 3 not responsive 4 unit sales (qualified) 5 dollar sales net profits unit sales cash balance stock price 6 not responsive 7 net profits unit sales dollar sales 8 net profits stock price dollar sales unit sales cash balance 9 unit sales net profits cash balance dollar sales stock price 10 predicted sales (units or dollars) 11 net profits Increase in revenues 12 net profits stock price cash balance dollar sales unit sales 13 not responsive 14 net profits market share dollar sales costs of production cash balance 15 net profits cash balance dollar sales unit sales stock price 16 unit sales net profits cash balance dollar sales stock price 17 unit sales net profits cash balance dollar sales stock price 18 net profits 19 dollar sales 20 not responsive 21 not responsive 22 not responsive 23 net profits dollar sales stock price cash balance unit sales 24 net profits stock price dollar sales 25 not responsive 26 not responsive 27 net profits 28 net profits 29 unit sales dollar sales cash balance net profits stock price Key: Items in italics show not responsive items or items off the suggested list. 213 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Of the responses that lent themselves to quantitative analysis the most popular item in terms of relevance was net profits. #of sales was second, and stock price was third. Cash balance was not a popular response. Since the authors had initially planned to evaluate variance with respect to net profits, # of sales, and cash balances, it was reassuring that two of the three items came to the top of the list for ABSEL members. Since the authors suspected that the number of respondents that would actually ask students to submit projections and, subsequently compare the projections with actual performance numbers would be low, the question was not asked directly as to which resulting variance measure would be most significant. CONCLUSIONS The first two questions raised in this paper were: • Do students who predict well, perform well? • What predictor variables serve as “better” predictors of performance? It turns out that students (as represented by student teams) tend to predict “better” as simulation play progresses, a finding that would seem to make intuitive sense. Further, it was shown that the closer a team was in its ability to accurately predict its sales (in dollars), the better it performed as game play progresses. This finding is consistent with what simulation instructors said was the most important in the lists of predictor variables that they used in the simulations (i.e., sales, net profits, etc.). These findings are important because they represent some progress in asking the question, what is the learning value of predictive variables. For example, many studies have asked the question: how and what is it that students learn from simulation play. This is a fairly broad and seemingly difficult question to answer, even if one conducted a series of rigorous studies. We call this the “macro” approach to learning. What is suggested from this study is that an important research focus should be on “micro” learning (and research) rather than on “macro” learning (and research). That means there may be significant value in investigating questions 3, 4 and 5 raised in the introduction of this paper, namely, • How do students determine values for their predictor variables • Do students “use” (and if so, how) the information they gain from performance to make decisions on future predictor variables? • Do students “learn” anything from the predictor/performance relationship? Goosen’s “know little” theory (2002) suggests that students (and presumably student teams) can do well on simulation performance due to luck and other factors not related to knowledge or skill. Since some have argued that game performance may not be a good predictor of performance (Anderson & Lawton; and Wolfe and Luethge, 2003), then perhaps a better way to measure “learning” is not game performance, but the ability to make good predictions. Further, let us raise several questions. Would a firm would prefer to hire a person (former student) who is a “better predictor” than one who is not as adept at prediction but who may have shown some degree of success in “managing” a simulated firm? How does success in prediction weigh against success in performance? Are there “better” ways to help students make better predictions? Obviously, further research of a different kind is needed to unravel these questions. REFERENCES Anderson, P. H. and Lawton, L. (1992) “The Relationship Between Financial Performance and Other Measures of Learning on a Simulation Exercise,” Simulation & Gaming, 23(3), 326-340. Gopinath, C. and Sawyer, J. (1999) “Exploring the Learning from an Enterprise Simulation,” The Journal of Management Development, Vol. 18, 477. Goosen, K. (2002) “Student Learning and Business Game Performances—Are they Linked?” Working paper. Hornaday, Robert and Curran, K. (1996) “Formal Planning and the Performance of Business Simulation Teams,” Simulation & Gaming, 27(2), 206-222. Markulis, Peter, and Strang, Daniel (1985) “The Use of Decision Support Systems (DSS) and Operations Research/Management Science (OR/MS) Techniques to Enhance the Learning Experience of Students Participating in Computerized Simulations” Developments in Business Simulation and Experiential Exercises 12, 30-34. Pray, Thomas F. and Strang, Daniel (1980) DECIDE, McGraw-Hill Inc., New York. Vaidyanathan, Rajiv and Rochford, L. (1998) “An Exploratory Investigation of Computer Simulations, Student Preferences, and Performance,” Journal of Education for Business, 73, 144-150. Wolfe, Joseph and Luethge, D. (2003) “The Impact of Involvement on Performance in Business Simulations: An Examination of Goosen’s “Know Little” Decision- Making Thesis,” Journal of Education for Business, 79, 69-75. 214 Table of Contents Volume 32, 2005 LEARNER BEHAVIOR IN THE ONLINE CLASSROOM EXPERIENCE THE EFFECTIVENESS OF A SIMULATION EXERCISE FOR INTEGRATING PROBLEM-BASED LEARNING IN MANAGEMENT EDUCATION DEMONSTRATION OF FOUR WEB-BASED SIMULATIONS THRESHOLD COMPETITOR: A MANAGEMENT SIMULATION ENTREPRENEUR: A NEW VENTURE SIMULATION MERLIN: A MARKETING SIMULATION MICROMATIX: A STRAGETIC MANAGEMENT SIMULATION LEARNING STYLES INFLUENCES ON SATISFACTION AND PERCEIVED LEARNING: ANALYSIS OF AN ONLINE BUSINESS GAME INTERNATIONAL INTERNSHIPS: DESIGN AND EXPERIENCES SIM MAP: TURNING ACTION-BASED LEARNING INTO SIMULATED CONSULTING PROJECTS NOTEL HEALTH SERVICES: A ROLE-PLAYING SIMULATION TEACHING EXPERIENTIALLY WITH THE MADELINE HUNTER METHOD: AN APPLICATION IN A MARKETING RESEARCH COURSE SIMULATING CUSTOMER LIFETIME VALUE: IMPLICATIONS FOR GAME DESIGN AND STUDENT PERFORMANCE VIRTUAL PROGRESS: SIMULATING ECONOMIC DEVELOPMENT ONLINE STRATEGIC MANAGEMENT: AN EVALUATION OF THE USE OF THREE LEARNING METHODS IN CHINA ADOPTION OF DISCUSSION-BASED TEACHING AND ASSESSMENT IN TEACHING STRATEGIC MANAGEMENT IN CHINA STUDENTS' VIEW ON THE USE OF CASE METHOD IN CHINA CHINESE STUDENTS' PERCEPTIONS OF BUSINESS GAMING EXPERIENCECSR - A CORPORATE SOCIAL RESPONSIBILITY SIMULATION CREATING DYNAMIC INTERACTION IN A VIRTUAL WORLD: ADD VALUE TO ONLINE CLASSROOMS THROUGH LIVE ELEARNING AND COLLABORATION: A DEMONSTRATION CAPABILITIES OF EXPERIMENTAL BUSINESS GAMING A COMPARISON BETWEEN SOLUTIONS AND DECISIONS IN A BUSINESS GAME VALIDATING BUSINESS SIMULATIONS: DOES HIGH PRODUCT QUALITY LEAD TO HIGH PROFITABILITY? ALIGNING ART AND EPISTEMOLOGY: ILLUSTRATIONS TO DISTINGUISH DISCOVERY FROM KNOWLEDGE BUILDING TUTORIALS USING WINK STUDENTS AS LAB RATS: THE ETHICS OF CONDUCTING NON-PEDAGOGICAL RESEARCH IN THE CONTEXT OF CLASSROOM SIMULATIONS AND EXPERIENTIAL LEARNING THE EFFECT ON GAME PERFORMANCE OF DIFFERENT MEASURES AND UNITS OF ANALYSIS IN QUANTITATIVE ANALYSIS ANALYZING AND THINKING WHILE PLAYING A SIMULATION COMPUTER BUSINESS SIMULATION DESIGN: THE ROCK POOL METHOD EXPANDING THE ROLE OF E-ROOMS IN DISTANCE LEARNING APPLICATIONS TO MANAGEMENT EDUCATION APPLICATION OF TRADITIONAL AND ONLINE JOURNALING AS PEDAGOGY AND MEANS FOR ASSESSING LEARNING IN AN ENTREPRENEURIAL SEMINAR DEVELOPING MANAGERIAL EFFECTIVENESS: ASSESSING AND COMPARING THE IMPACT OF DEVELOPMENT PROGRAMMES USING A MANAGEMENT SIMULATION OR A MANAGEMENT GAME INTERNATIONAL MANAGEMENT GAME Œ AN INTEGRATED TOOL FOR TEACHING STRATEGIC MANAGEMENT INTERNATIONALLY STUDENT EXPECTATIONS OF SIMULATIONS DISTANCE EDUCATION DELIVERY OF AN INTENSIVE SIMULATION BASED COURSE TEACHING SERVICE LEARNING USING A BUSINESS GAME ROLE-PLAY SIMULATION EDUCATIONAL PERSPECTIVE OF COLLABORATIVE VIRTUAL COMMUNICATION AND MULTI-USER VIRTUAL ENVIRONMENTS FOR BUSINESS SIMULATIONS SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? VIDEO CASE: JET-A-WAY INC. Œ FOCUSING ON DIVERSITY AND ENTREPRENEURIAL LEADERSHIP ACTIVE LEARNING: WHAT IS IT AND WHY SHOULD I USE IT? FACILITATING THROUGH COLLABORATIVE REFLECTIONS TO ACCOMMODATE DIVERSE LEARNING STYLES FOR LONG-TERM RETENTION ONLINE CUMULATIVE SIMULATION TEAM PERFORMANCE PACKAGE WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY USING EXPERIENTIAL LEARNING TO INTEGRATE THE BUSINESS CURRICULUM FORECASTING STOCK VALUE EMPLOYING PROGRESSIVE PRACTICES AND PRINCIPLES TO FACILITATE SEMINAR ROOM LEADERSHIP AMONG LEARNERS: SHARED POWER AND COLLECTIVE ACCOUNTABILITY INDIVIDUAL ACHIEVEMENT DOES NOT GUARANTEE TEAM PERFORMANCE: AN EVIDENCE OF ORGANIZATIONAL LEARNING WITH BUSINESS GAMES DECISION MAKING IN BUSINESS SIMULTION DESIGN ZUG UM ZUG 2015: COLLECTIVE BARGAINING AS A TWO-LEVEL GAME A NEW METHOD FOR MODELING INNOVATION AND R&D IN BUSINESS SIMULATIONS: ILLUSTRATED WITH A SIMULATION OF A NEW PRODUCT DEVELOPMENT PORTFOLIO DEVELOPING A MICRO SIMULATION EFFECT OF MARKET SHARE AND PRODUCTION EXPERIENCE ON COMPANY PROFITABILITY RE-DESIGNING A CURRICULUM THAT VALUES A WORK-INTEGRATED APPROACH TO STUDENT LEARNING HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP ACADEMICALLY SOUND COURSES THAT ALSO MEET STAKEHOLDER NEEDS EVALUATING SERVICE LEARNING: REFLECTION AND ASSESSMENT FROM THE STUDENT POINT OF VIEW SIMPLIFYING AND ENHANCING FINANCIAL ANALYSIS IN CASES AND SIMULATIONS OVERCOMING THE BUSINESS GAME COMPLEXITY PARADOX EXPLORING THE PREFERENCE IN LEARNING APPROACH AMONG THE HONG KONG UNIVERSITY STUDENTS: CASE STUDY, PROBLEM-BASED OR TRADITIONAL TEXTBOOK QUESTION AN EXERCISE FOR EXPLORING THE RELATIONSHIP BETWEEN JUNGIAN PSYCHOLOGICAL TYPES AND POLITICAL STYLE IN THE WORKPLACE EVALUATING THE DIRECTION OF RESEARCH IN ONLINE EDUCATION: ARE WE GOING ANYWHERE? AIS RAIL SYSTEM: A COMPUTER-BASED JOB-ORDER COST SIMULATION BLOOM BEYOND BLOOM: USING THE REVISED TAXONOMY TO DEVELOP EXPERIENTIAL LEARNING STRATEGIES DELIVERING A TECHNOLOGY-BASED CASE IN A TECHNOLOGICAL WAY: THE SMARTCART CASE USING THE INTERNET TO ENHANCE COURSE PRESENTATION: A HELP OR HINDRANCE TO STUDENT LEARNING TEACHING PRACTICES: A CLUSTER ANALYSIS OF STUDENTS IN HONG KONG TEACHING PRACTICES: A CLUSTER ANALYSIS OF TEACHING STAFF IN HONG KONG EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL A SYMBOLIC MODEL OF THE SIMULTANEOUS ACHIEVEMENT OF CONCRETE BENEFITS AND LEARNING BY PARTICIPATING GROUPS IN EXPERIENTIAL ACTIVITIES: ‚THE SPHERE OF EXPERIENTIAL LEARNING