Understanding the Relative Influence of Several Factors on ERP Simulation Performance: An Exploration of Ecological Validity Page 367 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 ABSTRACT This study evaluates the relative factors that influence busi- ness simulation game outcomes through exploring ecologi- cal validity. A regression model analysis was conducted to access the impact of real-world industry factors that influ- ence performance outcomes in a business simulation game. The factors represent a composite of typical company in- dustry measures to evaluate profitability. The study extends beyond prior literature performance outcome measures of return on investment (ROI), and return on assets (ROA) to evaluate business simulation performance. The results showed that 77% of the variance associated with perform- ance outcome is explained by this study’s independent vari- ables. The implications for this study is the first of a two- part research effort to examine the question of whether the ERP business simulation game, ERPSim exhibits natural market structures. The part one study provides the basis to understand the factors that influence profitability in the simulation game. These results lay the ground work to complete the ecological validation of the ERPSim. The findings strongly indicate important real-world factors to predict profitability outcomes in the ERPSim business simulation game. Thus, providing evidence to compare business simulation market share and profitability levels between ERPSim and PIMS (Profit Impact of Marketing Strategies) manufacturing industry project data for eco- logical validity. INTRODUCTION Business simulations used within a college or univer- sity course play an important role in exposing students to real-world business issues, decision making, and business concepts. It is vital for students to learn about business simulations as early as possible, as they will be most likely employed by large corporate enterprises performing in a similar capacity of the game environment (Winkelmann & Matzner, 2009) ( Fedorowicz, Gelinas, Usoff, & Hachey, 2004). Winkelmann et al., (2009) points out that compa- nies stand to benefit from the appropriate specialized busi- ness knowledge and skills obtained by the student-turned- employee when making investment decisions concerning adoption, upgrade, or modifications to their existing sys- tems. Additionally, many business simulation games are where students develop high-order reasoning and decision- making skills learning by doing (Hackney, McMaster, & Harris, 2003; Léger, 2006). Business simulation games provide a dynamic environ- ment for competition in business-like situations that invite participants to analyze, synthesize, evaluate and apply knowledge to compete successfully (Gosen & Washbush, 2004). By analyzing teams‟ performance in business simu- lation games, one can gain insight into the relative factors that influence outcomes that can be applied to further un- derstand how well the simulation game represents real- world business practice. The use of teams in the business game experience has become an accepted prerequisite for high-level learning and performance (Wolfe & Box, 1987). Business simulation games are a type of experiential pedagogy that has undertaken a wide variety of learning environments that result an increase in skills (Boyatzis & Kolb, 1991). In fact, Garris, Ahlers, & Driskell (2002) regard business simulation games as a method that epito- mizes experiential learning (Anderson & Lawton, 1988; Faria & Wellington, 2005; Ruben, 1977). In the mi- croworlds created by business simulations, students can better understand the dynamic business strategy, interactive market competition, and complex business integration (Romme, 2003). However, Lee, Koh, Yen, & Tang (2002) has argued that many technology pedagogical methods do not adequately prepare students to understand and cope with the ambiguities they will without doubt face in their chosen industry. ERPSim is the business simulation game artifact in this study and is a relatively new business simu- lation game. ERPSim was introduced in 2006 and devel- oped by faculty at HEC Montreal University, based on the major industry ERP software application, SAP. The busi- ness simulation game uses the SAP ERP platform to dem- onstrate a supply chain organization business processes, transactions, and management decision making in the con- text of a manufacturing entity. It is an ongoing effort of researchers to examine how realistic are business simulation games. This study exam- ines the ecological validity for an ERP business simulation game. Ecological validity is defined as “the process of assessing that the conclusions reached from a simulation are similar to those reached in the real-world system being modeled” (Feinstein & Cannon, 2002, p. 427). It is impor- UNDERSTANDING THE RELATIVE INFLUENCE OF SEVERAL FACTORS ON ERP SIMULATION PERFORMANCE: AN EXPLORATION OF ECOLOGICAL VALIDITY Mary M. Dunaway University of Arkansas – Fayetteville MDunaway@walton.uark.edu mailto:MDunaway@walton.uark.edu Page 368 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 tant when examining ecological validity in a research study to possess materials and setting to approximate the real-life situation that is under investigation (Goodyear & Ellis, 2008). Ecological validity offers a closer examination of the real-world business phenomenon than internal and external validity and is often confused with external validity which deals with the ability of a study‟s results to generalize (Brewer, 2000; Goodyear et al., 2008). Though these vali- dation forms are closely related, they are independent whereas a study may possess external validity but not eco- logical validity, and vice-versa. Brewers‟ (2000) research provides distinction and contrast to explain that an effect holds up across a wide variety of people or a setting is somewhat different than asking whether the effect is repre- sentative to what happens in everyday life which is the es- sence of ecological validity. The researcher‟s example, a mock-jury study shows how people might act if they were jurors during a trial, where many mock-jury studies simply provide written transcripts or summaries of trials, and do so in classroom or office settings. Such experiments do not approximate the actual look, feel, and procedure of a real courtroom trial, and therefore lack ecological validity. However, more importantly is the concern of external va- lidity whether the results from such mock-jury studies gen- eralize to real trials, and then the research is valid as a whole, despite its ecological shortcomings. Nonetheless, improving the ecological validity of an experiment typi- cally improves the external validity as well. The research gives support that ecological validity is crucial for research that is undertaken for descriptive or demonstration pur- poses. Therefore, this exploratory research is the basis to fur- ther understand relative factors that influence performance outcomes in an enterprise business simulation where a real- world ERP system is utilized in preparation to examine the business simulation natural market structures and scenarios to enhance learning. This research is the first in a two-part study to leverage the research of Wellington and Windor (2006). The goal of the part one research study is to under- stand the ERPSim profitability structure to the most current PIMS analyzing its real-world competitive market shares and rankings. THE ENTERPRISE BUSINESS SIMULATION The enterprise business simulation game examined in this research study is ERPSim known as mySAP. ERPSim utilizes an innovative learning by doing approach for teach- ing Enterprise Resource Planning (ERP) concepts (Léger, 2006). ERPSim is a multifaceted business simulation game designed using the SAP system foundation. SAP systems are one of the world's largest inter-enterprise software ap- plications developed and sold by the fourth-largest inde- pendent software supplier. The ERPSim captures perform- ance activity tracked by operational, reporting, decision, and accounting transactions. The performance activity is highly realistic and meant to simulate the total business enterprise environment. Students form teams and take on individual roles similar to those in a traditional corporation such as production controller, financial manager, sales manager, and warehouse logistics personnel. Each team operates within a made-to-stock manufac- turing supply chain interacting with customer demand, product delivery, materials request planning, and the whole cash to cash cycle (Léger, 2006) . Using standard reports and the business intelligence component of the ERP, teams analyze transactions and make business decisions to ensure the profitability of their operations. Within each business quarter, teams make two important key business decisions for the market segment they wish to target. The teams de- velop a pricing strategy and allocate marketing expenses. The teams operate a plant involved in manufacturing and distribution of muesli cereals. Each teams‟ company has six muesli cereals to produce and sell. ENTERPRISE SIMULATION GAME PERFORMANCE OUTCOMES One of the more common research topics in the use of business simulations has been an effort to determine factors that correlate with simulation performance. Most business simulation research has concentrated on the learning out- comes (Nulden & Scheepers, 2002), team dynamics (Wolfe et al., 1987; Anderson & Lawton, 2005) and pedagogical methods (Léger, 2006; Draijer & Schenk, 2004). Despite the widespread use of business simulation games, the ongoing issue of concern is whether the games are a meaningful experience. Prior literature has viewed validity of business simulations as a measure of how well business games model the real-world industry within the simulation execution (Carvalho, 1991). Wellington & Faria (2006) identified two approaches that were successful for external validation of business simulation games. The approaches focused on the correlation between a business executive‟s simulation game performance and his or her real-world performance and a longitudinal research design where a student‟s business game performance is compared to a business career success (e.g., number of promotions, salary level, etc.). Also two additional studies confirm the same findings (Wolfe & Roberts, 1986; Wolfe & Roberts, 1993). Yet, few studies have empirically examined the degree to which simulated companies behave like real ones (Mehrez, Reichel, & Olami, 1987; House & Napier, 1988; Feinstein & Cannon, 2002). Therefore, it is important to understand the degree to which a business simulation game behaves in ways that are similar to the organization and markets they represent. There is no consensus on what should or could be used as success criteria when evaluating the results of a business simulation game. House & Napier (1988) used profit driven measures of net income, return-on investment (ROI) and Page 369 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 return-on-assets (ROA) as the best indicators of financial performance on a business simulation exercise. Chien-Ta Bruce, Desheng Dash, & Olson (2009) conclude ROE is a more comprehensive and a better indicator than ROA or ROI in terms of a firm‟s profitability and potential growth and risk. Though Anderson and Lawton (1992) did not address evaluating performance in terms of profit meas- ures, they initiated a call to action to examine the possibil- ity of using prediction method results as a measure of busi- ness simulation game performance. METHOD SAMPLE The sample data was collected from an experiment using ERPSim at a university in Italy. Twenty-one MBA students participated in the study divided in six teams of three or four persons. They were asked to participate in a business simulation and responded to several questions before, during and after the simulation. The data for the analysis uses only transaction data extracted from the ERP- Sim game execution. PROCEDURE The ERPSim placed the teams in command of a muesli manufacturing company selling its products to three types of customers: hypermarket, supermarket and retailers. Team members discussed their simulation game strategy, chose which products to make and produce first, order raw material, produce muesli boxes and decide pricing and mar- keting strategy. The teams compete against each other on the same market in order to achieve the largest profit over the period. For this study, all participants had already experienced the simulation. However, confirmed before the start of the simulation; the teams were composed of individuals who had not participated in previous simulations together (as a team). Teams played the simulation over two periods of 30 minutes, alone on the market, competing against the com- puter managing the automated demand. The experiment was divided into three phases. Phase 1. Participation is performed in front of his or her computer. Teammates cannot see each other, are sepa- rated by partitions, and cannot talk to each other as they are wearing anti-noise ear covers. They were asked to answer a survey about their experience working with the Enterprise Resource Planning (ERP) system, data analysis (business intelligence), and their teammates and to determine their psychological profile according the Big Five survey. Fi- nally participants are given their SAP account, to operate their virtual company, and Skype account to enable com- munication with each other. Phase 2. Participants had fifteen minutes to discuss the strategy they would carry out during the first thirty minute period of the business simulation and prepare the system. The simulator was launched. Researchers only intervene during the period to resolve bugs and the simulator was never stopped. At the end of the thirty minutes, participants were asked to answer a second survey about their flow and their situation awareness over the period they just played. Phase 3. After answering the surveys, participants are given five minutes to discuss the corrections and adjust- ments to their strategy and prepare the system for a second period of the business simulation. They start the second period with their company and the market in the same state at the end of the first period. The simulator was launched. At the completion of the simulation, researchers extract usage and business intelligence data from ERPSim, which is imported into a Microsoft Access database. MEASURES The variables used in this study are measures typical of industry companies to assess company performance and profitability. This study examined not only measures of return on equity (ROE) and return on assets (ROA) from prior literature (Anderson et al., 1992; Peach, 1997), but other industry measures having an effect on profitability. The additional variables measured are cumulative net profit, marketing/sales ratio, days sales outstanding, raw material (RM) inventory days, return on sales, mean distri- bution center price, gross margin, and leverage. Cumula- tive net profit represents a company‟s bottom line revenue. Cumulative net profit is calculated by subtracting a com- pany‟s total expenses from total revenue, thus showing what the company has earned (or lost) in a given period of time (usually one year). The marketing/sales ratio is a con- trol measure used to determine whether the cost of the mar- keting activities engaged in to produce the level of sales in a given period was excessive. The ratio evaluates the total marketing expenses expressed as a percentage of total sales revenue. Days sales outstanding (DSO) is the number of days it takes to collect receivables in a given amount of time. It is an important financial indicator as it shows both the age of a company‟s accounts receivable and the average time it takes to turn those receivables into cash. DSO re- veals how many days‟ worth of sales are outstanding and unpaid within a specific period. RM inventory days is a ratio to indicate how many days on average a company turns its raw material inventory into finished products. The ratio is calculated as the raw material inventory divided by consumption of raw material * 365. This measure indicates how efficient raw materials are used to produce a com- pany‟s finished goods. The return on sales (ROS) ratio is widely used to evaluate a company's operational efficiency. ROS is also known as a firm's operating profit margin. This measure is helpful to a company‟s management, providing insight into how much profit is being produced per dollar of sales. The mean distribution price represents the average price of Page 370 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 products across each of the distribution channels. Gross margin is measured as the percentage by which profits ex- ceed production costs. It is a measure of how well a com- pany controls its costs. The calculation for gross margin is a company's profit divided by its revenues and expressing the result as a percentage. Leverage also referenced as the debt-to-equity ratio, which shows how much of the assets of the company are financed by debt and how much by equity (ownership). Leverage is useful for a company to fund growth and development through the purchase of as- sets. If a company has too much outstanding liability, it may not be able to pay back all of its debts. RESULTS Model 1. An exploratory multiple linear regression model was developed to account for variability in cumula- tive net profit. Predictor variables included return of assets (ROA), return on equity (ROE), mean distribution center price, gross margin and marketing. Regression analyses were conducted using SAS 4.2 Enterprise Miner. The analysis showed that variable “marketing” was excluded from the model due to a lack of variability among the data points (the majority of data points (75%) were zero values, while the remaining entries included only nine numbers: 100, 500, 1000, 1200, 1500, 2000, 3000, 4000, 5000). The system thus treated it as a constant and it was not included into the prediction equation. Next, variables ROA and ROE showed problems with multiple collinearity (tolerance values < .10 and VIF > 10). Further inspection of their bivariate correlations with the criterion (i.e. cumulative net profit) indicated perfect rela- tionships with the dependent variable (r=1.00). This is not surprising as these variables were probably derived using cumulative net profit values in the numerator of the for- mula for both ROA and ROE. Variables „gross margin‟ and „mean distribution center price‟ were retained in the model and together accounted for 13% of the variance in the cumulative net profit (R2 = .14), which was statistically significant (p<0.05). Inde- pendently each of the variables was a significant predictor of the cumulative net profit with „gross margin‟ providing the greatest contribution to the equation (standardized beta = -.343). The relationship with the DV was negative, with a smaller gross margin associated with a larger cumulative net profit. Model 2. Since „gross margin‟ and „mean distribution center price‟ could only account for 14% of variability in the dependent variable, I attempted to identify other vari- ables that may improve the predictive power of the model. First, all remaining numeric variables were simultaneously entered in the equation to identify predictive redundancies (multiple collinearity). This strategy yielded problems with multiple collinearity for the following variables: equity, current asset turnover, leverage, debt/equity, current ratio, plant asset turnover, and cumulative sales. Bivariate corre- lations showed that all of variables were significantly cor- related with each other and with the cumulative net profit. Equity showed a perfect correlation (r=1.00) with the DV and was thus excluded from further analysis. Leverage had the second highest correlation with the criterion (r = -0.766, p<0.01) and was thus selected for inclusion into the final model. The remainder of the variables Mktg/Sales, Sales- DaysOutstanding, RMInventoryDaysOutstanding, and ReturnOnSales, did not show problems with multiple col- linearity and together accounted for 64% of variability in the cumulative net profit (adjusted R2 = 0.642), which was statistically significant. Final model. Next I attempted to improve the predic- tive power of the final model by combining viable variables from the previous two models. Thus the final model in- cluded seven unique (non-redundant) predictors: mean dis- tribution center price, gross margin, leverage, Mktg/Sales, SalesDaysOutstanding RMInventoryDaysOutstanding, and ReturnOnSale. The results of the final regression analysis showed that the model has improved its predictive power to 77%, which was highly significant, (R2 = 0.766, F (7,309) = 144.58, p = .000). None of the final eight variables showed problems with multiple collinearity. Table 1 pre- sents the descriptive statistics and correlation matrix of all Variable M SD 1 2 3 4 5 6 7 8 1. CumulativeNetProft -6.77 2.56 2. Mktg/Sales 0.14 0.10 -0.39 3. SalesDaysOutstanding 18.21 4.11 -0.16 0.22 4. RMInventoryDaysOutsanding 18.41 14.56 0.58 -0.13 -0.17 5. ReturnOnSales -0.28 0.61 0.39 -0.39 0.47 -0.12 6. MeanDCPrice 4.62 1.28 0.13 -0.25 -0.03* '0.04* 0.18 7. GrossMargin -0.51 0.23 -0.35 0.17 0.18 -0.12 -0.24 -0.01* 8. Leverage 2.15 0.07 -0.78 0.26 -0.06* -0.67 -0.30* -0.00* 0.20 Table 1 Descriptive Statistics and Correlations Note: Correlations in Rows 1 through 8 (n=317) with an absolute value of .05 or greater is statistically significant at p < .05. Page 371 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 variables in the final analysis. Six independent variables had correlations with performance at the .05 significance level. Statistically significant individual contributions were observed for „SalesDaysOutstanding‟ (standardized beta = - 0 . 3 4 2 ; p < 0 . 0 1 ) , „RMInventoryDaysOutstanding‟ (standardized beta = 0.194; p<0.01), „ReturnOnSales‟ (standardized beta = 0.399; p<0.01), and „leverage‟ (standardized beta = -0.551; p<0.01). As can be deduced from the standardized beta values the most significant predictor in the model was „leverage‟ followed by „ReturnOnSales”. While greater values of the latter variable were associated with greater cumulative net profit, smaller leverage values were associ- ated with higher values of the cumulative net profit. Table 2 summarizes the results of the regression modeling. DISCUSSION AND CONCLUSION This research study is the first to approach the ecologi- cal validity of the ERPSim business simulation game. The intention of this study was to ascertain the relative influ- ence of a myriad of variables on performance in an enter- prise business simulation game. The study attempted to discriminate between those variables that significantly af- fected performance and those whose influence was super- fluous in order to capture the real-world business environ- ment and its impact on simulated game profitability. These findings eliminate the common-method variance to explain true variance due to the data collection method. The data collection approach for the study captured real-time actual transaction usage data rather than self-report from a ques- tionnaire. Therefore, the data collection approach amelio- rates the effect of common-method variance. The results of this regression analysis showed that the independent variables in the study explained 77% of the variance of performance in the simulation. This is a sig- nificant contribution over prior literature results of 44% to explain profit outcomes in a business simulation game. Thus, it can be concluded that these factors associated with real-world business measures did in fact significantly influ- ence performance outcomes providing support for the busi- ness simulation game outcome profitability. These find- ings provide the confirmation to support part two of the ecological validation. The study has a few limitations noted that can be ad- dressed in future studies. The results can be replicated in perhaps other cultural contexts for differentiation in results. Individual learning styles, decision-making, and business acumen may vary from country to country. The sample used in this research study is limited to participants from the country of Italy. Learning an enterprise resource planning system is typically time and resource consuming, and is a great frus- tration on the part of faculty and students alike to grasp understanding. However, when successful, the effort greatly improves the content and pedagogy of business education. Students gain a better understanding of the real- world business processes, company roles and responsibili- ties, and business strategy skills that has immediate benefit for their initial jobs. Additionally, due to SAP‟s wide exposure across in- dustry companies, students who have participated in ERP- Sim are more favorable to potential employers. Companies gain entry level employees with considerable enterprise Variable β ρ β ρ β ρ Constant 2160 -3.83 4.6 CumulativeNetProft - - - - - - Mktg/Sales - - - - 0.31 0.38 DaysSalesOutstanding - - - - -0.34 0.00 RMInventoryDaysOutsanding - - - - 0.20 0.00 ReturnOnSales - - - - 0.40 0.00 MeanDCPrice 0.00 0.70 0.128 0.02 0.05 0.10 GrossMargin 0.00 0.24 -0.343 0.00 -0.06 0.06 Leverage - - - - -0.55 0.00 ROA 1.79 0.00 - - - - ROE -0.78 0.00 - - - - R2 1.00 0.00 0.13 0.00 0.77 0.00 Adjusted R2 1.00 0.00 0.14 0.00 0.76 0.00 Model 1 Model 2 Model 3 Table 2 Results of Regression Modeling: CumulativeNetProfit as Dependent Variable Page 372 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 resource planning system awareness. This is a win-win situation for faculty and companies to benefit from the SAP expertise. Companies benefit from having a more sophisti- cated labor pool from which they can recruit with mini- mized transition. In conclusion, business simulation games such as ERP- Sim are building a strong presence with universities as an effective alternative to traditional teaching methods. Busi- ness simulation games offer substitutive real-world expo- sure that aligns to the realism of modern companies in to- day‟s business environment. REFERENCES Anderson, P. H., & Lawton, L. (1988). Assessing student performance on a business simulation exercise. Devel- opments in Business Simulation & Experiential Exer- cises, 15, 241-245. Anderson, P. H., & Lawton, L. (2005). The effectiveness of a simulation exercise for integrating problem-based learning in management education. Developments in Business Simulations and Experiential Exercises, 32, 10-18. Anderson, P. H., & Lawton, L. (1992). The relationship between financial performance and other measures of learning on a simulation. Simulation & Gaming, 23(3), 326. Boyatzis, R. E., & Kolb, D. A. (1991). Assessing individu- ality in learning: The learning skills profile. Educa- tional Psychology, 11, 279-295. Brewer, M. (2000). Research Design and Issues of Valid- ity. In Reis, H. T. (Ed.), & Judd, C. M.(Ed.), Hand- book of research methods in social and personality psychology(pp. 3-16). Cambridge University Press. Carvalho, G. F. (1991). Evaluating computerized business simulators for objective learning validity. Simulation & Gaming, 22(3), 328. Chien-Ta Bruce, H. O., Desheng Dash, W. U., & Olson, D. L. (2009). A risk scoring model and application to measuring internet stock performance. International Journal of Information Technology & Decision Mak- ing, 8(1), 133-149. Draijer, C., & Schenk, D. (2004). Best practices of business simulation with SAP R/3. Journal of Information Sys- tems Education, 15(3), 261. Faria, A. J., & Wellington, W. J. (2005). Validating busi- ness gaming: Business game conformity with PIMS findings. Simulation & Gaming, 36(2), 259. Fedorowicz, J., Gelinas Jr., U. J., Usoff, C., & Hachey, G. (2004). Twelve tips for successfully integrating enter- prise systems across the curriculum. Journal of Infor- mation Systems Education, 15(3), 235. Feinstein, A. H., & Cannon, H. M. (2002). Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425. Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441. Goodyear, P., & Ellis, R. (2008). University students' ap- proaches to learning: Rethinking the place of technol- ogy. Distance Education, 29(2), 141. Gosen, J., & Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simu- lation & Gaming, 35(2), 270. Hackney, R., McMaster, T., & Harris, A. (2003). Using cases as a teaching tool in IS education. Journal of Information Systems Education, 14(3), 229. House, W., & Napier, H. (1988). A comparative study of strategic performance factors in actual and simulated business environments. Developments in Business Simulation & Experiential Exercises, 15, 50-55. Lee, S., Koh, S., Yen, D., & Tang, H. (2002). Perception gaps between IS academics and IS practitioners: An exploratory study. Information & Management, 40(1), 51. Léger, P. (2006). Using a simulation game approach to teach enterprise resource planning concepts. Journal of Information Systems Education, 17(4), 441. Mehrez, A., Reichel, A., & Olami, R. (1987). The business game versus reality: A case study. Simulation and Gaming, 18(4), 488-500. Nulden, U., & Scheepers, H. (2002). Increasing student interaction in learning activities: Using a simulation to learn about project failure and escalation. Journal of Information Systems Education, 12(4), 223. Peach, E. (1997). Enhancing simulation learning through objectives and decision support systems. Developments in Business Simulation & Experiential Exercises, 23, 61-67. Romme, A. G. L. (2003). Learning outcomes of mi- croworlds for management education. Management Learning, 34(1), 51. Ruben, B. D. (1977). Toward a theory of experience-based instruction. Simulation and Games, 8(2), 211-231. Wellington, W., & Faria, A. (2006). Validating business simulations: Do simulations exhibit natural market structures?. Developments in Business Simulation and Experiential Learning, 33, 118-123. Winkelmann, A., & Matzner, M. (Eds.). (2009). Teaching medium sized ERP systems - a problem-based learning approach. New York: Springer. Wolfe, J., & Roberts, C. R. (1986). The external validity of a business management game: A five-year longitudinal study. Simulation and Games, 17(1), 45-59. Wolfe, J., & Roberts, C. R. (1993). A further study of the external validity of business games: Five-year peer group indicators. Simulation & Gaming, 24(1), 21. Wolfe, J., & Box, T. M. (1987). Team cohesion effects on business game performance. Developments in Business Simulations and Experiential Exercises, 14, 250-255. Table of Contents Volume 38, 2011 Simulated Tabletop Exercise for Risk Management - Anti Bio-terrorism Scenario Simulated Tabletop Exercise From Business Games to Simulations - Simuworlds & Microworlds Demand Equation Redux: The Design and Functionality of the Gold/Pray Model in Computerized Business Simulations Managing Client-Based Learning: Insights from Successful Teaching Project Courses in Marketing Tracking Forecast Error Type, Frequency and Magnitude with the Forecast Error Package Responding to Facilitate Collaboration Simulating Sudden Change and the Value of Timely Information Managing Human Resources Simulation A Study on Collectivism and Group Decision-Making: An International Comparison of Japan, China, and Russia Using a Gaming Simulation The Use of Management Games in the Management Research Agenda Gaming On-Line: A Simulation Application Positioning and Performance in Simulated Networks Supply Chain Management: A Simulation Application Simulation as a Teaching Method in Strategic Management Distance Studies Entrepreneurship: A Game of Risk and Reward Phase II: The Start-Up Return to the Paradise Islands: From Confrontation to Cooperation Effect on Market Performance of Displaying Supply and Demand Curves in a Business Simulation Appreciating Complexity: The Chief of Staff of the Army Game Managing Organizations: Experiential MBA Course Teaches Alternatives to the Machine Model A Simulation Model for Analyzing the Night-Time Emergency Health Care System in Japan The Continuing Evoluation of Assessing Project Management as an Academic Learning Outcome (ALO) Should College Instructors Change Their Teaching Styles to Meet the Millenial Student? The Mouse Game and its Effects on Team Interdependence Learning from the Gulf Oil Spill to Prepare for a Brighter Future: A New Game Engaging Stake Holders in Triple Bottom Line Accounting & Strategic Planning Video Killed the Biblio Star: The Impact of Digital Media on Student Learning Outcomes Exploring Motivation: Using Emoticons to Map Student Motivation in a Business Game Exercise An Alternative to PC and Internet Based Simulations: The Internet Integrated Mode MiddleState University -- A Crisis in Education Complexity Avoidance, Narcissism and Experiential Learning Examining the Cognitive, Affective, and Psychomotor Dimensions in Management Skill Development Through Experiential Learning: Developing a Framework A Situational Leadership Exercise Based on the Biology of a Starfish JOGAI CEFET -- The Industrial Administration Undergraduate Game Would You Take a Marketing Man to a Quick Service Restaurant? Modeling Corporate Social Responsibility In A Food Service Menu-Management Simulation Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met A Team Based Information Literacy Exercise ABSEL Marketing Communications Plan An Interdisciplinary Study of the Impact of Playing a Marketing Simulation Game on Student Knowledge of Management Accounting/Finance Principles Analyzing Construction Planning of Interiro Finish Work of Apartment Building by Simulation Doing Murder One Again The Simple Business Game and Simulation Transfering the Knowledge of Middle Management to Novices Infectious Disease Simulation Model for Estimation of Spreading Understanding the Relative Influence of Several Factors in ERP Simulation Performance: An Exploration of Ecological Validity Tragedy of the Commons: An Exercise Using Clickers to Illustrate and Teach a Key Concept in Negotiations The Meaning of Firm Demand in Business Simulations If the Games Work, Why Aren't More Faculty Willing to Play? Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players