Creating Decision Support Systems in Business Simulation Games CREATING DECISION SUPPORT SYSTEMS IN BUSINESS SIMULATION GAMES Ben-Zvi Tal Stevens Institute of Technology, tal.benzvi@stevens.edu ABSTRACT This study uses a business game as a vehicle for implementing decision support systems (DSS). Fifty-Eight companies, consisting of about 300 senior graduate students participating in a business game, developed DSS and reported on the systems developed. Questionnaires were later used to evaluate a number of relevant variables: use of systems, contribution of systems, and user satisfaction. Findings, consistent with previous empirical studies, strengthen the validity of the simulation exercise as a useful tool for measuring DSS effectiveness. INTRODUCTION Information systems studies have used a variety of instruments to measure information systems (IS) effectiveness (see, for example, Bharati and Chaudhury, 2004; DeLone and McLean, 2003; Reinig, 2003; Sharda et al., 1988; Srinivasan, 1985). The focus of this study is decision support systems (DSS). DSS is used to provide computer-based support to decision makers involved in solving semi-structured and unstructured problems. Studies show that DSS will be effective if both the user and the system work toward the cooperative purpose of improving decision-making. That is, if the objectives or the expectations of the system are met, the system is effective. This is because the information needs of the users (the decision makers) are appropriately supported by the DSS (Khazanchi, 1991). Consequently, the question of measuring the effectiveness of a DSS appears to be in the hands of the users. This study investigates DSS with a focus on factors that affect their effectiveness. We use a game simulation method for this research, where the game becomes the platform for the participants to experience DSS. We also examine the dissimilarity between the developed systems. This research follows an approach akin to that of Ein-Dor and Segev (1984) and of Ben-Zvi (2007) in their business game studies. As both studies considered a very limited number of participants, we augment this investigation by significantly extending the number of participants and parameters of the game. We emphasize that all the studied games hold the same basic characteristics (several executive functions, simulated environment, etc.). The paper is organized as follows: First, we review business game simulations. Then, we describe the employed game and set the study’s hypotheses. Next, we examine the implementation of DSS in the proposed game and analyze related variables. Finally, we discuss the applicability of this study and draw conclusions. BUSINESS SIMULATION GAMES A general-purpose business game is, by definition, a highly complex man-made environment. Its objective is to offer participants the opportunity to learn by doing in as authentic a management situation as possible and to engage them in a simulated experience of the real world (e.g., Garris et al., 2002; Martin, 2000). This usually enhances the characteristics of the game as a simulation of real life, and behavior observed may be generalized to reality (e.g., Lainema and Makkonen, 2003). Over the years, researchers have reported the extent of usage of simulation games in academe and business (e.g., Ben- Zvi and Carton, 2007; Courtney and Paradice, 1993; Dickson et al., 1977; Faria, 1987, 1998). In 2001, a special issue of Simulation & Gaming (Volume 32, no. 4, 2001) was dedicated to the state of the art and science of simulation and gaming. Wolfe and Crookall (1998) assessed the state of simulation and gaming as a scientific discipline. Furthermore, the application of simulations and games is occasionally described also in IS literature. For example, in 2003, a special issue of Communications of the ACM, named “A Game Experience in Every Application”, was dedicated to simulation games in diverse applications; Nulden and Scheepers (2001) suggested a system development simulation in which failure and escalation are introduced to Information System students. Draijer and Schenk (2004) and Léger (2006) used a business simulation game to teach Enterprise Resource Planning concepts. Parker and Swatman (1999) explored an Internet-mediated business game simulating an electronic commerce environment; Yeo and Tan (1999) used a simulation in supporting a course in decision technology. However, empirical studies employing simulations and measuring DSS effectiveness present mixed results. Some researches provide no support for the premise that the use of DSS improves group decision making effectiveness (Affisco and Chanin 1989, Goslar et al. 1986, Kasper 1985). The game we employed represents a tool that successfully enables participants to develop analytical decision making skills, including problem identification skills; data handling skills and thinking skills. Furthermore, with the improvement of technology, simulation exercises have become more sophisticated and user friendly. We elaborate on the game in the next section. HYPOTHESES AND METHODOLOGY THE GAME EMPLOYED This study employs the International Operations Simulation Mark/2000. We use the game to establish a managerial decision- making context: The game involves the participants in the executive process, motivates their need for decision-making aids and forces them to adopt a managerial viewpoint associated with DSS. The game is played for a full semester. Each simulated company may cover any combination of the functions of manufacturing, marketing products or selling to overseas 103 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 mailto:tal.benzvi@stevens.edu distributors, serving as a distributor or a subcontractor, exporting, importing, financing and licensing. The incoming participants play 6 to 10 game-periods. The task of the companies is to make decisions which will guide operations (simulated by the easy to realize computerized system) in the forthcoming period and which will affect operations in subsequent periods. Decisions are made once a week. The length of the each time period simulated is usually referred to as one year. Dozens of decisions, covering the entire range of a typical business, are required of a company in each period. The decision-making process is based on an analysis of the company’s history, interaction with other companies and the constraints stated in the player’s manual (e.g., procedures for production, types of available marketing channels). The performance of a company in each period is affected by its past decisions and performance, the current decisions, simulated customer behavior, and the competition – the other companies in the industry. The game has become highly realistic as a result of the efforts invested in it to simulate the total environment. Participants in the game immerse themselves in this artificially created world. They form teams (without external intervention or manipulation), allocate responsibilities for specific functions, and work to achieve common goals which they themselves define. PARTICIPANTS The study was conducted in a university accredited by the Association to Advance Collegiate Schools of Business (AACSB). The participants were senior graduate students. The students were divided into 5-participant-groups (companies). We explored three semesters: (1) the spring 2005 semester, consisting of 18 companies; (2) the summer 2005 semester – 20 companies; and (3) the spring 2006 semester – 20 companies. In the fall semester of 2005 we experienced only 9 groups, and therefore, decided not to include that semester in this research. Table 1. Characteristics of Systems Developed by Companies in the Spring Semester of 2005 Co. System Area Nature of System Data Analysis Graphics 1 Production, Finance, Market Analysis Electronic Sheet Yes No 2 R&D, Production, Finance, Marketing Electronic Sheet No Yes 3 Production, Finance, Market Analysis Electronic Sheet Yes No 4 R&D, Production, Finance, Marketing, Market Analysis Electronic Sheet, Regressions Yes No 5 Production, Finance Electronic Sheet No No 6 R&D, Production, Finance, Marketing, Market Analysis Electronic Sheet Yes No 7 Production, Finance Electronic Sheet No No 8 R&D, Production, Finance, Marketing, Market Analysis Electronic Sheet Yes No 9 Production, Finance Electronic Sheet No No 10 Production, Finance, Marketing Electronic Sheet No No 11 R&D, Production, Finance, Marketing Electronic Sheet No No 12 R&D, Production, Finance, Market Analysis Electronic Sheet, Regressions Yes No 13 R&D, Production, Finance Electronic Sheet No Yes 14 Marketing, Market Analysis Electronic Sheet, Regressions Yes No 15 Finance, Marketing, Market Analysis Electronic Sheet Yes Yes 16 Production, Marketing Electronic Sheet Yes Yes 17 Production, Finance Easy Plan, Electronic Sheet No No HYPOTHESES This study aims to measure the effectiveness of the developed DSS. For that, we measure the participants’ perceived benefits from using a DSS, variables related to DSS use, user satisfaction, and success. As we use the business game as a tool for measuring DSS, we follow hypotheses examined by Ein-Dor and Segev (1984) and Ben-Zvi (2007). The first hypothesis in this study relates variables in DSS studies to DSS effectiveness. Many IS researchers have studied the success and failure of DSS from several perspectives. Common measured criteria of DSS success include system’s reliability and flexibility (Srinivasan, 1985), the ability of a system to support decision- making and problem-solving activities (Garrity and Sanders, 1998), use and user satisfaction (Baroudi et al., 1986; DeLone and McLean, 2003), and decision confidence (Goslar et al., 1986). In this study we examine the following DSS success variables: usefulness, user satisfaction, system contribution to functional area and company success, own use and colleague use. The first hypothesis relates to both the individual and company level: 104 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 Hypothesis 1: The measures of success present high and significant correlation between their criteria. The second hypothesis relates DSS effectiveness variables to company performance: Hypothesis 2: The measures of DSS success are highly correlated with company performance. As each company functions as a distinct entity in the game, we also examine the dissimilarity between the companies: Hypothesis 3: Company differentiation in DSS: Variance between companies is significantly different from the variance within companies. PROCEDURES At the end of each semester, after the last set of decisions had been made, each group was required to present its DSS in class and to submit a report consisting of: (1) a definition of the scope of the system; (2) a decision analysis; (3) a system design; and (4) a discussion of the contribution of the system in achieving the group’s objectives during the game. At that same meeting, each of the students was asked to complete a short individual questionnaire on the DSS assignment (see the appendix for the text of the questionnaire). RESEARCH FINDINGS DEVELOPED SYSTEMS Two-thirds of the companies in all three semesters nominated a Chief Information Officer (CIO). All companies reported developing an information system but none of the companies reported major modifications during the semester. We present an example of the systems developed in the spring semester of 2005. Eighteen companies were created in that semester, most of which developed a Microsoft Excel spreadsheet-based DSS. The major characteristics of the systems developed are exhibited in Table 1. For this study, the most relevant aspect of Table 1 is the extent to which the companies differed on their systems. Companies, in all three semesters, adopted different application areas with models including various statistical analyses, spreadsheets—and even linear regressions. Only 3 companies (5% of all companies) employed any type of package software. Thirty five companies developed complicated data analysis tools (mostly statistical or engineering analyses) for their systems (60% of all companies). Only 19 companies developed graphic outputs (about a third of all companies), while the remaining 39 did not. Finally, the sophistication and complexity of the models employed varied significantly from simple spreadsheet analyses (companies 5 and 7 in the spring semester of 2005) to a complex linear model (company 4 in that same semester). While it cannot be claimed that the distribution of attributes of systems exactly measures that in the real world, the degree of diversity of systems developed, based on existing tools, does appear to be quite real. Figures 1 and 2 present a sample of those systems. Figure 1 demonstrates the market analysis conducted by company 1 in the 6th played period of the spring semester of 2005. Part I of Figure 1 presents an analysis of the US market. Company 1 mainly operated in the US market and therefore, a full analysis of prices, models, market share and inventory was required. Part II analyzes the company’s inventory in the US market. Part III exhibits an aggregated analysis of all companies’ world-wide. Figure 2 illustrates a DSS developed by company 5 in the spring semester of 2006. It shows the average investment in Research and Development (R&D) of each company against the investment made by company 5 in the corresponding periods. As company 5 followed a strategy of R&D superiority, a non-linear regression was constructed to make predictions of future investments to stay ahead of the R&D investment curve. Part I Part II Part III Figure 1. A Sample of DSS Developed by Company 1in the Spring Semester of 2005. 105 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 R&D Inv Figure 2. A Sample of Graphical R&D Analysis Made by Company 5 in the Spring Semester of 2006. ANALYSIS In order to enhance the validity our results, we compared them to previous findings reported by Ein-Dor and Segev (1984) and Ben-Zvi (2007). The analysis of the data relates both to individuals and to companies. Company data in this study aggregate the individual data of the company’s members, and is conducted in order to determine whether the participants in the game coalesce into distinguishable companies. First, the customary variable in DSS studies, degree of success, is analyzed. Next, company performance is analyzed with regard to the developed DSS. Finally, we discuss company differentiation. The internal consistency among the items, Cronbach’s alpha (Cronbach, 1951), is 0.8345 at the individual level and 0.8532 at the company level. Means and variance of responses to the first 10 questions are exhibited in Table 2. SUCCESS OF DSS In this section we examine the following six DSS success variables: 1. Usefulness of the system as evaluated by participants (question 2). 2. Own use by respondents (question 3). 3. Use by colleagues (question 6). 4. The system’s contribution to the company’s performance in respondents’ functional areas (question 4). 5. The system’s contribution to the company’s overall success (question 7). 6. User satisfaction (question 5). We adopt the approach taken by Ein-Dor and Segev (1984) and regard all success criteria as being co-determined; that is, we do not assume cause-and-effect relationships between them. Table 3 exhibits all correlations between the success criteria for individual respondents in this study, as defined above. The table shows strong and highly significant relationships between the criteria, except for the correlation between own use and colleague use. The strong correlations found would seem to indicate that the criteria are indeed all related and presumably all measure some aspect of success. The lack of mathematical correlation between the own use and the colleague use variables does not imply that those two variables are not correlated. A detailed analysis showed that participants were divided into two major groups, by company: one with a highly positive estments y = 50.691x3 - 594.05x2 + 1860.7x - 516.47 R2 = 0.9479 0 200 400 600 800 1000 1200 1400 5 6 7 8 9 10 Period industry Average Company 5 Trendline Table 2. Means and Standard Deviations (S.D.) of Responses for Individual and Companies Individuals (n=290) Companies (n=58) Variable Mean S.D. Mean S.D. Familiarity 5.62 1.35 5.55 0.82 Usefulness 5.52 1.12 5.42 0.91 Own use 5.29 1.52 5.06 0.88 Contribution to functional area 5.33 1.48 4.99 1.04 User satisfaction 5.42 1.45 5.02 1.27 Use by colleagues 4.89 1.21 4.87 0.81 Contribution to company success 5.45 1.39 5.04 1.18 Participation 4.56 1.91 4.57 1.14 Disturbance 2.88 1.98 2.87 0.78 Met expectations 5.21 1.73 4.59 1.41 106 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 correlation and one with a highly negative one. This caused the average correlation between the two variables to become small. Table 4 demonstrates all correlations between the success criteria at the company level. It appears that there are very strong correlations between the measures of success at this level, and in most cases the relationships are significant. Note that the grouping procedure by companies largely increased the correlation between own use and use by colleagues. Thus, the data in the study strengthen the hypothesis concerning the nature of success and failure of DSS and replicates previous empirical findings. COMPANY PERFORMANCE ANALYSIS This section investigates company performance versus all measured variables. In all three studied semesters, company performance was measured by the companies’ accumulated retained earnings (accumulated profits). Table 5 exhibits the correlations between company performance and all DSS measured variables of this study. Correlation was made for the company level. The results indicate that five variables are strongly related to the company’s performance: system’s usefulness, user satisfaction, contribution of the DSS to the diverse functional areas and to the entire company success and whether the DSS met its expectations. It seems that the greater the satisfaction from the developed system in meeting its intended aim as set by Table 3. Relationships between Criteria of DSS Success for Individual Respondents (n=290) Table entries: Spearman’s rho correlation coefficient Significance level Use Contribution Own Use Use by Colleagues Functional area Company success User satisfaction 0.412 0.441 0.62 0.673 0.726 Usefulness p=0.001 p<0.001 p<0.001 p<0.001 p<0.001 0.028 0.651 0.373 0.291 Own use p=0.437 p<0.001 p=0.002 p=0.01 0.259 0.409 0.378 Colleague use p=0.01 p<0.001 p=0.001 0.609 0.569 Contribution to functional area p<0.001 p<0.001 0.702 Contribution to company success p<0.001 Table 4. Relationships between Criteria of DSS Success for Companies (n=58) Table entries: Spearman’s rho correlation coefficient Significance level Use Contribution Own Use Use by Colleagues Functional area Company success User satisfaction 0.407 0.631 0.706 0.741 0.814 Usefulness p=0.043 p=0.002 p<0.001 p<0.001 p<0.001 0.297 0.572 0.322 0.27 Own use p=0.102 p=0.006 p=0.091 p=0.139 0.399 0.454 0.455 Colleague use p=0.048 p=0.027 p=0.028 0.589 0.633 Contribution to functional area p=0.005 p=0.002 0.803 Contribution to company success p<0.001 107 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 the users, the better the company’s performance in the game. Nevertheless, the two variables related to the participation of users in defining the DSS present negative correlation with the company’s performance. It seems that added involvement in developing the DSS impairs performance. Furthermore, we measured a correlation of 0.29 between the number of functions the DSS cover (e.g., production, finance, market analysis) and the companies’ performance. There is also a correlation of 0.35 and 0.05 between a company’s performance and its use of data analysis tools and graphics, respectively. To summarize, it can be claimed that a successful DSS in the eyes of the users is related to better company performance in the game. However, investing a lot of human resources in developing a complicated system that makes use of several features does not necessarily guarantee enhanced company performance. COMPANY DIFFERENTIATION Ein-Dor and Segev (1978) indicated that the organizational and external environments of information systems were recognized as one of the factors impacting the success and failure of information systems. Those environmental factors are usually uncontrollable and as a result, they invariably cloud the meaning of data collected in trans-organizational comparisons of DSS. One of greatest advantages of the business game is the common and controlled external environment it provides for all participating companies. Despite the identity of initial conditions, significant differences in DSS emerged by the end of the game in each semester. Table 6 exhibits the analysis of variance, by all 58 companies, for each variable in the questionnaire. The data indicate that, for 5 of the 10 variables, the variance between companies is significantly different (at the .05 level) from the variance within companies. There is a degree of consensus within companies as to their success. For two measures of success, the level of performance Table 5. Correlation between Company Performance and All Measured Variables (n=58) Variable Correlation Familiarity 0.02 Usefulness 0.60 Own use 0.19 Contribution to functional area 0.62 User satisfaction 0.87 Use by colleagues 0.36 Contribution to company success 0.77 Participation -0.21 Disturbance -0.01 Met expectations 0.72 Table 6. Analysis of Variance of All Variables by Companies (n=58) Variable F value Sig. of F Familiarity 1.014 0.478 Usefulness 2.049 0.029 Own use 1.263 0.262 Contribution to functional area 2.037 0.030 User satisfaction 3.541 0.000 Use by colleagues 0.861 0.622 Contribution to company success 3.491 0.000 Participation 0.918 0.561 Disturbance 0.349 0.991 Met expectations 3.769 0.000 108 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 and the user satisfaction, results exhibit highly significant F values, indicating that the variance of responses within companies are appreciably smaller than those between companies. The third measure of success, the system’s use, does not exhibit low variance of responses within companies. This can be attributed to the fact that some companies introduced a relatively high use of the systems developed by all members, while other companies performed with only one or two members using the system. To summarize, it can be claimed that differentiated companies emerged from the game. The differences cannot be artifacts of the environment, which is common to all. Thus, the business game permits the analysis of differences in DSS in organizational contexts unhindered by uncontrollable external environmental influences. DISCUSSION AND CONCLUSIONS This study examined simulated companies. Although the general environment was mutual to all participants, the companies became differentiated. Each company assumed a considerably different strategy, different operating decisions, and a different approach to DSS. And leaving DSS development decisions to the companies resulted in a variety of applications and a wide array of models, programs and modes of operation. It appears that these companies reflect most real life business approaches to DSS. In addition, this study tested three hypotheses. All three hypotheses were confirmed, replicating a number of previous findings. Overall, results at both the individual participant and the company levels underscore that the business game may be used as a vehicle for implementation of DSS. More generally, our experience suggests that the efficacy of business games as platforms for implementing DSS is twofold. First, participants practice the art of decision-making; participants are excited, motivated and strive to make better decisions; they become actively involved in the simulated decision-making process and in the development of DSS of their choice. Second, because the game is very practical, the participants themselves frame the relationship between the decision-making processes, the designed information systems and the outcomes of their use. This exemplifies how decision- making is more successful using DSS and also provides an integrative view of some of the tasks and practical uses of DSS. The ultimate result is more successful DSS in the real world. In the games associated with this study, most companies developed a spreadsheet-based DSS. Although some may regard spreadsheets as too simplified DSS, our study reveals that complicated systems do not guarantee better company performance. Nowadays, even the frequently used spreadsheets are sufficient tools to create extremely powerful and useful DSS. Moreover, spreadsheets offer some substantial pedagogical advantages: Individuals today, not necessarily IS oriented, are familiar with spreadsheet tools, so they can quickly employ them for the development of a DSS. Spreadsheets also allow a dynamic data updating and facilitate data visualization. Also, modern spreadsheet programs contain powerful data analysis tools (e.g., Analysis ToolPak in Excel); Sixty percent of all participating teams incorporated data analysis tools into their DSS. However, while feedback from participants is favourable and the game is sufficiently complex to provide challenges and a realistic simulation of decision making, no business game can encompass all aspects of information systems. Because the game decisions are more simplistic than those of the real world, the DSS required to support the decisions are less complicated than those in reality. Therefore, there is a need to determine how business games, as learning laboratories, can be augmented to study the more complex, dynamic aspects of the DSS domain: use and performance can be easily measured and evaluated, but the cost/benefit or return of investment of a specific information system is as vague in the game as it is in real life. REFERENCES Affisco, J. F., and Chanin, M.N., “The Impact of Decision Support Systems on the Effectiveness of Small Group Decisions - An Exploratory Study”, Developments in Business Simulation & Experiential Exercises, Vol. 16, 1989, pp. 132-135. Baroudi, J.J., Olson, M.H., and Ives, B. “An Empirical Study of the Impact of User Involvement on System Usage and Information Satisfaction”, Communications of the ACM, Vol. 29, No. 3, 1986, pp. 232-238. Ben-Zvi T., “Using Business Games in Teaching DSS”, Journal of Information Systems Education, Vol. 18, No. 1, 2007, pp. 113-124. Ben-Zvi T. and Carton, T.C., “From Rhetoric to Reality: Business Games as Educational Tools”, INFORMS Transactions on Education, Vol. 8, No. 1, 2007, pp. 10-18. Bharati, P., and Chaudhury, A. “An Empirical Investigation of Decision-Making Satisfaction in Web-Based Decision Support Systems”, Decision Support Systems, Vol. 37, No. 2, 2004, pp. 187-197. Courtney J. F., and Paradice D. B., "Studies in Managerial Problem Formulation Systems," Decision Support Systems, Vol. 9, 1993, pp. 413-423. Cronbach, L.J., “Coefficient Alpha and Internal Structure of Tests”, Psychometrics, Vol. 16, No. 3, 1951, pp. 297-334. DeLone, W.H. and McLean, E.R. “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update”, Journal of Management Information Systems, Vol. 19, No. 4, 2003, pp. 9-30. Dickson G.W., Senn, J.A., and Chervany, N.L. “Research in Management Information Systems: The Minnesota Experiments”, Management Science, Vol. 23, No. 9, 1977, pp. 913-923. Draijer, C., and Schenk, D-J. “Best Practices of Business Simulation with SAP R/3”, Journal of Information Systems Education, Vol. 15, No. 3, 2004, pp. 244-261. Ein-Dor, P., and Segev, E. “Business Games: Laboratories for Implementation Research In MIS”, Applications of Management Science, Supplement 1, 1984, pp. 133-158. Ein-Dor, P., and Segev, E. “Organizational Context and the Success of Management Information Systems”, Management Science, Vol. 24, No. 10, 1978, pp. 1064- 1077. Faria, A. J. “Business Simulation Games: Current Usage Levels – An Update”, Simulation and Gaming: An International Journal, Vol. 29, No. 3, 1998, pp. 295-308. 109 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 Faria, A.J. “A Survey of the Use of Business Games in Academia and Business”, Simulation and Games: An International Journal, Vol. 18, No. 2, 1987, pp. 207-224. Garris, R., Ahlers, R., and Driskell, J.E. “Games, Motivation and Learning: A Research and Practice Model”, Simulation & Gaming: An Interdisciplinary Journal, Vol. 33, No. 4, 2002, pp. 441-467. Garrity, E.J., and Sanders, G.L. “Dimensions of Information Systems Success”, in Garrity, E.J., and Sanders, G.L. (Eds.), Information Systems Success Measurement, Idea Group Publishing, Hershey, PA, 1998, pp. 13-45. Goslar, M.D., Green, G.I., and Hughes, T.H. “Decision Support Systems: An Empirical Assessment for Decision Making”, Decision Sciences, Vol. 17, No. 1, 1986, pp. 79-91. Kasper, G. M., “The Effect of User Developed DSS Applications on Forecasting Decision-flaking Performance in an Experimental Setting”, Journal of Management Information Systems, Vol. 2, No. 2, 1985. pp. 26-39. Khazanchi, D. “Evaluating decision support systems: A dialectical perspective”, Proceedings of the twenty-fourth Annual Hawaii International Conference on Systems Sciences (HICSS-24), IEEE Computing Society Press, III, 1991, pp. 90-97. Lainema, T., and Makkonen, P. “Applying constructivist approach to educational business games: Case REALGAME”, Simulation & Gaming: An Interdisciplinary Journal, Vol. 34, No. 1, 2003, pp. 131-149. Léger, P-M. “Using a Simulation Game Approach to Teach Enterprise Resource Planning Concepts”, Journal of Information Systems Education, Vol. 17, No. 4, 2006, pp. 441-447. Martin, A. “The Design and Evolution of a Simulation/Game for Teaching Information Systems Development”, Simulation & Gaming: An Interdisciplinary Journal, Vol. 31, No. 4, 2000, pp. 445-463. Nulden, U., and Scheepers, H. “Increasing Student Interaction in Learning Activities: Using a Simulation to Learn about Project Failure and Escalation”, Journal of Information System Education, Vol. 12, No. 4, 2001, pp. 223-232. Parker, C. M., and Swatman, P. M. C. “An Internet-mediated Electronic Commerce Business Simulation: Experiences Developing and Using TRECS”, Simulation & Gaming: An Interdisciplinary Journal, Vol. 30, No. 1, 1999, pp. 51–69. Reinig, B.A. “Toward an Understanding of Satisfaction with the Process and Outcomes of Teamwork”, Journal of Management Information Systems, Vol. 19, No. 4, 2003, pp. 65-84. Sharda, R., Barr, S. H. and McDonnel, J. C. “Decision Support System Effectiveness: A Review and an Empirical Study”, Management Science, Vol. 34, No. 2, 1988, pp. 139-159. Srinivasan, A. “Alternative Measures of System Effectiveness: Associations and Implications, MIS Quarterly, Vol. 9, No. 3, 1985, pp. 243–253. Wolfe, J., and Crookall, D., “Developing a Scientific Knowledge of Simulation/Gaming.” Simulation and Gaming: An International Journal, Vol. 29, No. 1, 1998, pp. 7-19. Yeo, G. K., and Tan, S. T. “Toward a Multilingual Experiential Environment for Learning Decision Technology”, Simulation & Gaming: An Interdisciplinary Journal, Vol. 30, No. 1, 1999, pp. 70–83. 110 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 APPENDIX Questionnaire – Decision Support Systems Report The following questions relate to the Decision Support System, which was developed in your company. Please indicate your answers: Not at all To a very small degree To a small degree To a degree To a large degree To a very large degree Maximally 1. I am familiar with the system developed in the company 1 2 3 4 5 6 7 2. The system is useful for decision making 1 2 3 4 5 6 7 3. I personally used the system for making decisions in my role in the company 1 2 3 4 5 6 7 4. The system contributed to the company’s performance in my functional area 1 2 3 4 5 6 7 5. I am satisfied with the system 1 2 3 4 5 6 7 6. My colleagues in the company used the system for decision making 1 2 3 4 5 6 7 7. The system contributed to the company’s success 1 2 3 4 5 6 7 8. I participated in defining the system 1 2 3 4 5 6 7 9. Developing the system interfered with my functional role in the company 1 2 3 4 5 6 7 10. The system’s benefits met my expectations 1 2 3 4 5 6 7 111 | Developments in Business Simulation and Experiential Learning, Volume 36, 2009 Table of Contents Volume 36, 2009 Protecting Academic Integrity: Student Assessment In The Online Environment Pedagogical Shift: Teaching Report Writing For Accountants Online Increasing Student Learning In An Investment Management Course Through The Innovative Use Of Experiential Learning Pedagogy A Moral Development Unit For Business Courses Dividing Up Grandma’s Things: A Multifaceted Exercise In Critical Thinking Application Of Haekel’s Thesis To ABSEL Development Linking Stories With On-Line Threaded Discussions For Critical Thinking In Management Curriculum Why Have We Neglected Vicarious Experiential Learning? A Triadic Multi-Disciplinary Approach To Enhancing The Efficacy Of Experiential Learning Movement Toward Increased Student Roles In The Design Of Experiential Exercises Ready-Mix Concrete Company: An Experiential Exercise In Management Theory Using Portfolio Theory In A General Management Simulation An Experimental Analysis Of Advertising Strategies And Advertising The Relationship Between Goal Orientation And Simulation Performance With Attitude Change And Perceived Learning Beyond The Profitable-Product Death Spiral: Managing Product Mix In An Environment Of Constrained Resources Online Marketing Control With The Strategic Business Unit Analysis Package Beat The Market Game Creating Decision Support Systems In Business Simulation Games The Use Of Computer-Assisted, Interactive Role-Play Simulation In China Asia Marketing: An International Business Game An Empirical Test Of “Behavioral Immersion” In Experiential Learning Existing And Emerging Business Simulation-Game Design Movements Computerized Business Simulations: A Systems Dynamics Process Service Launch Evaluating Business Plans In A Simulation Environment Teaching Software Development By Means Of A Classroom Game: The Software Development Game Group Decision Experiments Using Business Game - Problem Solving With Conflict Luna: A Role Play Game For Learning Incoterms 2000 Entrepreneurship: A Game Of Risk And Reward Phase I -- The Search For Opportunity The Wee Game: A Pre-Game Learning Inhibitors In Business Simulations And Games New Version Of An Old Simulation Helps Build A Perfect Capstone Course Project Competitor: A Simulation Game For Project Management With 2 Models And 2 Modes Mitigating The Winner’s Curse In The Auction Market Of A Computer-Assisted Business Gaming Simulation Evaluation Of Collaborative Filtering By Agent-Based Simulation Considering Market Environment Marketing Simulation Game Decision Making Experience And Its Impact On Indecisiveness Among Introductory Marketing Students Individual And Organizational Learning In A Top Management Game In Pursuit Of Stockholder Value: Reinforcing Core Concepts In A Business Strategy Simulation With A “Shadow” Stock Market Competition The Simplicity Paradox: Another Look At Complexity In Design Of Simulations And Experiential Exercises In Search Of The Ethnocentric Consumer: Experiencing “Laddering” Research In International Advertising The Paradise Islands Revisited: Trouble In Paradise Developing And Assessing Student Information Literacy Competency Do First Mover Advantages Exist In Competitive Board Games: The Importance Of Zugzwang The Natural Debriefing Approach: A Case In A Simple Business Game Pursued For Perfect Communications New Product Development Simulation Dominance In On Line Business Games Competitions Cooperative Business Game With Framing Effect Experiential Exercises In The Online Environment Enhancing Web-Based Simulations With Game Elements For Increased Engagement The Ginseng Game Experience Business Using a Simple Business Simulation