The Influence of Variables easily Controlled by the Instructor/Administrator on Simulation Outcomes: In Particular, the Var... Developments in Business Simulation and Experiential Learning, Volume 31, 2004 THE INFLUENCE OF VARIABLES EASILY CONTROLLED BY THE INSTRUCTOR/ADMINISTRATOR ON SIMULATION OUTCOMES: IN PARTICULAR, THE VARIABLE, REFLECTION. Jerry Gosen University of Wisconsin –Whitewater ABSTRACT This paper deals with two related topics. The first is the influence of variables under the control of the teacher/administrator on simulation outcomes. It contends that many teacher-controlled variables have considerable influence in simulations and this influence should be further studied. The second topic is reflection, which is part of the learning process. This paper explores this important aspect of learning, divides the reflection process into four sub-processes, argues that teacher/administrators can influence how and how much students reflect on their simulation experience, and explores the potential for undertaking research on the relationship between reflection and simulation outcomes. TEACHER/ADMINISTRATOR INFLUENCE Many scholars contend that teacher/administrators have considerable influence over the process and outcomes in simulations. For example Christopher (1999, pg. 453) stated, “ at the end of the day, what players take away will be defined by my situational behavior as game leader. The way I present, direct and debrief the game will affect participants` responses critically, no matter what the actual content of the game may be.” Garis, Ahlers, and Driskell (2002, pg 458) proposed instructional content as an important input variable in the learning process and contended, “To the extent that we pair game features with appropriate instructional content, we can harness these motivational forces to achieve desired learning outcomes.” In 1999, John Washbush and I wrote an S&G anniversary article reviewing our teaching styles. We pointed out that our teaching styles differed, that John’s students showed higher learning scores than mine (on a test that we both give to our simulation students), and that our teaching style differences might have been responsible for the learning score differences. We concluded that teaching style/behavior probably has great influence on the degree to which students learn when playing a simulation. At that time, we said, “We believe that behavior, design, and atmosphere variables, both singly, and in combination, probably produce unique and substantial effects on student learning in simulation experiences. Many of these design features are difficult to specify and measure, but that should not stop us from identifying methods and environments that enhance learning. There is great potential benefit that should accrue from such investigations. After all, learning is why we are involved with simulations in the first place. There are few more valuable research results than those showing how students can more effectively learn” (Gosen and Washbush, 1999, pp. 201- 202). We then offered 15 variables under control of the instructor/administrator which are likely to affect outcomes (especially learning) in the simulation. In my research for this article I’ve discovered two more variables for the list and took one out. Below is the list of 16. 1) Instructional intent in choosing a simulation. Wolfe and Roge' (1997) have argued that choice of game should be consistent with teaching objectives and students' levels of knowledge and sophistication. Aspects of games worthy of consideration include functional-area integration potential, the strategic management knowledge base, and analytic methods to be utilized. 2) Instructor behaviors in introducing the simulation. These include whether proper explanations of the purposes and the unique features of the simulation are provided to players before the game begins (Pimentel, 1995), the extent to which the mechanics of the simulation are introduced before play begins (Wheatley, 1994), and how much game-content information players receive before playing (Hill and Lance, 2002) 3) The amount and form of practice experienced by players before the game. For example, we are aware that Tim Scott of Mankato State University and the senior author of MICROMATIC (Scott, et al., 1992) uses a non-competitive, individualized version of his simulation before students begin to play the competitive version in teams. 4) Player objectives under administrator control. These objectives might reflect a competitive standard, for example accumulated profits, or some measure of excellence, for example organizational stability or quality (Rausch, 1995). 318 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 5) The instructor’s role. Wolfe (1990) has argued that business games are invariably accompanied by a more labor-intensive effort on the part of the instructor. In his ABSEL Guide article (Wolfe, 1990), he cites research showing simulation performance gains with increases in active guidance on the part of the instructor, and Wolfe himself (Wolfe1975) found that learning was enhanced when the instructor took an active role. 6) Number of office hours available for student help (Affisco, 1994) 7) The degree to which the instructor helps the students process the experience, i.e., helps them discover what they are learning (Lederman, 1994) and (Keys, 1994). 8) Features of the game. These include the proximity of competitors to each other and to the game administrator(s) (Rausch, 1995), game duration (Wolfe, 1990), timing for starting the game (Anderson and Lawton, 2003), and game complexity, in terms of decisions per round, words in the players manual, and the size of the simulation program(Wolfe, 1990). Wolfe (1990) cited three relevant studies involving complexity: one (Wolfe, 1978) indicating a positive relationship between game complexity and learning, the other two (Butler, Pray and Strang, 1979 and Raia, 1966) showing no relationship between game complexity and learning. 9) The context of the game and the degree to which it is integrated with the course or the training session of which it is a part (Goodman, 1995; Snow, Gehlen, and Green, 2002; Walford, 1995). Snow, Gehlen, and Green (2002) compared a MBA section in which the game was integrated with the rest of the course with one in which the game was administered as an exercise separate from other features of the course. Students in the integrated section had more positive attitudes towards their experience than students in the separated section. 10) The degree to which “external interventions” (Green, Mcquaid, and Snow, 2002) are integrated into the simulation experience. These include strikes (Green, Mcquaid, and Snow, 2002), potentially unethical purchasing opportunities (Biggs, 1995, Rausch, 1995), group-dynamics-oriented interventions (Biggs, 1995), opportunities for buying comprehensive insurance (Green, Mcquaid, and Snow, 2002), and reviews by regulators (Green, Mcquaid, and Snow, 2002). 11) Percent of grade allocated to game performance. Keys and Wolfe (1990) suggest this as a potential factor for influencing game outcomes. They cite one study by Wolfe and Roberts (1985-86) that found no correlation between learning levels and grade weights. 12) Team characteristics including size and composition. Regarding size, Keys and Wolfe (1990) cited a study by Wolfe and Chacko (1983) which indicated that learning levels were highest for students in three or four person teams. Regarding composition, some administrators assign team membership randomly, some allow players to choose their own teammates, and other try to spread skills and major. 13) Analytic aids. These are systems that help students analyze their data and help them project their team’s potential performance into the future. An example would be an expert system (Rausch, 1995). 14) Financial indices used for scoring. Dickinson (2003) has pointed out that different games have different built-in options for assessing player performance and that administrators have choices as to which of those options and how many to use. Most use returns to grade players, but some use more obscure options such as employee compensation or bond rating. 15) The way players are assessed in addition to game performance scores. The following have been identified as supplemental methods (to game performance) to assess player performance: Peer assessment of player contribution (Anderson and Lawton, 1992; Biggs, 1995), a paper analyzing the team’s performance (Anderson and Lawton, 1992; Biggs, 1997; Knotts and Keys, 1997; Keys, 1994), an oral presentation on the team’s performance (Anderson and Lawton, 1992; Biggs, 1997, Knotts and Keys, 1997), a written plan (Anderson and Lawton, 1992), a test on rules and procedures (Anderson and Lawton, 1992), and the ability to predict the results of a decision (Anderson and Lawton, 1992). 16) The debriefing of the simulation experience, the debriefing session’s length, content and degree of structure. This is a key issue according to many authors including Cavanagh (1994), Sanders (1994), and Thiagarajan (1994). Both Sanders (1994) and Cavanagh(1994) wonder about how to debrief correctly, but provide no suggestions. Note that some of scholarship cited above includes empirical research. Just from these few studies in the above 16 paragraphs, we have empirical evidence that learning levels are higher for students in three or four person teams (Wolfe and Chacko, 1983), that learning (Wolfe, 1975) and simulation performance (Wolfe, 1990) increase with increases in active guidance on the part of the instructor, and that students have more positive attitudes towards their simulation experience when the simulation in integrated into the rest of the course than when the simulation is separated (Snow, Gehlen, and Green,2002). 319 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 It is beneficial to perform research studying the influence of instructor-controlled variables. All of the above variables are under control by the teacher/administrator, which should make undertaking research easier. Because these independent variables are easy to control, results emerging from studies are likely to be relatively clear. If we are interested in discovering the influences on outcomes in simulations, studying the impact of teacher/administrator controlled variables is a great start, in that doing so can help us to identify methods and environments that enhance learning. As John Washbush and I have said (Gosen and Washbush, 1999) in 1999, “there is great potential benefit that should accrue from such investigations. After all, learning is why we are involved with simulations in the first place.” Furthermore using teacher/administrator-controlled variables is feasible for cross-institutional research, which is beneficial because we can begin to generalize what enhances simulation outcomes across universities and not limit our conclusions to one school. REFLECTION The rest of this paper focuses on reflection. Reflection is a part of the overall learning process, a process that for Kolb (1984) includes active experimentation, concrete experience, reflective observation, and abstract conceptualization. According to Thiaragajan (1994) reflection is a key component of the learning process. “People don’t learn from the experience…They learn from reflecting on the experience” (pg. 523). Bowen (1987) also believes that reflection is important, but the process for him involves more than just reflecting. It involves both analysis on the part of the learner and input from an instructor/expert. From a review of the experiential literature, he suggests that learning has greater impact when accompanied by adequate processing time and a clear summary providing a cognitive map for understanding the experience (pg. 197). So borrowing from Bowen (1987), in a simulation, the reflection process involves the following overlapping sub-processes: 1) the learner/player receiving feedback in the form of results from decisions, 2) a contemplation process in which results are taken in, 3) a debriefing procedure in which contemplations are discussed with teammates, other industry players, and/or the teacher/administrator, and 4) an analysis/thinking process in which contemplations are organized into plans, reports, and/or conclusions. The overall construct, reflection, combines Kolb’s reflective observation and abstract conceptualization, in that it involves observing behavior and results, contemplating, perhaps discussing, and formulating ideas on the basis of the contemplation and discussion. The following paragraphs focus on the four sub-processes of the reflection process and on accompanying simulation-related literature. It is relatively clear from this literature that instructors/administrators have considerable control over how and how much learners attend to these sub-processes. For each sub-process, I will refer to articles written on the sub-process and ways in which features of the sub-process can vary as a result of instructor/administrator control. THE SUB-PROCESSES Feedback. The first part of the process involves receiving feedback. Feedback is receiving results from action taken. In virtually all the computerized simulations I know of, players receive feedback after each decision round. Although many authors stress the importance of feedback (see Peach and Roberts, 2003 for a review), few discuss or imply how feedback can vary. Those that do include Fujita and Murahura F&M (2000), Gentry and Burns G&B (1983), Peach and Roberts P&R (2003), and Rosenthal and Werner R&W (1992). Exhibit 1 contains a list of ways the feedback process can vary as a result of instructor influence. Exhibit 1: Ways in which instructors/administrators can vary the feedback process Whether the instructor is involved P&R Whether the instructor adds interpretation to results G&B Consistency from one period to the next P&R Timeliness P&R Degree to which reports are required on the basis of feedback F&M Degree to which the students must actively analyze on the basis of feedback F&M Whether feedback is provided in a setting where debriefing occurs R&W author citation in bold Exhibit 2: Ways in which instructors/administrators can vary a journal assignment Frequency of journal entry McD K&K Amount of structure T Frequency in which journals are turned in K&K Whether or not journals are mandatory K&K Limits on journal length K&K author citation in bold 320 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Contemplation. This term does not frequently appear in the pedagogical literature. I use ‘contemplation’ here as the sub- process or component, so as not to confuse with the broader construct ‘reflection’, which is a major focus of this article. So for this article, reflection is the construct; contemplation is the sub-process. Contemplation by itself is a mental activity; therefore it would seem hard to concretize. The one way that contemplation has been operationalized is with journals. Taylor (1998) argues that keeping a journal means writing and by writing one learns. Knotts and Keys (1997) agree and say journals prompt players to think through their activities, which insures greater learning. Knotts and Keys K&K (1997), McDevitt McD (2000), and Taylor T (1998) are among the authors that write about both reflection (in the narrow sense) and players keeping journals. Exhibit 2 contains a list of the ways in which keeping journals while playing a simulation can vary as a result of instructor influence. Debriefing. Although there have been other definitions, (See, Markulus and Strang, 2003), this paper defines debriefing narrowly as consisting of an oral interchange or discussion that helps to shed light on an experience. Lederman (1984) defines it as an oral student-teacher question-and-answer session to guide students through the reflective process, and Harry (1971) also suggests that debriefing includes a debriefing ‘session.’ For Markulus and Strang, (2003), debriefing is not feedback. The exact nature and form of debriefing varies. The following authors write about debriefing and suggest or imply ways in which debriefing can vary: Dutton D (1979), Hunsaker H (1978), Knots and Keys K&K (1997), Markulus and Strang, M&S (2003), Rosenthal and Werner R&W (1992), Warrick Wa(1978), and Wolfe Wo (1990). The ways in which these authors suggest instructors can vary debriefing is contained in exhibit 3. Analysis. The last component of the reflection construct is the analysis that comes after contemplation. It involves thinking and the formulation of the conclusions that result from thinking. If this part of the learning process takes place during the game, planning and predictions emerge from the analysis and thinking. If this part of the learning process takes place at the end of a simulation, conclusions or summations emerge. I’m not aware of any scholar who focuses on this part of the learning process, as a learning process. On the other hand, some do write about the kind of reports that can students can produce as a result of analysis. These writers include Anderson and Lawton A&L (1992), Biggs B (1997), Comers and Nichols C&N (1994), DiBattista DiB (1986), Faria and Nulton F&N (1974), Knots and Keys K&K (1997), and Zalatan Z (2000). Exhibit 4 lists the ways in which the results of analysis can vary as a result of instructor influence given the ideas of the above authors. Exhibit 3: Ways in which instructors/administrators can vary debriefing Format (e.g., stock holders’ meetings and press conferences) R&W Wo Amount of structure M&S Wa Use of technology M&S Whether experts are used K&K M&S R&W Whether there is an evaluatory purpose M&S Degree to which debriefing is linked to instructor specified content or skills D Hu Wa Whether debriefing includes feedback from and for every participant D Degree of planning Wa Degree to which and ways in which participant contribution is encouraged Hu Whether debriefing occurs between decision making periods R&W author citation in bold Exhibit 4: Ways in which instructors/administrators can vary the outcomes of analysis Types of reports A&L F&N Whether graded (sometimes oral presentations are not) K&K Grade weights A&L Whether non-class members are involved in assessment or as observers K&K B Comprehensiveness of reports Z Whether repeated or one time only F&N Z Whether reports cover other parts of the course and other courses as well as the simulation Z Degree to which requirements are structured C&N Z Whether students are required to turn in reports DiB author citation in bold “Reports” includes presentations, conclusions, and plans 321 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 DISCUSSION REGARDING REFLECTION AND CONCLUSION These sub-processes probably overlap. People receive information, process some, come to conclusions, process more, receive more information, think, change conclusions, and go back and forth between receiving, processing, and concluding. In addition, some of the authors who write about the reflection construct write as if the overlap exists. For example, Bruce’s (2001) focal topic is refection, but in his article he seems to be measuring not reflection but how long it takes students to analyze and predict. In addition to assessing reports, Biggs’s (1997) board of directors participates in a debriefing session. Wolfe’s (1990) paragraph that begins by focusing on the importance of feedback contains suggestions for debriefing. Fujita and Murahura (2000) and Rosenthal and Werner (1992) both contend that their feedback system is an effective debriefing tool, and Fujita and Murahura (2000) argue that the feedback allows students to establish plans. A few of the above papers report research results. Comer and Nichols (1994) found that students gave the course higher ratings when the requirements for the reports were more structured. Bruce (2001) found that time spent in the reflection/analysis process decreased over the course of the game to a greater degree for high performing teams than low performing teams. DiBattista (1986) found that a group required to turn in an assignment scored higher on an achievement test than a group given the same assignment but not required to turn it in. Unfortunately, of the papers that deal with the reflection process, the above three are among the few that report empirical research. However, the fact that these studies have been performed supports the argument that such research is feasible. Above I broke the reflection construct into components and showed how many of the specific aspects of the process could be manipulated by the instructor/administrator (IA) in the simulation to affect outcomes. For example, whether IA’s interpreted quarterly results for students could influence outcomes in a simulation, as could the quantity of required input by students in a debriefing session, as could the degree of structure required for annual report submission. The variables that influence simulation outcomes need not be the ones depicted above, and they need not be as specific or tied to a particular refection sub-process. Researchers could study the influence of variables that are properties of the entire reflection process rather than just a component of it. For example one could study the influence of structure. In one condition, feedback, debriefing, and report requirements could be highly structured while in another condition, feedback, debriefing, and report requirements could be less structured. One could also study the degree to which the reflection is a creative process for students. In one condition, assignments could require students to undertake analysis and create documents. In another condition the reflective process might be more passive, with students listening to instructor interpretation and responding to already-developed specific questions. Studying the influence of teacher-controlled reflection- related variables is an area ripe for cross-institutional research. There’s no reason why our research results must be unique to one university. For example, instructor A at university A and instructor B at university B could debrief with a structured format and require reports following strict guidelines, while instructor A at university B and instructor B at university A could debrief with an unstructured formant and require reports with less structured guidelines. As I said above, cross- institutional research is beneficial because we can begin to generalize what enhances simulation outcomes across universities and not limit our conclusions to one school. Reflection is a construct that covers learning from immediately after the experience to just before what Kolb (1984) calls active experimentation. This paper contends that reflection contributes considerably to learning and that in the simulation, variables in the reflection process can be significantly influenced by the teacher/administrator (IA). If reflection contributes significantly to overall learning, and if reflection variables are easily controlled by IAs, then it should be both easy and achievable to undertake the research to discover variables that influence learning in the simulation. REFERENCES Affisco, J. F. (1994). “ My experiences with simulation/gaming.” Simulation & Gaming, 25, 166-171. Anderson, P.H. and Lawton, L. (1992). “A survey of methods used for evaluating n student performance on business simulations.” Simulation & Gaming, 23, 490-498. Anderson, P.H. and Lawton, L. (2003). “The optimal timing for introducing business simulations.” Developments in Business Simulations and Experiences, 30, 1-4. Biggs, W. D. (1995). “Reflections on my involvement with computerized business simulations.” Simulation & Gaming, 26, 159-167. Biggs, W. D. (1997). “ The use of boards of directors to evaluate reports and presentations in an undergraduate business policy course.” Developments in Business Simulations and Experiences, 24, 274-277. Bowen, D. D. (1987). “Developing a personal theory of experiential learning.” Simulation & Gaming, 18, 192-206 Bruce, M. L. (2001) “Reflecting on reflection: an examination of reflective learning and assessing outcomes.” Developments in Business Simulations and Experiences, 28, 8-12. Butler, R. J., Pray, T.F. and Strang, D. R. (1979). “An extension of Wolfe’s study of simulation game complexity”. Decision Sciences, 10, 480-486. Cavanagh, T.K. (1994). “From Simgames to Africa (and back): A retrospect from Canada.” Simulation & Gaming, 25, 185- 192. Christopher, E. M. (1999). “Simulation and gaming as subversive activities.” Simulation & Gaming, 30, 441-455 Comer, L. B. and Nichols, J. A. F (1994). “Enhancing a computer simulation with a structured reporting environment.” Developments in Business Simulations and Experiences, 21, 40-44 DiBattista, R. A. (1986). “Using a business simulation in a Principles of Management course - learning outcomes and 322 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 perceptions.” Developments in Business Simulations and Experiences, 17, 28-32. Peach B. and Roberts R. (1999). “Assessing the affects of feedback: Multi-methods and multi-directions in multi- pedagogical courses.” Developments in Business Simulations and Experiences, 26, 124-129 Dickenson, J. R. (2003). “The feasibility of the balanced scorecard for business games.” Developments in Business Simulations and Experiences, 30, 90-98 Pimentel, F.Q. (1995). “Gaming/Simulations: A Third World Experience.” Simulation & Gaming, 26, 480-488. Dutton, R. E. (1979). “Customized debriefing: the Achilles Heel of experiential learning.” Insights into Experiential Pedagogy, 6, 313-314 Raia, W.E. (1966). “A Study of the Educational Value of Management Games.” Journal of Business, 39, 339-352. Faria, A. J. and Nulsen R. O. (1974). “The use of internal and external assignments with a marketing simulation game.” Simulations, Games and Experiential Learning Techniques, 1, 54-60. Rausch, E. (1995). “Simulations - One Man’s Experience and a New Challenge.” Simulation & Gaming, 26, 213-218. Rosenthal, W. and Werner, W. J. (1992). “The use of a ‘board of directors’ to evaluate and validate decisions in a competitive graduate management simulation course.” Developments in Business Simulations and Experiences, 19, 221 Fujita, K. and Murahara, S. (2000). “Management game review system development.” Developments in Business Simulations and Experiences, 239-241 Saunders, D. (1994). “Accidental Involvement in Simulation Gaming.” Simulation & Gaming, 25, 245-250. Garris, R., Ahlers, A., and Driskell, J. E. (2002).”Games motivation and learning: A research and practice model.” Simulation & Gaming, 33, 441-467. Scott, T.W., Strickland, A.J., Hofmeister, D.L. and Thompson, M.D. (1992). Micromatic: A Management Simulation. Boston: Houghton Mifflin. Gentry, J. W. and Burns A. C. (1983). “Do we learn from experience?” Developments in Business Simulations and Experiences, 10, 139-142 Snow, S., Gehlen, F., and Green, J., (2002). “Does student preparation matter in a simulation? A comparison of pedagogical styles.” Developments in Business Simulations and Experiences, 29, 208-217 Goodman, F.L. (1995). “Practice in theory”. Simulation & Gaming, 26, 178-189. Taylor, D. B. (1998). Using student Journals to Enhance Learning from Simulations. Developments in Business Simulations and Experiences, 25, 154-155. Gosen, J. and Washbush, J. (1999). “As teachers and researchers, where do we go from here?” Simulation & Gaming, 30, 192-203 Thiagarajan, S. (1994). “How I Designed a Game - and Discovered the Meaning of Life.” Simulation & Gaming, 25, 529-535. Green, J. C., McQuaid, B., and Snow, S. (2002). “The use of external interventions in computer bases simulation.” Developments in Business Simulations and Experiences, 29, 74-81 Walford, R. (1995). “A Quarter Century of Games and Simulations in Geography.” Simulation & Gaming, 26, 236-248. Harry L. (1971). “Using Simulations in the classroom. Feedback on Instructional Simulation Systems” Simulation and Games, 3, 127-134 Warrwick, D. D. (1978). “ Insights into Debriefing Experiential Learning Exercises.” Hill, J. L. and Lance C. G. (2002). “Debriefing stress.” Simulation & Gaming, 33, 490-503 Experiential Learning Simulations and Experiential Exercises, 5, 9-10 Hunsaker, P. L. (1978). “Debriefing: The key to experiential learning.” Exploring Experiential Learning Simulations and Experiential Exercises, 5, 3-4 Wheatley, W.J. (1994). “Personal Reflections on My Experiences with Simulations.” Simulation & Gaming, 25, 269-273. Knotts, U. S. Jr. and Keys B. (1997). “Teaching Strategic Management with a business game.” Simulation & Gaming, 28, 360-376 Wolfe, J. (1975). “A comparative evaluation of the experiential approach as a business policy learning environment.” Academy of Management Journal, 18, 442-452 Kolb D. A. (1984). “Learning and problem solving: On management and the learning process.” In D.A. Kolb (ed.), Experiential Learning. Englewood Cliffs NJ: Prentice-Hall, 27-41 Wolfe, J. (1978). “The Effects of Game Complexity on the Acquisition of Business Policy Knowledge.” Decision Sciences, 9, 143-155. Keys, J.B. (1994). “A Business Gaming Odyssey.” Simulation & Gaming, 25, 487-501. Wolfe, J. (1990). “The Evaluation of Computer-Based Business Games.” In James W. Gentry (ed.), Guide to Business Games and Experiential Learning. London: Nichols/GP Publishing. Keys, B. and Wolfe, J. (1990). “The Role of Management Games and Simulations in Education and Research.” Journal of Management, 16, 307-336. Wolfe, J. and Chacko, T.I. (1983).”Team-Size Effects on Business Game Performance and Decision-Making Behaviors.” Decision Sciences, 14, 121-132. Lederman, L.L. (1994). “Give a Small Child a Hammer and Soon Everything Needs Hammering.” Simulation & Gaming, 25, 215-220. Wolfe, J. and Roberts, C.R. (1985-86). “The Effects of Different Grade Weights on Business Game Learning Levels.” Journal of Educational Technology Systems, 17, 45-59. Markulus, P. M. and Strang D. R. (2003). “A brief on debriefing: What it is and what it isn’t.” Developments in Business Simulations and Experiences, 30, 177-184. Wolfe, J. and Roge’ (1997). “Computerized General Management Games: The Effectiveness of Business Games in Strategic Management Course Work.” Simulation & Gaming, 28, 423-441. McDevitt, P. (2000). “Using Journals to Enhance Computer Simulation Based Learning.” . Developments in Business Simulations and Experiences, 27, 57-62 323 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 324 Zalatan, K. A. (2000). “Changing the assessment paradigm: Using student portfolios for assessing learning from simulations.” Developments in Business Simulations and Experiences, 27, 172-178. Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes