EVALUATING SIMULATION LEARNING IN A DISTANT LEARNING INSTRUCTIONAL MODEL Developments In Business Simulation & Experiential Learning, Volume 24, 1997 EVALUATING SIMULATION LEARNING IN A DISTANT LEARNING INSTRUCTIONAL MODEL Denise R. Markovich. University of North Dakota ABSTRACT This paper discusses the use of the simulation Fingame in a graduate class, which was offered via interactive video network (P/N) to two distant sites. It was expected that learning and direct instructional contact would be positively correlated and that correlation would be reflected in superior performance among students at the host site. However, tests of differences between population means indicated that learning and simulation performance were significantly better at the distant sites. Moreover there was a strong, positive correlation between learning and simulation performance. These findings contradict previous research and suggest the need for further investigation into the role of simulations in the learning process. INTRODUCTION Simulations, accompanied by Decision Support Systems (DSS) can play an important role in the learning process (Keys. 1989: Corner & Nicholls, 1994; Nelson & Roussos. 1994: & Peach, 1996). With the rapid growth of long distance learning it is important to know whether simulations can also be an effective learning tool among students who have limited personal interaction with instructors and no personal interaction with students at other sites. This paper compares learning and simulation performance of students at the host site and at distant sites to address this question. Hypotheses The following hypotheses were investigated: H1 There is no significant difference in learning between students at the host site and the distant sites. H2 There is no significant difference in simulation performance between students at the host site and the distant sites. METHOD The graduate course in finance was offered to thirty-seven students at three sites. Among the twenty-three students at the home site, many were recent graduates and most had limited experience in the business world. In contrast, there were eight students at one distant site and six at the other: all were older than average, all had significant business experience, but only two had recent experience in finance. For this investigation, performance was proxied by the final stock price in the Fingame simulation and learning was proxied by the final grade. The final grade was comprised of four equally weighted components: a test: a final exam; four reports evaluating specific financial characteristics of the simulation, and a comprehensive financial analysis of the simulation. Although the simulation was included in the final grade, simulation grading was based on analysis of data and outcomes rather than financial indicators; thus, there is no multicollinearity in the variables, final grade and stock price. For tests of mean differences at significance level of a = 01 and 35 degrees of freedom the critical value of t ±2.727. RESULTS To test the first hypothesis that there is no significant difference in learning between students at the host site and distant sites, data for eight teams were pooled into two groups: the host site and distant sites. Findings are reported in Table 1. 78 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 The hypothesis is rejected; the observed differences indicate that students at the distant sites had significantly higher grades than students at the host site. To the extent that final grades serve as a proxy for learning, statistical evidence indicates students at the distant sites were not adversely affected by lack of on site instruction. Results on simulation performance were similar. Summary data of team performance indicates that students at the distant sites performed significantly better on the simulation than students at the host site. The findings also indicate that learning and simulation performance vary directly. To test the correlation between the variables, final grade was regressed on stock price: the result, a correlation coefficient of 0.82, indicates a strong, positive correlation. When final grade was regressed on stock price, by rank, the correlation coefficient was 0.95 indicating that simulation performance was almost a perfect predictor of learning. The upward bias appears to be a result of ranking. CONCLUSIONS Unlike the findings of Washbush and Gosenpud (1994) that "learning and simulation performance do not co-vary,” this study finds a strong positive correlation. This paper also finds that for students at distant sites learning and simulation performance are not adversely affected: in fact, both learning and performance of students at distant sites were significantly better. However, results are for one class and the results do not include a measure of age. experience, or effort on learning and performance. With the rapid expansion of distance learning it would be helpful to have further studies on the effectiveness of simulations as a teaching tool at remote sites in the assessment of simulations there is a need for continuing research, for more precise identification of variables, and for exploring consistent ways of assessing and integrating qualitative factors into the analysis. Research collaboration across institutions could give greater validity to results and possibly provide an indirect evaluation of the various Decision Support Systems used at participating institutions. REFERENCES Comer, Luecette R & Nicholls J. A. F. (1994) Enhancing a computer simulation with a structured reporting environment. Developments in Business Simulations and Experiential Learning, 21, 40-44 Keys, J.B. (1989). The management of learning grid for management development revisited. Academy of Management Review, 8, 5-12 Nulsen, R. O, Faria, A. J., & Roussos, D. S. (1994) The use of decision support systems with a marketing simulation: the future is now. Developments in Business Simulations and Experiential Learning, 21, 169. Peach, E. Brian. (1996). Enhancing simulation learning through objectives and decision support systems. Developments in Business Simulation & Experiential Learning, 23, 61-66 Rosenshine, B. (1971) Teaching Behaviors and Student Achievement London: National Foundation for Educational Research. Washbash John & Gosenpud Jerry (1994) Simulation performance and learning revisited. Developments in Business Simulations and Experiential Learning, 21, 83-86 79 Table of Contents Volume 24, 1997 Incorporating Computer Telephony into the MIS Course A Learner Oriented Infrastructure for videoconferencing Based Distance Education Courses The Identification of Temporally Related Structural Elements of the Experiential Component of Electronic Spreadsheet Tasks Does Involvement Influence Learning from Simulation Participation? Some Relationships with Helpfulness and Performance Outcomes Musings on Business Garne Performance Evaluation Performance on a TE Simulation: What does it represent? The Business Policy Game: An International Simulation - An Assessment Tool Students in Free Enterprise as Experiential Learning The Role of Computer Models in Wargames: A Practitioner's view Using Focus Groups as a Tool to Research Learning: A Demonstration and a Discussion Using Medical Simulations to Teach Multi-Cultural Diversity Using the World Game to Internationalize International Exchange Game Community Services Needs Assessment: An Innovative Approach Introducing Students to Potential Total Quality Management Ethical Dilemmas Privacy in the Workplace: A Situational Analysis Antecedents of Learning in Simulations Demonstrating the Learning Effectiveness of Simulation: Where we are and Where we need to go The Use of Computer Simulations as a Pedagogical Aid in Teaching Management Information Systems Evaluating Simulation Learning in a Distant Learning Instructional Model Financial Engineering of Global Investments The Labor History Game: Playing with the Past Perks Participation Assessing Negotiator's Proficiency with a Negotiation role-play The Art of Negotiating Enhancing Learning and Employee Development through the Assessment of Learning Pedagogy Preferences across selected Dimensions of Culture: A Preliminary Investigation Cooperative Learning: What are we Learning? Effective Use of Mastery Based Experiential learning in a Project Course to improve skills in system analysis and Design Simulations and Learning: Can we prove a relationship? (Seminar) Predicting and Reviewing NYSE Stock Prices by use of Basic Statistical Analysis and Logic Toy Car Depreciation Exercise ABSEL as Home Community: An Interactive Exploration Service Learning: Linking Academic Study to Community Development and Business Enhancement Ability of Efficient Evaluation of Knowledge-Based Management Strategies Corporate Ethics Training Programs Modeling Attributes in Demand Functions of Computerized Business Simulations: An Extension of Teach's Gravity Flow Algorithm Expert Systems Combined with Neutral Networks: Tools to Benefit the Marketing Researcher Computer Game Design: New Directions for Intercultural Simulation Game Designers Consistency in Simulation Performance over Time and Across Simulation Games The Impact of an Artificial Market Leader on Simulation Competitor's Strategies Business Plans, Case Studies, and Total Enterprise Simulations: A Natural Co-existence An Exploration into the Non-Use of Business Simulations The Market Game: Interactive Learning Through Market Simulation Plotting Brand Trajectories with the COMPLETE PPM package: A Market Segmentation Analysis and Positioning Tool Me and Mine Inc: An Exercise in Management Theory Learning to Differentiate Leadership from Managerial Position Business Ethics Survey: A Perspective from the Retail Industry Experiential Learning in Demand Analysis for an Agricultural Commodity Marketing on the Internet: A Pedagogical Exercise College Students Need Simple Computer Simulations, Especially for International Business Courses An Analysis of Student Attitudes, Performance, and Strategies in a Simulation Competition Based on a Controlled Product-Market-Entry Game Structure The Crystal Enterprise: Application of a Non-Computerized Simulation Model in a Process of Organizational Change Threshold: A Windows-Based Behaviorally Oriented Total Enterprise Simulation Coaching Business Game Teams Using a Decision Variable Optimizer Rock and Roll is here to Stay: Enhancing Experiential Pedagogy with Musical Exercises An Example of Business Process Analysis Simulation for Customer Software Support The Use of Business Gaming in Hong Kong Academic Institutions Using the Integrative International Simulation INTOPIA Mark 2000 in a Concentrated MBA Curriculum A Group Experience as an Integrated Part of the Core Management Course Empowered Learning in the Classroom The Inter-Group Interaction: An Innovative Approach to Cooperative Learning Courses that Utilize Student Team: An Approach to enhancing their Effectiveness Thoughts about the Measurement of Learning: The Case for Guided Learning and Associated Measurement Issues Measuring Student Learning Using Business Simulations: A Theory Based Perspective On the Use of PC Fingame in an Undergraduate Finance Course How Managers get Things Done: A Virtual Soundbite Internet Experiential learning in the Principals of Marketing Classroom: A Pedagogical Approach Current Student Perceptions Relative to Business Simulations Beyond Capitalism: Designing Business Policy and Social Justice The Use of Boards of Directors to Evaluate Reports and Presentations in an Undergraduate Business Policy Course Beacon Lumber: An Experiential Introduction to Financial Accounting The Application of Organizational Motivation Principles: The Experiential Business Simulation Motus Manufacturing Avoiding a Bogey: Grading Case Discussions Scientifically Communicating Consumer Behavior II: A Modified Exercise Using Personal Consumption Journals in Condensed Courses Designing Instruments for Assessing the Effectiveness of Simulations, (Seminar) 360º Performance Feedback: Appraisal vs. Assessment The Cafeteria Approach to Managing an Academic Career: Remaining Non-Perishable while Doing Your Own Thing An Experiential Exercise Related to Person-Organization Fit and it's Consequences For Today's Dynamic and Changing World, Business Simulations need to be expanded to encompass Much Greater Complexity: A Demonstration The Incident Process: A Case in Reverse Contextually-Anchored Business Simulations The Energy Factor: Building Motivation in the Simulation Gaming Environment The Design of an Internet Game