ThE DESIGN OF AN ITS-BASED BUSINESS SIMULATiON: A NEW EPISTEMOLOGY FOR LEARNING Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 THE DESIGN OF AN ITS-BASED BUSINESS SIMULATION: A NEW EPISTEMOLOGY FOR LEARNING Steven C. Gold, Rochester Institute of Technology ABSTRACT This paper discusses the design and use of an intelligent tutoring system (ITS) for computerized business simulations. It is argued intelligent tutoring systems are a natural complement to computerized business simulations and could significantly enhance their pedagogical effectiveness. The use of ITS as an instructional technology and its success are reviewed. A fundamental model for an ITS-based business simulation is presented. The user interface and the linkage between the ITS and the simulation are examined. It is recommended the ITS consist of three distinct modules: expert consultant, diagnostic testing, and pedagogical support. The integration of ITS with business simulations offers a new epistemology for ‘earning, the significance of which has yet to be fully understood. INTRODUCTION The purpose of this paper is to consider the potential role of an intelligent tutoring system (ITS) within a simulation environment and to discuss the design elements of such a system. Intelligent tutoring systems are not new, they have been under development and evaluation in multidisciplinary laboratories of top scientists in physics, computer science, engineering, and psychology for over a decade.1 Both the ITS and simulation & gaming research communities focus on facilitating or improving learning but surprisingly there is little dialogue between the two communities. Notably, there has been almost no research papers on ITS in the ABSEL conferences and in the journal of Simulation and Gaming over the past ten years! One explanation may be the two communities focus on different stages of the learning process. Consider the apprenticeship model of learning by Collins, Brown & Newman (1987) in which four stages are identified: (1) Modeling; (2) Coaching; (3) Fading; and (4) Reflecting. Modeling occurs when the apprentice (student) observes the master (teacher) illustrating a model or process, which could be a simulation. Coaching involves the teacher advising and directing (mentoring) the student while the student is attempting to work with a model or simulation. Fading is when the teacher’s assistance is reduced as the student becomes more comfortable with the process and performance improves. Reflecting requires the student to evaluate his/her own performance and assess learning. 1 Garcia, Anne L., "Intelligent Training Systems: Smart Tutors", Bulletin of the American Society for Information Science, August/September 1990, p.8. The simulation researchers have concentrated on modeling and the pedagogical process of using the simulation with some attention to reflecting by students as a way of learning from the experience. Experiential learning or learning by doing is the modus operandi. The instructor’s involvement in coaching the students and the process of fading are given relatively little attention in the simulation literature. There is an assumption that the instructor is teaching or coaching the students and providing constructive feedback in the process of using the simulation But the way this is done and the extent to which it is done is not clear. In sharp contrast, intelligent tutoring system (ITS) researchers focus primarily on coaching the student, with some attention to fading, but very little detail on modeling or reflecting.2 ITS research deals with the most effective ways to intervene constructively and guide students that are having trouble performing a certain task (i.e. coaching).3 In this paper we will try to link the pedagogical advantages of the ITS to business simulations to develop a new epistemology for learning. WHAT IS AN INTELLIGENT TUTORING SYSTEM? Intelligent tutoring systems are instructional software programs with the abilities of a human teacher, working on a one-on-one basis with the student, carefully diagnosing what the student knows, how the student reasons, what deficiencies exist in the student’s knowledge, and suggesting ways to help the student to 2 Parkes, A.P. and Self, J.A., “Towards Interactive Video: A Video-Based Intelligent Tutoring Environment”. In Frasson & Gauthier (ed), Intelligent Tutoring Systems (Ablex Publishing Corp. 1990), p.56 3 lbid., p.56. 36 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 best learn the subject material.4 Intelligent tutoring systems are a significant step beyond computer-assisted instruction (CAl). An intelligent tutoring system is CAl with artificial’ intelligence in the knowledge domain and in teaching pedagogy. Burns and Capps (1988) described three types of intelligence that are necessary for a CM to become intelligent. First, the subject matter must be known well enough by the ITS for it to draw inferences or solve problems in the domain, i.e. it must be an expert system. Second, the system must be able to assess a student’s knowledge of the subject matter, i.e. is the student doing what the expert would do? Third, the system must be able to effectively tutor the student and reduce any differences between the student’s performance and that of the expert’s.5 The tutoring system must have effective teaching strategies and be articulate enough to help the student learn and improve his/her performance. Extending intelligence to teaching strategy, a fourth type of intelligence is needed, i.e. the ability of the system to improve its teaching effectiveness with time and experience. The system must have control over its tutorial strategies, determining the best time to interrupt a student’s problem- solving activity, what to say and how to say it.6 The more students the system tutors, the more intelligent (more effective) it will become as a tutor. Just as a human teacher, it will learn from experience (store in its database) the suggestions and advice that helped students and those that did not seem to work. This would require the system to consider and record the background of the student it is tutoring and modify its teaching strategy, if necessary, based on this information and its own experiences. 4 Angelides, MC. and Doukidis, G.I,”Is There a Place in OR for Intelligent Tutoring Systems?” Journal of Operations Research, Vol.41, No.6, 1990, p. 491. 5 Burns H.L. & Capps, C. G., “Foundations of Intelligent Tutoring Systems: An Introduction”, found in Poison, MC. & Richardson, J.J. (ed), Intelligent Tutoring Systems (Lawrence Erlbaum Associates, Inc.: 1988), p.1 6 Sleeman, D. and Brown, J.S. “Introduction: Intelligent Tutoring Systems”, found in Sleeman, D. and Brown, J.S. (ed.) Intelligent Tutoring Systems (Academic Press, 1982), p.2 HAVE INTELLIGENT TUTORING SYSTEMS BEEN SUCCESSFUL? intelligent tutoring systems have been used on a limited bases for more than a decade, but are now emerging from research laboratories in a wide range of areas from foreign language skills and reading, to engineering and optics.7 The reviews on learning effectiveness are mixed, yet it is expected that in the 21st century we will see artificial intelligence become a major pedagogical tool.8 Shute (1990) offered two fundamental reasons for the mixed reviews on intelligent tutoring systems. Many of the ITSs were designed by seat of the pants” engineering and intuition regarding system components, (Koedinger & Anderson, 1990). Second, many of the evaluations were sacking systematic control groups, e.g. Baker (1990) and Littman & Soloway (1988). Bonar, Cunningham, & Shults (1987) argued that most ITSs evaluated prior to 1987 were not only complex and unwieldy, but contained fundamental design flaws related to the use of knowledge within the intelligent tutoring system. Specifically; these systems did not allow for additions of new domain knowledge or new approaches to the pedagogical tasks.9 Recognizing some of the deficiencies of the past studies with evaluating the pedagogical effectiveness of ITSs, Shute (1990) selected four simulations to test that were well designed, these included: (1) The LISP tutor by Anderson, Farrell, & Sauers (1984), teaching Lisp programming skills; (2) SMITHTOWN by Shute & Glaser (1 990), a tutor for scientific inquiry skills in microeconomics; (3) SHERLOCK by Lesgold, Lajoie, Brunzo, and Eggan (1990), teaching avionics troubleshooting; and (4) PASCAL ITS by Bonar, Cunningham, Beatty, & Weil (1988), a tutor on Pascal programming. With each ITS, Shute compared the student’s performance to a control group. Shute’s findings were threefold. First, learning rates were 7 Garcia, Anne L., Intelligent Training Systems: Smart Tutor?, Bulletin of the American Society for Information Science, August/September 1990. 8 Piskurich, George “The Possible Futures of Instructional Technology”, Training and Development, March 1993, p.52 9 9Bonar, J., Cunningham, R., and Schults, J. “An Object- Oriented Architecture for Intelligent Tutoring Systems”, Learning Research and Development Center, University of Pittsburgh, Technical Report No. LSP-3, August 14, 1987, page 2. 37 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 increased without any decrease in final outcomes, i.e. students were able to learn the knowledge and skills faster than from traditional pedagogical approaches. Only one-half to one-third the time was needed by the students to ‘earn the material and perform equally as well as the control’ group on exams. Second, the range of learning outcomes (variation in student test scores) were the same (no significant difference). Other recent studies have shown similar results. Anderson, Boyle, et.al (1990) evaluated a geometry and an algebra tutor and found that student do seem to learn from the programs. Pre and posttests of knowledge showed statistically significant improvement in test scores. But only the algebra tutor compared the results to a control group. The geometry tutor study did not use a control group. Godin and Rao (1991) developed a prototype tutoring system for teaching an EOQ model using VP-Expert. They found it to be an effective device to provide a flexible advisory system to the student but did not detail the tests used to support their conclusion. Despite the success stories, development and use of ITSs have been slow. A number of reasons were offered by Woolf, Soloway, et.al (1991). First, there is still a lack of tutoring-specific artificial intelligence development tools, such as shells. Second, it is not a simple task to reduce cognitive analysis to field specific applications. Third, the development cycle is lengthy from research lab to salable products. WHY DO WE NEED AN INTELLIGENT TUTORING SYSTEM FOR BUSINESS SIMULATIONS? Much learning in business simulations occur through the use of the scientific discovery process. The scientific process involves four types of activities (Langley, et.al, 1987): experimental data gathering; the search for important cause and effect relationships that can be described using quantitative or qualitative theorems or rules; explanations of the outcomes by formulating hypotheses; and finally, testing the hypotheses by predicting and verifying outcomes. Bergeron and Paquette (1990) have characterized this process in five main phases: 1. Free exploration to determine the general way the major variables impact the simulation. 2. Structured exploration where specific variables are systematically examined to discover more exact cause and effect relationships. 3. Hypothesis generation to clearly explain the likely impacts of key variables, usually described in the form of equations, rules or theorems. 4. Prediction formulation to test the validity of the hypothesis. 5. Reviewing to identify new knowledge and reinforce strategies, or start new explorations. In order to go through this process effectively it is assumed: (1) the student has the metacognitive skills to know what data to explore, how to develop and test hypotheses, and when to review and start new explorations; (2) students are not reluctant to ask questions; and (3) there are adequate opportunities to ask questions on a timely basis, i.e. when the answers to the questions are needed. Problems arise in the learning process when students are confused and are not able to get their questions answered satisfactorily and on a timely basis. Misunderstandings may propagate and compound as execution of the simulation continues. If questions are not addressed as they occur they may be forgotten. Post-execution teaching/ tutoring is not as effective as “real-time” teaching/tutoring, i.e. when the student is executing the simulation. These limitations are summarized in Table 1. Phases 1 & 2 are supported by allowing students to change decisions, execute the simulation, and generate new data and reports. Decision support programs like spreadsheets are also packaged with many simulations. However, the queries are limited to those contained in the simulation program and the information provided is mostly quantitative. Phases 3 & 4 and 5 are supported by market research reports and the ability, in some simulations, to do "what-if” analysis where the student can re-enter decisions to see what would have happened if different decisions were made. However, no qualitative feedback concerning the reasons for the solution or inferences relating to the solution can be obtained from the simulation program. The essential role of an intelligent tutoring system for simulations is to help overcome these problems and improve learning effectiveness. The ITS will be able to provide the student with immediate help on-line" and support the coaching phase of an apprenticeship method of teaching. The ITS will be able to assist students in exploring the simulation data and constructing conceptual models prior to executing the simulation. This will enhance the support structure to better understand the simulation environment and process, which would not occur with passive observation. The ITS will not tell students what decisions to make but will intervene constructively when students are having trouble making decisions, performing, or when students simply have questions or need guidance. The anxiety and frustration of students so commonly reported in the initial phases of simulation 38 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 TABLE 1 EVALUATION OF SIMULATION DESIGN BY PHRASE IN THE SCIENTIFIC DISCOVERY PROCESS Phases Supporting Functions Limitations 1 & 2 Free & Structured Explorations -selection -data & report generation -graphical presentations -spreadsheet analysis -limited queries & processing -rigid display of data -informative mostly quantitative 3 & 4 Hypothesis & Predictions -market research reports -what-if analysis -no qualitative computer feedback 5 Reviewing -historical reports -replay of simulation -no qualitative computer feedback THE DESIGN OF AN ITS-BASED SIMULATION User Interface In a conventional PC-based business simulation the user goes through the following fundamental procedure: reviewing past reports, entering decisions, executing the simulation, studying the results and, perhaps, doing some data analysis using other decision support systems like a spreadsheet program. The student repeats the procedure each period of play. With an ITS the student can interrupt the simulation at any time (by clicking an icon of a tutor) and enter into a discussion with the tutor” about the simulation, which may include the following categories: 1. The objects or words on the screen, e.g. What does Menu Choice X do?; What does this word mean? What are the effects of this decision? 2. The event in progress, e.g. suppose the student is viewing past reports on a computer screen and asks: what should I do with this information? How can I interpret the information or how can it help me? What should I do next? 3. The performance of the user, e.g. How am I doing? What went wrong? How can I improve? Category 1 above (explaining objects or words) may seem like a standard “help” routine, but an ITS can do much more.. The ITS would respond using natural language explanations and could behave just as an instructor, i.e. by first asking the student what he/she thinks is the answer. The ITS could discuss the answer with the student and then based on the dialogue and the student’s progress in the simulation, make suggestions and refer the student to other sources of information. For categories 2 and 3 the ITS would enter into a “natural language” dialogue with the student. The responses of the ITS would be based on the pedagogical approach or teaching style embodied in the system. The ITS may also interrupt the simulation automatically and provide the student with advice and coaching. ITS Model The tutoring system needs to be intelligent in three areas. One is the knowledge the system has of how to manage and operate the business simulation. Second is the understanding of the student’s knowledge, both strengths and weaknesses. Third is the principles by which it helps the student learn. Given these areas of intelligence, it is recommended the ITS be designed with three distinct modules: expert consultant, diagnostic testing, and pedagogical 39 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 support.10 Each module may be programmed using an expert system shell or artificial intelligence (Al) development tool with the appropriate domain knowledge. The expert consultant module contains the domain knowledge pertaining to the use and management of the business simulation. The consultant module must contain all the knowledge & skills necessary to manage effectively a business simulation. The diagnostic testing module evaluates how well the student is doing and what the student knows. The purpose of the module is to: (1) determine the differences between the student and expert decision-making; and (2) identify the types of knowledge the student needs to learn. This determination is based on the types of questions the student asks the ITS and on how the student responds to questions asked by the ITS to the student. The pedagogical support module identifies which deficiencies in knowledge to focus on and selects pedagogical strategies to present that knowledge. The pedagogical support module needs to determine when students need help, how to provide that help and respond to student questions. The three modules are linked to the user interface and simulation program as shown in Figure 1. The user interface allows the student to run the simulation model by viewing past reports, entering decisions, and generating results. Based on these results the student has the option to continue playing the simulation without using the ITS or the student my request some help. The expert consultant module would respond to any question relating to the management of the simulation; and the diagnostic testing module would then assess the student’s knowledge (and may respond with some additional queries to the student). This information would be accessible to the pedagogical support module to determine the most effective tutoring strategy. If the student does not request tutoring help, the ITS still continues to monitor and diagnose the student’s performance. The pedagogical support module would determine if some intervention would be appropriate based on its teaching strategy and the findings of the diagnostic- testing module. The decision to intervene on the student’s behalf is important to minimize frustration and provide some guidance to prevent the student from going too far off track. Early intervention, however, could hinder learning-by-doing and add to the frustration of the student. The pedagogical approach of the designer must be carefully considered with respect to intervention. 10 A thorough discussion of the design elements of! ITSs may be found in Poison & Richardson, (eds.) Foundations of Intelligent Tutoring Systems, Hillsdale, N.J.: Lawrence Erlbaum Associates, Inc., 1988. FIGURE 1 DESIGN OF ITS-BASED SIMULATION Simulation Model U S E R I N T E R F A C E View Past Reports Enter Decisions Generate Results Student Requests Tutor Tutor Automatically intervenes ITS MODEL •Expert Consultant Module •Diagnostic Testing Module •Pedagogical Support Module CONCLUSIONS It is argued Intelligent Tutoring Systems are a natural complement to computerized business simulations and could significantly enhance their pedagogical effectiveness. Considering the apprenticeship model of learning by Collins, Brown & Newman (1987), where “coaching” and “fading” are identified as important elements in the experiential learning process, it is critical for an instructor to be available for students while they are working with a business simulation. Owing to the many students in a typical classroom, this is not always practical. With an ITS-based simulation, the actions performed by the student while running the simulation may be monitored by the ITS. If the student is successful, the system could make relevant comments concerning the student’s progress to help reinforce important concepts and motivate the student further. The system could also ask meaningful questions and, based on the student’s response, point out those aspects of the student’s management 40 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 strategy that are most valuable to successful business performance. If the student is not doing well, the ITS could raise relevant questions to help the student understand the reasons for the low outcomes and offer some suggestions. The advance in computer technology has made the development and use of intelligent tutoring systems economically feasible. Expert system shells and Al software development tools are commercially available at reasonable prices. It is expected that Al applications like intelligent tutoring systems will become major pedagogical tools by the 21st century (Piskurich, 1993). The time is now for designers of business simulations to leap into the future of instructional technology and consider the use of intelligent tutoring systems. REFERENCES Anderson J.R., Boyle, C. F., Corbett, A.T., and Lewis, M.W. (1990). Cognitive Modeling and Intelligent Tutoring”, Artificial Intelligence, Vol. 42, p.7-49. Anderson, J.R., Farrell, R., & Sauers, R. (1984) “Learning to Program in LISP,” Cognitive Science, Vol. 8, pp. 87-129 Baker, E.L. (1990) “Technology assessment: Policy and methodology issues, in J.L. Burns, J. Parlett, and C. Luckhardt (Eds.), Intelligent Tutoring Systems: Evolutions in Design, Lawrence Erlbaum Associates, Hillsdale, NJ Bergeron, A. and Paquette, G. (1990).“Discovery Environments and Intelligent Learning Tools in Frasson and Gauthier, (eds.), Intelligent Tutoring Systems; At the Crossroads of Artificial Intelligence and Education, (Norwood, N.J.: Ablex Publishing Corporation), pp. 34-55. Bonar, J., Cunningham, R., Beatty, P. and Weil, W (1988). "Bridge: Intelligent Tutoring System with Intermediate Representations. Technical Report, Learning Research & Development Center, University of Pittsburgh, Pittsburgh, PA. Bonar, J., Cunningham, R., and Schults, J. (1987) “An Object-Oriented Architecture for Intelligent Tutoring Systems”, Learning Research and Development Center, University of Pittsburgh, Technical Report No. LSP-3, August 14, 1987 Burns, H.L., & Capps, C.G. (1988). “Foundations of Intelligent Tutoring Systems”, found in Poison & Richards(eds.), Intelligent Tutoring Systems. Lawrence Erlbaum Associates, Inc., pp.1-20. Collins, A., Brown., J.S., & Newman, SE. (1987) Cognitive apprenticeship: Teaching the creaft of reading, writing and Mathematics. In L.b. Resnick (Ed.) Cognition and Instruction: Issues and Agendas. Hillsdale, NJ: Lawrence Erlbaum. Garcia, Anne L. (1990). “intelligent Training Systems: Smart Tutors”, Bulletin of the American Society for Information Science, August/September issue. Godin, V.B. & Rao, A. (1991) “Utilize Expert Systems As Teaching Assistants”, Industrial Engineering, January issue, p.50-52. Koedinger, K.R. and Anderson, J.R. (1990) "Theoretical and Empirical Motivations for the Design of ANGLE: A New Geometry Learning Environment,” Working Notes: AAAI Spring Symposium Series, Stanford University, Stanford, CA. Langley, P., Simon, H.A., Bradshaw, G.L., & Zytkow, J.M. (1987). Scientific Discovery Computational Expectations of the Creative Processes. Cambridge, MA: MIT Press. Lesgold, A., Lajoie, S.P., Bunzo, M., and Eggan, G. (1990) “A Coached Practice Environment for an Electronics Troubleshooting Job”, in J. Larkin R. chabay and C. Sheftic (Eds.) Computer-Assisted Instruction and Intelligent Tutoring Systems: Establishing Communication and Collaboration, Lawrence Erlbaum Associates, Hillsdale, NJ. Littman, D. and Soloway, E. (1988) “Evaluating ITSs: The Cognitive Science Perspective," in M.C. Poison and J. Richardson, Foundations of Intelligent Tutoring Systems, Lawrence Erlbaum Associates, Hillsdale, NJ Piskurich, George (1993) “The Possible Futures of Instructional Technology”, Training and Development, March issue, p.51-53. Shute, V.J. and Glaser, R. (1990) “An Intelligent Tutoring System for Exploring Principles of Economics,” in R.E. Snow & D. Wiley (Eds.) Improving Inquiry In Social Science: A Volume in Honor of Lee J. Cronbach, Lawrence Erlbaum Associates, Hillsdale, NJ Shute, Valerie, J. (1990) Rose Garden Promises of Intelligent Tutoring Systems: Blossom or Thorn?”, Paper presented at the Space Operations, Applications and Research (SOAR) Symposium, also contact V. Shute, AFHRL, Brooks Air Force Base, TX 78235-5601 41 Developments In Business Simulation & Experiential Exercises, Volume 23, 1996 Woolf, Beverly P., Soloway, E., Clancey, W., Van Lehn, K., and Suthers, D (1991) "Knowledge-based Environments for Teaching and Learning” Al Magazine, Special Issue, pp. 74- 77. 42 Table of Contents Volume 23, 1996 Modeling Advertising Effectiveness Simulation as an Aid to Learning: How Does Participation Influence the Process? Administering Business Simulations in Transitioning Economies: The Introduction of Simulation Gaming to Estonia Business Simulation Games: Current Usage Levels. A Ten Year Update The Relationship Between Interpersonal and Task Cohesiveness and Performance in a Business Simulation Game The Design of an ITS-Based Simulation: A New Epistemology for Learning Correlates of Learning in Simulations How Do We Know where we're going if we don't know where we have been: A Review of Business Simulation Research Making Cash Flow Come Alive and Sensible in the Classroom Enhancing Simulation Learning through Objectives and Decision Support Systems An Analysis of Deliberate and Emergent Strategies Relative to Porter's Generic Differentiator and Cost Leader: A Bias and Variance Modeling Approach Introducing Ethical Dilemmas into Computer-Based Simulation Exercises to Teach Business Ethics CEO Strategic Locus of Control Effects on Game Performance and Playing Behavior The Relational Database As a Link between Operations and Cost Accounting Goal Setting over Time in Simulations Computerized Business Simulations: A Workshop Exploring the Tutor's Role, Task & Needs Strategic Analysis of the Product Portfolio with the COMPLETE PPA Package: A Strategic Market Planning Tool Draft Standards and Registration Procedure for Assessment Instruments Perspectives on a New Generation of Business Games An Economic Multiple Regression Case In Experiential Learning Changing Institutional Norms and Behavior, Not Culture: Experiential Learning Comes to Myanmar Strategic Management and the Case Method: Survey and Evaluation Individual Differences in Internet Attitude and Use Long Live the Plan - or Should It? Examining the Impact of Detailed Strategic Plans on Organizational Performance Do Your Students Really Read the Manual? A Computerized Contextual Tool As A Surrogate for the Traditional Student Manual Pilot Analyses of Self-Peer Evaluations in an Experiential-Exercise Human Resources Management Course Leader Behavior Feedback: A Learning Exercise Dilemma-Dilemma: An Exercise for Teaching Significance Of Communication Computer Mediated Conferencing: Technology and Classroom Learning Multimedia in the Workplace: Who is really using it and where is it Headed? Using Experiential Exercises for Collecting Research Data: Integrating Teaching and Research Interactive Distance Learning as a Tool in a College's Theory and Practice The President's Decision: An Experiential Exercise in Decision Making Integrating Computer Literacy Skills in the Undergraduate Curriculum: The Advanced Accounting Experiment Imperatives for the Transfer of Experience-Based Training Deciding How to Decide The Internet as a Pedagogical Tool Internet Scavenger Hunt Two Management Exercises Based on Committee Work Multimedia in the Year 2000: How Will It Affect Our Lives? Bootstrap Benefit Segmentation: Finally A Way to Teach Benefit Segmentation without Primary Data or Those Fancy Statistical Methods Multimedia and Learning: Is There A Connection? A Changing Business Policy Collaborative Learning Through Real-Life Assignments in Accounting Classes The Necessity Of Incorporating Local Cultural Aspects Into International Business Experiential Exercises Utilizing Cultural and International Landmark Constructs to Assess Business Student's International Awareness Legal Issues related to the Use of Application Blanks: An Experiential Exercise The Family in the Classroom: An Experiential Exercise for Teaching Issues Related To Expatriate Assignments Using Internet Resources to Enhance Teaching of Information Systems Courses: A Demonstration Proposal Chalkboards to Chipboards for Teachers and Consultants How Do We Measure The Learning In Experiential Learning and How Do We Best Simulate It?