EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL Developments in Business Simulations and Experiential Learning, Volume 32, 2005 EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL James Schreier Marquette University james.schreier@marquette.edu ABSTRACT BACKGROUND The Implications Wheel® is part of Joel Barker’s Strategic Exploration Tools based on the Possibilities Cone®. This is a series of tools that an organization can use to explore the future. The Possibilities Cone® represents how we view the possibilities of our future, and how we filter them through our paradigms, intuition, strategic exploration, strategic planning, and tactical implementation. Strategic exploration, according to Barker, is the least well done part of our thinking process. TIPS Teams® allow use a technique for collecting information on Trends, Innovations, and Paradigm Shifts. The Vision Album® allows an organization to envision possibilities, early in the process for exploration, later for detailed vision creation. The Strategy Matrix® gives organization a method for evaluating trends, new objectives, policy changes against current objectives and key organizational characteristics. Most simulations have an objective of allowing students to explore the future. Ideally, they have the opportunity to try out different scenarios in an attempt to make better decisions. From a different perspective, futurist Joel Barker has promoted the use of “strategic exploration” tools to better understand the future. One of these tools, the Implications Wheel®, is specifically designed to explore the possible outcomes of a trend, a policy, or a strategic objective. This paper will briefly explain the Implications Wheel® and present the results of an innovative experiment using the Wheel® as an evaluation process for a human resources management simulation. INTRODUCTION It would be easy to argue that most simulations have an objective, perhaps unstated, of allowing students to explore the future. Ideally, they have the opportunity to try out different scenarios in attempt to make better decisions, not only in the future periods of the simulation but also in their professional futures. Over thirty years of experience with management, international, human resources, and bank management simulations has proven to me that actual results, measured by stock price, ROI, and ROA show learning curves that show learning occurring at an increasing rate over time. THE EXPERIMENT In the Fall Semester, 2004, 25 MBA students were enrolled in an elective graduate course, Ethical Issues, Regulatory Environment, and Human Resources Management. A major component of the class was team participation in the Human Resources Management Simulation, Gerald R. Smith and Peggy Golden, Prentice- Hall, 2nd Edition, 2005. The Human Resources Management Simulation is the simulation of a Human Resource department with decisions on hiring, promotions, salary levels, benefits, numerous program decisions on training, quality, and management, plus a short case study with each quarterly decision. From a completely different perspective, futurist Joel Barker has advocated the use of “strategic exploration” tools for several years to better understand the future. He argues that it is a “leader’s responsibility to take care of the future.” He has put these beliefs into practice by developing a suite of practical tools for exploring the future. One of these tools, the Implications Wheel®, is specifically designed to explore the possible outcomes of a trend, a policy, or a strategic objective. It is a popular theory today that innovation occurs at the verge – the intersection of seemingly disconnected ideas. If so, perhaps there’s a new alternative for evaluating learning in a computer simulation by using Joel Barker’s Implications Wheel®. This paper will briefly explain the Implications Wheel® as a strategic exploration tool, discuss the specifics of an experiment using this tool to evaluate a simulation used in a graduate business class, and present the results of this evaluation process. As part of the class, students were also taught how to use Joel Barker’s Implications Wheel®. It was introduced in the second week of class, three weeks before the simulation began, to explore the possible implications of workforce demographic changes forecasted for 2008 and beyond. This training is briefly described below to provide an initial example of the process. Then the students, at the halfway point of the simulation, four decisions into an eight decision game, were assigned the task of evaluating the simulation using the Implications Wheel®. 389 mailto:james.schreier@marquette.edu Developments in Business Simulations and Experiential Learning, Volume 32, 2005 THE PROCESS The Implications Wheel® is a strategic exploration tool which empowers organizations to effectively explore the future. It can reduce the likelihood of “unanticipated consequences,” and increase the likelihood of an organization’s ability to avoid the barriers which might prevent the successful achievement of its strategic objectives. It can build bridges to ensure the success of a positive innovation or reaction to a trend. While the Implications Wheel® won’t predict or guarantee the future, it will help organizations realize the value of futurist Kenneth Boulding’s comment: “The future will always surprise us, but we don’t have to be dumbfounded.” The Implications Wheel® has been successfully used in the strategic exploration of innovations, trends, strategic objectives, policy changes and events, for anything which poses the question: “Just what the heck are we getting ourselves into?” “We need to do a Wheel on that,” has become a mantra in a variety of organizations aware of its value in uncovering implications that otherwise may never have been seen until it was too late. The process capitalizes on the power of diversity, the ability to make connections, and the arguments presented in the widely acclaimed “The Wisdom of Crowds.” The Implications Wheel® works by creating a process in which groups of individuals explore the possible implications of a trend, an innovation, a new strategic objective, a policy change, or any event worth considering “what might happen if…” It is a decision-enhancing tool that can identify possible implications before they happen, which gives an organization time to prepare alternative responses. The process follows a strict set of rules to create a series of connected “arcs” exploring what might happen (first-order implications), then what might happen (second- order implications), and then what might happen (third- order implications), and further if needed. Scoring, for desirability and likelihood, and timing options give the Implications Wheel® additional decision-enhancing power. And finally, the results from the Implications Wheel® provide information from which at least two things can occur. First, bridges can be constructed to increase the likelihood of highly desirable implications. Second, barriers can be constructed to decrease the likelihood of something negative occurring. Before explaining the evaluation process used for this experiment, I want to explore the conceptual foundations of the Implications Wheel® in a little more detail. If you scan the stories about trends or key issues, you can quickly find articles and reports that discuss the “implications.” However, these reports often suffer from what would be labeled as “paradigm-limited implications.” First of all, many of these lists or discussions of implications are nothing more than action alternatives, things individuals or organizations are saying they are doing in response to the issue or trend. Secondly, even though these “implications” can be seen as possibilities, they are almost always “first- order” implications – they are confined by the limitation of “what will happen next?” They fail to identify what happens after that, or what happens after that? And finally, because they are limited to alternatives, they tend to be “positive” implications and they lack the detail of scoring that adds significant value in the Implications Wheel® process. Some examples will make it easier to discuss some of the concepts that underlie the process. CONCEPTS, LAWS, AND POLICIES In his 2000 book, “That’s Not What We Meant To Do,” Steven Gillon examined “Reform and Its Unintended Consequences in Twentieth Century America.” Of particular concern to human resource management are chapters discussing Affirmative Action and the Civil Rights Act of 1964, both of which had consequences not even close to what the original lawmakers intended. In a November, 2003 Training Magazine article, a case involving performance standards was discussed. A telecommunications company imposed a “time limit,” monitored clearly so that call center employees could self- monitor their handling of customer call times. Over a period of months, management consistently lowered the time standard. When employees realized they could not meet the lowered standards, they simply hung up on customers – enough to lower the “average” to the standard. In one Nashville center, 1500 customers were intentionally disconnected in one day. In an April 8, 2004, Reuters’ story, the Union Pacific Railroad revealed serious crew shortages required diversion of rail shipments and extensive delays. After a government change in the retirement age from 62 to 60, the results of a survey of employee intentions “didn’t reflect what ended up happening.” Freight trains have been coming to a stop with no crews available to take over. As these examples illustrate, it easy to find the interesting, sometimes humorous examples, of “unintended consequences.” But it is very important to note that each of these examples carried significant consequences in terms of cost, service, and reputation to the organizations that failed to consider what might if… Why does this occur? One argument is, of course, paradigms, the desire to see those things we expect to see. Another is time. It takes time to scout the future, it takes time to explore and discover possibilities. And it takes more time to do this than to pursue the path of the familiar. Another component is a fear of the future. The future is full of unpredictable, sometimes nasty, consequences. This fear slows down our decision-making, preventing us from making snap decisions. But that also means this fear may prevent us from seeing new ideas, innovations, and the possible positive outcomes – as well as missing the potentially really dangerous possibilities. 390 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 THE IMPLICATIONS WHEEL® SCOUTING THE FUTURE – MAKING THE JOURNEY The Implications Wheel® process begins by carefully defining the “center” of the Wheel, the issue to be discussed. It may be a trend, an innovation, a policy change, or a new strategic objective. For a quick example of how the Wheel is used in its traditional setting, I will use a Wheel that’s been developed by several groups of HR Managers and managers with hiring responsibilities. This Wheel examines the implications of the demographic changes projected for the workforce over the next 5-10 years. This on-going project has been labeled the Workforce 2008 Implications Wheel® project. Because this is what we call a “Highly Facilitated Implications Wheel®, the process starts with identification of “first order” implications. For this project, we initially identified some major “first order” implications and then started a variety of groups with some of these implications. A sample of these “first orders” is shown in Figure 1. Then, after a two hour training sessions, groups of five participants took a “first order” implication, and identified the possible “second order” and “third order” implications. A sample “arc” from one of the first orders is shown in Figure 2. In this Wheel, the participants identified “second order” implications based on assuming the “first order” happened: “Ability to recruit new employees may become critical.” Then for each of the “second order” implications, “third order” implications were identified. SCORING Scoring an Implications Wheel® allows the wheel to be shared with much more precise information. Scores force teams to be more precise in denoting the significance of implications. Scoring is generated based on two factors: Desirability and Likelihood. Desirability is scored on a +5 (Highly Desirable) to -5 (Highly Undesirable) with precise definitions provided for each of the scores. Likelihood is scored on a 1 (Very Unlikely to Happen) to 9 (Very Likely to Happen) Scale. Figure 3 shows a different arc from the Workforce 2008 Wheel, with scoring indicated. (In Figure 3, the lighter shaded circles are scored highly desirable (+4/+5), the darker shaded circles are scored highly undesirable (-4/-5). A second circle indicates likelihood of 6 or greater. The starred circle indicates it is extremely negative (-50).) The Implications Wheel® software records these scores in color. Figure 1 – First Order Implications 391 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 2 – First Order Implication – With Second and Third Order Implications Figure 3 – An Implications Wheel® arc with scoring. 392 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 The Workforce 2008 Implications Wheel created by the MBA students contains fourteen of these “arcs.” See the notes at the end of the paper for information on how to obtain a complete copy of this Wheel. This information would then be used by HR Professionals to enhance decision-making and avoiding “unintended consequences. THE EVALUATION EXPERIMENT The twelve teams participating in the Human Resources Management Simulation were given two worksheets representing two different “centers” to be explored as they participated in the simulation. The first asked the question: What are the implications of participating in a computer- based simulation on human resources management? This strategic exploration started with the basic objectives of the simulation as possible outcomes, but offered students the opportunity to add other “first-order” implications. The students were then instructed to explore the possible “second-order” and “third-order” implications of these “first-order” implications. The second “center” focused on “What are the implications of running a Human Resources Department?” This strategic exploration asked the students to explore and record the possible implications of their specific decisions. Again, the students were given a “mega-wheel” with some first orders entered and were asked to add to this, then move on to building arcs with “second-“and “third-order” implications. Figures 4 and 5 show the “mega-wheels” with the “first-order” implications that they added to the Wheels. As the student teams participated in the final four decisions of the simulation, they explored the implications of these two questions. The teams scored their Wheels and the information was entered into the Implications Wheel® software. Figure 4 –“Mega-Wheel” for Participating in a Human Resource Management Simulation 393 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 5– Sample “Mega-Wheel” for Managing a Human Resource Department RESULTS Twelve individual teams worked on the two Implications Wheels® for this project. As a result, there was some duplication; some teams worked on the same “first- order” implications as other teams. The information was combined and duplicate implications were eliminated. For each of two different Implications Wheels, twelve different “first-order” implications were kept to the final evaluation. Each of these “first-order” implications had five “second- order” implications – that’s 60 per Wheel or 120 total. Each of the “second-order” implications had five “third-order” implications – that’s 300 per Wheel or 600 total implications. One of the metaphors that Joel Barker uses frequently for the Implications Wheel® is that of a scout heading out to explore before the wagon master rolls the wagons. Without the scouting process, we may be following a map that has the key cities or geographic features on it. With the scouting, we generate the “TripTik®” with all of its details: optional routes, barriers, shortcuts, opportunities. All the results generated cannot be presented here (see note at the end for information on obtaining a complete copy of the results). On the following pages, two arcs are presented from each of the Wheels. In each case, the arc is presented without scoring data, and then on the following pages, the same arc is presented with scoring. This will enable the reader to experience a piece of the process. How would you score some of these implications? Implications are scored for “desirability” using a +5 to -5 scale, a “0”is neutral, it would mean it doesn’t make any difference – positive or negative. Implications are also scored for “likelihood.” Using a scale of 1 – Very Unlikely to Happen to 9 – Very Likely to Happen. Scoring is always completed from a “Point of View.” From a human resource perspective, HR Managers might score implications very differently than Unions would. 394 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 395 Figure 6 – Ethical Decisions Arc – Without Scores Developments in Business Simulations and Experiential Learning, Volume 32, 2005 396 Figure 7 – Ethical Decisions Arc – With Scores Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 8 – Recruiting Talent Arc – Without Scores 397 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 9 – Recruiting Talent Arc – With Scores 398 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 10 – Value of HR Arc – Without Scores 399 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 11 – Value of HR Arc – With Scores 400 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 12 – Value of Team Work Arc – Without Scores 401 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Figure 13 – Value of Team Work Arc – With Scores 402 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 CONCLUSIONS The results of this evaluation experiment parallel the results achieved when the Implications Wheel® is used in its stated environment of exploring a trend, innovation, policy change, or strategic objective. There are of course, limitations to this experiment. It was not intended, nor possible in this setting, to set up an experiment comparing this process to a more traditional evaluation scheme. The intention was simple: to do an initial “exploration” of the use of the tool in this manner. So it will be highly appropriate to challenge some of the assumptions in the following statements. These challenges will likely play a key role in future research on the use of the Implications Wheel® as a learning evaluation tool. There is, in the case of some of the points drawn from this experiment, supporting evidence of the comparisons that are made. Having laid this foundation, here are some concluding observations from this project: 1. The Implications Wheel® evaluation resulted in both a broader and deeper identification of outcomes than a typical end-of-simulation learning inquiry. 2. The Implications Wheel® evaluation produced new lear outcomes as a result of initial outcomes (in Whe rms, second and third order implications) that wou ot be identified in a typical end-of-simulation lear nquiry. 3. The Implications Wheel® evaluation identified impl ons significantly beyond the scope or para rs of the simulation itself. 4. Whi is evaluation format required greater time and effo the part of participants, the process, which depe on principles of speed, balance, and the “wis of crowds,” is an efficient evaluation tool. The e two of these observations that require a brief explanation. Earlier, it was stated that one of the foundations of the Implications Wheel® is that it is different in that it explores beyond “first-order” implications. There is some interesting support research to back up this claim for the experiment conducted here. This same group of MBA students created the Workforce 2008 Wheel described earlier. Before learning anything about the Implications Wheel® process, they were given a timed quiz to identify “What might happen if the workplace demographic trend continues?” They were given 30 minutes to list as many implications as they could. The students generated 312 implications, which included some duplicates. However, 86% of these were “first-order,” 12% were second-order, and only 2% were “third-order.” Rarely did any student use the 30 minutes allowed. Many stopped after identifying less than ten implications. When they examined the same issue using the Implications Wheel®, they identified over 600 implications, the overwhelming majority “second- or third- order.” Joel Barker believes that most of the important implications, and all of the “unintended consequences,” occur in the “second- or third-order.” In addition, there are at least two more applications that appear from this experiment. The Implications Wheel® process allows students to do a comprehensive reflection on their learning. Again, because of the structured nature of the process, this appears to me a more comprehensive evaluation of learning than traditional methods. Secondly, the results of this experiment provide a fascinating roadmap of the instructor. Students clearly identified issues and learning points beyond the specific scope of the simulation. There are a lot of implications that clearly reference text material. More importantly there are many implications that reflect subjects that weren’t discussed during class periods. Because this journey will be taken again with different participants, the scouting report created by this group of explorers will provide an excellent map to help guide the new explorers on a more successful learning journey. That, in conclusion, makes the process a valuable addition to the subject of evaluation. NOTES 1. The Implications Wheel® and the Implications Wheel® software are used with permission of Joel A. Barker and Infinity Limited. 2. Copies of the complete Implications Wheels® HR Evaluation experiment and the Workforce 2008 Wheel can be requested from the author or downloaded from www.hrtracking.com ning el te ld n ning i icati mete le th rt on nds dom re ar REFERENCES Chakravorty, Jui “Update: Union Pacific seeks to divert some rail shipments,” Reuters, April 5, 2003 Dolezalke, Holly “Eye on Ethics,” Training Magazine, November, 2003 Gillon, Steven M. “That’s Not What We Meant to Do,” Reform and Its Unintended Consequences in Twentieth- Century America, W. W. Norton and Company, 2000. Surowiecki, James, The Wisdom of Crowds, Doubleday, 2004 403 Table of Contents Volume 32, 2005 LEARNER BEHAVIOR IN THE ONLINE CLASSROOM EXPERIENCE THE EFFECTIVENESS OF A SIMULATION EXERCISE FOR INTEGRATING PROBLEM-BASED LEARNING IN MANAGEMENT EDUCATION DEMONSTRATION OF FOUR WEB-BASED SIMULATIONS THRESHOLD COMPETITOR: A MANAGEMENT SIMULATION ENTREPRENEUR: A NEW VENTURE SIMULATION MERLIN: A MARKETING SIMULATION MICROMATIX: A STRAGETIC MANAGEMENT SIMULATION LEARNING STYLES INFLUENCES ON SATISFACTION AND PERCEIVED LEARNING: ANALYSIS OF AN ONLINE BUSINESS GAME INTERNATIONAL INTERNSHIPS: DESIGN AND EXPERIENCES SIM MAP: TURNING ACTION-BASED LEARNING INTO SIMULATED CONSULTING PROJECTS NOTEL HEALTH SERVICES: A ROLE-PLAYING SIMULATION TEACHING EXPERIENTIALLY WITH THE MADELINE HUNTER METHOD: AN APPLICATION IN A MARKETING RESEARCH COURSE SIMULATING CUSTOMER LIFETIME VALUE: IMPLICATIONS FOR GAME DESIGN AND STUDENT PERFORMANCE VIRTUAL PROGRESS: SIMULATING ECONOMIC DEVELOPMENT ONLINE STRATEGIC MANAGEMENT: AN EVALUATION OF THE USE OF THREE LEARNING METHODS IN CHINA ADOPTION OF DISCUSSION-BASED TEACHING AND ASSESSMENT IN TEACHING STRATEGIC MANAGEMENT IN CHINA STUDENTS' VIEW ON THE USE OF CASE METHOD IN CHINA CHINESE STUDENTS' PERCEPTIONS OF BUSINESS GAMING EXPERIENCECSR - A CORPORATE SOCIAL RESPONSIBILITY SIMULATION CREATING DYNAMIC INTERACTION IN A VIRTUAL WORLD: ADD VALUE TO ONLINE CLASSROOMS THROUGH LIVE ELEARNING AND COLLABORATION: A DEMONSTRATION CAPABILITIES OF EXPERIMENTAL BUSINESS GAMING A COMPARISON BETWEEN SOLUTIONS AND DECISIONS IN A BUSINESS GAME VALIDATING BUSINESS SIMULATIONS: DOES HIGH PRODUCT QUALITY LEAD TO HIGH PROFITABILITY? ALIGNING ART AND EPISTEMOLOGY: ILLUSTRATIONS TO DISTINGUISH DISCOVERY FROM KNOWLEDGE BUILDING TUTORIALS USING WINK STUDENTS AS LAB RATS: THE ETHICS OF CONDUCTING NON-PEDAGOGICAL RESEARCH IN THE CONTEXT OF CLASSROOM SIMULATIONS AND EXPERIENTIAL LEARNING THE EFFECT ON GAME PERFORMANCE OF DIFFERENT MEASURES AND UNITS OF ANALYSIS IN QUANTITATIVE ANALYSIS ANALYZING AND THINKING WHILE PLAYING A SIMULATION COMPUTER BUSINESS SIMULATION DESIGN: THE ROCK POOL METHOD EXPANDING THE ROLE OF E-ROOMS IN DISTANCE LEARNING APPLICATIONS TO MANAGEMENT EDUCATION APPLICATION OF TRADITIONAL AND ONLINE JOURNALING AS PEDAGOGY AND MEANS FOR ASSESSING LEARNING IN AN ENTREPRENEURIAL SEMINAR DEVELOPING MANAGERIAL EFFECTIVENESS: ASSESSING AND COMPARING THE IMPACT OF DEVELOPMENT PROGRAMMES USING A MANAGEMENT SIMULATION OR A MANAGEMENT GAME INTERNATIONAL MANAGEMENT GAME Œ AN INTEGRATED TOOL FOR TEACHING STRATEGIC MANAGEMENT INTERNATIONALLY STUDENT EXPECTATIONS OF SIMULATIONS DISTANCE EDUCATION DELIVERY OF AN INTENSIVE SIMULATION BASED COURSE TEACHING SERVICE LEARNING USING A BUSINESS GAME ROLE-PLAY SIMULATION EDUCATIONAL PERSPECTIVE OF COLLABORATIVE VIRTUAL COMMUNICATION AND MULTI-USER VIRTUAL ENVIRONMENTS FOR BUSINESS SIMULATIONS SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? VIDEO CASE: JET-A-WAY INC. Œ FOCUSING ON DIVERSITY AND ENTREPRENEURIAL LEADERSHIP ACTIVE LEARNING: WHAT IS IT AND WHY SHOULD I USE IT? FACILITATING THROUGH COLLABORATIVE REFLECTIONS TO ACCOMMODATE DIVERSE LEARNING STYLES FOR LONG-TERM RETENTION ONLINE CUMULATIVE SIMULATION TEAM PERFORMANCE PACKAGE WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY USING EXPERIENTIAL LEARNING TO INTEGRATE THE BUSINESS CURRICULUM FORECASTING STOCK VALUE EMPLOYING PROGRESSIVE PRACTICES AND PRINCIPLES TO FACILITATE SEMINAR ROOM LEADERSHIP AMONG LEARNERS: SHARED POWER AND COLLECTIVE ACCOUNTABILITY INDIVIDUAL ACHIEVEMENT DOES NOT GUARANTEE TEAM PERFORMANCE: AN EVIDENCE OF ORGANIZATIONAL LEARNING WITH BUSINESS GAMES DECISION MAKING IN BUSINESS SIMULTION DESIGN ZUG UM ZUG 2015: COLLECTIVE BARGAINING AS A TWO-LEVEL GAME A NEW METHOD FOR MODELING INNOVATION AND R&D IN BUSINESS SIMULATIONS: ILLUSTRATED WITH A SIMULATION OF A NEW PRODUCT DEVELOPMENT PORTFOLIO DEVELOPING A MICRO SIMULATION EFFECT OF MARKET SHARE AND PRODUCTION EXPERIENCE ON COMPANY PROFITABILITY RE-DESIGNING A CURRICULUM THAT VALUES A WORK-INTEGRATED APPROACH TO STUDENT LEARNING HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP ACADEMICALLY SOUND COURSES THAT ALSO MEET STAKEHOLDER NEEDS EVALUATING SERVICE LEARNING: REFLECTION AND ASSESSMENT FROM THE STUDENT POINT OF VIEW SIMPLIFYING AND ENHANCING FINANCIAL ANALYSIS IN CASES AND SIMULATIONS OVERCOMING THE BUSINESS GAME COMPLEXITY PARADOX EXPLORING THE PREFERENCE IN LEARNING APPROACH AMONG THE HONG KONG UNIVERSITY STUDENTS: CASE STUDY, PROBLEM-BASED OR TRADITIONAL TEXTBOOK QUESTION AN EXERCISE FOR EXPLORING THE RELATIONSHIP BETWEEN JUNGIAN PSYCHOLOGICAL TYPES AND POLITICAL STYLE IN THE WORKPLACE EVALUATING THE DIRECTION OF RESEARCH IN ONLINE EDUCATION: ARE WE GOING ANYWHERE? AIS RAIL SYSTEM: A COMPUTER-BASED JOB-ORDER COST SIMULATION BLOOM BEYOND BLOOM: USING THE REVISED TAXONOMY TO DEVELOP EXPERIENTIAL LEARNING STRATEGIES DELIVERING A TECHNOLOGY-BASED CASE IN A TECHNOLOGICAL WAY: THE SMARTCART CASE USING THE INTERNET TO ENHANCE COURSE PRESENTATION: A HELP OR HINDRANCE TO STUDENT LEARNING TEACHING PRACTICES: A CLUSTER ANALYSIS OF STUDENTS IN HONG KONG TEACHING PRACTICES: A CLUSTER ANALYSIS OF TEACHING STAFF IN HONG KONG EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL A SYMBOLIC MODEL OF THE SIMULTANEOUS ACHIEVEMENT OF CONCRETE BENEFITS AND LEARNING BY PARTICIPATING GROUPS IN EXPERIENTIAL ACTIVITIES: ‚THE SPHERE OF EXPERIENTIAL LEARNING