EVALUATING EXPERIENTIAL TRAINING: CASE STUDY AND RECOMMENDATIONS Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 166 EVALUATING EXPERIENTIAL TRAINING: CASE STUDY AND RECOMMENDATIONS Darryl Scott Albertson, University of Southern California ABSTRACT This paper explores the process of evaluating experiential business training. First, overviews of experiential learning and training evaluation are presented. Research questions from a case study of experiential training evaluation (N = 495) include: 1) How do participants rate key aspects of experiential training?; 2) How do ratings of aspects of experiential training relate statistically?; and 3) Which predictor items are most related to overall evaluations of experiential training? (Can a model of “drivers” of experiential training effectiveness be constructed?) Study methods and outcomes are discussed. Of particular interest, stepwise regression generated a five-item model explaining a significant amount of variance in overall evaluation scores (R2 = .54; p. < .0001). The most Important items in the model of experiential training effectiveness concerned degree of learning enhancement and interest in concepts. Finally, recommendations for evaluating experiential training are made. INTRODUCTION We are currently witnessing a dramatic increase in interest in the issue of how to most effectively develop the world’s managers. According to recent figures (Hoberman & Mailick, 1992), there are about 7.5 million participants in management development programs annually, and about 75% received some form of pre-entry or qualifying training (largely in college or university business programs). However, deliverers of business training may have gone overboard in their attempt to attract the expanding learner base. James F. Bolt, in his 1990 article “How Executives Learn: The Move From Glitz to Guts,” argued that many trainers have created programs that are entertaining, but fail to meet business needs. Bolt encouraged the training community to design programs that are both fun for participants and accomplish specific organizational goals. Experiential business training has recently grown in popularity as a means of accomplishing the goal of providing training that is both interesting and relevant. Hoberman and Mailick (1992) explained that experiential management development is an alternative to more traditional methods, such as case studies, lectures, and discussions. According to the authors, experiential training differs from other kinds of training in that it is specifically designed to motivate active participation, and “leads to transferring learning to the work venue more effectively” (p. 131). Geber (1994) added that experiential learning involves using exercises “in which the elements of one system are used to represent the elements of another system” (p. 10). Typical forms of experiential activities include outdoor, adventure-based activities, and other business simulations and games. The process of developing experiential training involves the same basic steps as other kinds of organizational training: needs assessment, program design and delivery, and program evaluation/modification. While there has been a great deal written on the subject of how to evaluate training programs, in general, very little has been written on the subject of how, specifically, to evaluate experiential training. The purpose of this paper, therefore, is to explore the process of evaluating experiential business training. First, overviews of experiential learning and training evaluation are presented. Research questions related to a case study of experiential training evaluation are posed. Next, details of study methods and outcomes are discussed. Finally, recommendations for evaluating experiential training are made. It is hoped that this paper will contribute to a better understanding of the process of evaluating experiential training, as well as to the betterment of management development efforts. EXPERIENTIAL LEARNING AND TRAINING EVALUATION: AN OVERVIEW This section provides an overview of theoretical and practical issues related to experiential business training and the subject of training evaluation. Despite scholarly and business interest in the subjects of experiential learning and training evaluation separately in recent years, little has been done to integrate what is known about the two topics. In particular, the question “How does one best evaluate the impact of experiential training?,” remains largely unanswered. This section concludes by offering research questions designed to further explore the process of evaluating experiential business training. Experiential Learning Much of what has been written about experiential learning in recent years gives credit to the learning theories of KoIb (1984). KoIb drew upon the ideas of Dewey, Lewin, Piaget, and others, to create a four-step model of how humans learn through life experiences. First, we take part in “concrete experiences,” which may be synthetic (experiences that are ‘designed”) or natural (experiences occurring naturally in daily life). Second, we engage in “reflective observation” in which we reflect upon our experiences from a variety of perspectives. Third, we engage in “abstract conceptualization,” in which we attempt to integrate our observations into sound theories. Finally, we enter the “active experimentation” phase, in which we begin to use our new theories to solve problems. Repeated movement through the four steps was seen by KoIb as explaining how people learn and change what they view as “the truth” over time. In a more recent work, Boud, Cohen, and Walker (1993) summarized what is known about the experiential learning process by offering five propositions about learning from Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 167 experience. Proposition 1: Experience is the foundation of, and the stimulus for; learning. This proposition holds that experience and learning cannot be separated, and raises the possibility that experiences may be “created” for the purpose of teaching. Proposition 2: Learners actively construct their experience. This proposition holds that each person attaches their own meaning to events and that experience should be viewed as an intersection between the learner and their outside environment. Proposition 3: Learning is a holistic process. This proposition holds that perception of a given experience is influenced by all cognitive, affective, and cognitive aspects of prior learning experiences. Proposition 4: Learning is socially and culturally constructed. This proposition holds that learners construct meaning in particular social and cultural settings. Proposition 5: Learning is influenced by the socioemotional context in which it occurs. This proposition holds that emotions and feelings affect the learning process. Together, Kolb’s experiential learning model and the propositions offered by Boud, et al., reveal some of the most important theoretical assumptions regarding the experiential learning process. Given the preceding brief overview of theory related to experiential learning, a reasonable question becomes, “How can experiential methods be used to help develop today’s managers?” Experiential management development involves placing participants in an “experiential activity” of one form or another. Geber (1994) noted that some experiential activities are “very simple board games that take less than an hour to play,” while others are “complicated exercises that cast participants in the roles of team members in a make-believe company” (p. 10). Literally thousands of different experiential business activities have been developed to teach participants about concepts such as organizational communication, quality management principles, teamwork, and group problem solving. Of equal importance to the activity itself is the facilitated debrief, or discussion that follows the activity. According to Cudworth (1994), the debrief is of vital importance, because it is here where the facilitator works to ensure the participants learn something of benefit to their organizations, rather than simply have another “fun experience.” Pfeiffer and Jones (1980) have been credited with developing a “classic” model of the process of debriefing experiential exercises. According to the authors, facilitators should encourage participants to: 1) “publish” what they experienced (share observations): 2) “process” what they learned (discuss patterns/dynamics); 3) “generalize” from their learning (infer “real world” principles); and 4) “apply" what they learned (plan future behavior). It is no coincidence that the steps in Pfeiffer and Jones’ “Experiential Learning Cycle” correspond directly to the steps in Kolb’s model of experiential learning. Prior to moving to the topic of training evaluation, it is useful to consider some of the benefits and disadvantages associated with experiential training. According to Hoberman and Mailick (1992), Geber (1994), and others, benefits of experiential training include: 1) improved transfer of learning to the work venue; 2) well-suited for teaching participants how to respond to change; 3) relatively risk-free environments in which to try new behaviors; 4) higher participant involvement and motivation; 5) ability to manipulate several variables at once; and 6) potential for immediate feedback. Additionally, researchers have identified several benefits that are unique to experiential business training, including: improved ability to teach teamwork and problem solving (Hemmasi & Graf, 1991), unique contribution to the managerial skill set (Teach & Govahi, 1993), and positive relationship between business game experiences and outcomes such as income and organizational position (Wolfe & Roberts, 1993). Experiential training, however, is not without its potential drawbacks. According to Hoberman and Mailick (1992), Geber (1994), and others, potential problems associated with the experiential approach include: 1) synthetic experiences are different from the real world and, therefore, participants may act differently in them; 2) simulations may lack the realism necessary to motivate participants; 3) debriefing may be poorly conducted; and 4) participants may make hasty generalizations based on a single experience. The challenge for providers of experiential training, therefore, is to provide experiences that attain the benefits associated with this method, while avoiding or minimizing the pitfalls. Proper training evaluation, the subject of the following section, is an important tool for determining the sort of training that is most appropriate. Training Evaluation As noted in the introduction to this paper, the training process typically has three main steps: needs assessment, design/delivery, and evaluation. The final step in the process, evaluation, was defined by Goodstein and Goodstein (1991) as a “systematic activity undertaken to determine the merit of something—in this case, the value of the training process” (p. 267). Hoberman and Mailick (1992) added that training evaluation is undertaken to determine the degree of change that takes place, whether change is due to the training or some other factor, and/or the durability of the training effects. Given the seemingly obvious benefits of conducting effective training evaluations, it is puzzling why, until recently, evaluation was not generally considered to be an important part of the training process. May, Moore, and Zammit (1987) noted that in the past, training evaluation was ‘frequently left behind as a ‘nice to have’ addition, if practical” (p. xi), due to time and budget constraints. Feelings about the training evaluation process, however, have started to change. Due to the large-scale entry of universities into the training delivery market, and the estimated $40 billion per year that corporations spend to train their employees (Asgar, 1990), organizations are demanding more relevant training, along with a more effective means of measuring training results. Organizations conduct training evaluations to measure several different outcomes, use a variety of research methods, and follow several steps when implementing a training evaluation process. Boverie, Mulchahy, and Zondlo (1994) identified several outcomes organizations attempt to measure in training evaluations, including participant Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 168 reactions (the subject of the present study), learning, behavioral change, and bottom-line results. According to Phillips (1983), organizations employ several methods in order to measure these outcomes, including questionnaires, tests, interviews, observations, and performance records. Finally, Ban and Faerman (1990) identified several steps organizations may use when implementing an evaluation process, including research design selection, design implementation, demonstrating relationship between training and performance, and analyzing factors that affect the impact of training. The type of outcome measured in the present case study was participant reactions. Reactions were assessed using evaluation questionnaires, which have been labeled by Jones (1990), and others as “smile sheets,” since they primarily measure participant feelings, as opposed to learning. Jones (1990), Boverie, et al. (1994), and others, have identified several advantages, as well as disadvantages, associated with the uses of reaction questionnaires. Advantages of using reaction questionnaires include: they are a good measure of participant feelings about the training, they are easy to design and administer, training sponsors and staff typically like to have instant feedback, and the data obtained look valid. Problems with the use of reaction questionnaires include: ratings may not correlate with training application, data are sensitive to participant mood, ratings are sensitive to item wording nuances, surveys are often completed quickly/without a great deal of thought, and the surveys are often given once only, and therefore, can’t assess training concept retention. (Consult sources such as Goodstein & Goodstein, 1991; and Medsker & Roberts, 1992, for sample evaluation forms. Note: Neither of these sources provide sample surveys specifically tailored to experiential training.) Research Questions While there have been a few articles written about how to evaluate experiential training (e.g., Dev, 1990; McEvoy & Buller, 1990), what is known about this process is very far from complete. The following research questions have been developed as a means of further exploring the experiential training evaluation process. These questions have been answered using data gathered in the Experiential Learning Center (ELC), in the School of Business Administration, University of Southern California. Research Question I (RQ1): How do participants rate key aspects of experiential training? (What are the relevant descriptive statistics?) Research Question 2 (RQ2): How do ratings of aspects of experiential training relate statistically? (How do scores on items designed to predict the overall training evaluation correlate? /Do the predictor items group to form factors?) Research Question 3 (RQ3): Which predictor items are most related to overall evaluations of experiential training? (Can a model of “drivers” of experiential training effectiveness be constructed?) Question I was designed to assess strengths and weaknesses of ELC training, as well as to identify patterns in the data (such as skewness). Question 2 was designed to test how individual items relate to each other. Question 3 was designed to answer the question identified by Ban and Faerman (1990): ‘Which factors most affect the impact of training?”. METHOD This section provides an overview of the method used in the present study. Issues related to subjects, instrumentation/ procedure, and data analysis are discussed below. Subjects The subjects for the study consisted of 495 undergraduate business students enrolled in Business Communication or Organizational Behavior classes at University of Southern California during the Fall 1993 and Spring 1994 semesters. Students each participated in several experiential activities in the Experiential Learning Center (ELC), a facility specifically designed to provide experiential training. Special features of the facility include two-way mirrors, that allow trainers to observe participants without being noticed, and remote video cameras, which allow participants to be taped for subsequent video feedback. The activities the students participated in included: “Hi Fli Fireworks” (full enterprise simulation), ‘Who Gets Hired?” (hiring/persuasion activity), ‘Patterns of Organizational Behavior’ (organizational communication activity), “Bafa Bafa” (intercultural simulation), “Ethics” (decision-making activity), “Subarctic Survival” (decision-making activity), “Compensation Decision” (motivation activity), and “Meta-4” (organizational design activity). Three different professional facilitators conducted the activities. Instrumentation/Procedure Participants completed a ten-item questionnaire that asked them to rate various aspects of their ELC learning experience. (See Table I for item text.) Questionnaires were distributed in the students’ regular classrooms. Students were directed to consider all ELC activities they participated in during the semester while completing evaluation forms. It should be noted that the ELC also employs two other questionnaire types in their evaluation process, one designed to gather information about facilitators, and the other to gather information about specific activities. These additional questionnaires are not the focus of the present study. Analyses All analyses were completed using SPSS/PC+, version 4.0. To answer RQ1 (How do participants rate key aspects of experiential training?), descriptive statistics, including means and standard deviations, were computed. A visual inspection of the data, as well as statistical tests for skewness, indicated that the data for each of the items were severely negatively skewed. The positive implication associated with the skewness was that most students rated Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 169 aspects of the ELC highly: either 4 or 5 on a 5-point Likert scale. The negative implication associated with the skewness was that the data, without some form of transformation, would not be acceptable for subsequent parametric statistical tests, due to violations of the assumption of normality. The researcher first tried the “reflect and logarithm” data transformation, recommended by Tabachnick and Fidell (1989). While the transformation reduced the degree of skewness for all variables, skewness remained excessive. The transformation procedure that was ultimately used was to re-code the data into three groups: Ratings of 1-3 were re-coded as a I, ratings of 4 were re-coded as a 2, and ratings of a 5 were re-coded as a 3. This transformation method was selected for two reasons. First, a visual examination of the data indicated that about half of the ratings were a 4, about a quarter of the ratings were a 5, and about a quarter of the ratings were 1-3. As such, the transformation was successful in reducing the skewness to an acceptable level, permitting the use of subsequent parametric tests. Second, the transformation was consistent with how the management of the ELC viewed the data. In particular, ELC management has been satisfied with ratings of 4 or 5 and concerned with ratings of 3 or below. More will be said about the impact and advisability of this kind of data transformation in the Results and Discussion section. To answer RQ2 (How do ratings of aspects of experiential training relate statistically?), two statistical tests were used. First, correlations between predictor items were computed, and second, a factor analysis was attempted. Finally, to answer RQ3 (Can a model of “drivers” of experiential training effectiveness be constructed?), predictor items were regressed on the direct variable (DV) (overall Center evaluation). In order to examine which factors accounted for more of the variance, a stepwise multiple regression was performed. RESULTS AND DISCUSSION Measures were assumed to be sufficiently reliable and valid. A reliability coefficient could not be computed because the DV (overall Center evaluation) consisted of a single item. However, it is believed that clear instructions and response format contributed to reliability. The ELC may want to add additional items related to overall Center evaluation (to compute scale reliability) and/or ask particular students to re-complete the questionnaire later (to compute test-retest reliability). High correspondence between item content and assumptions associated with experiential learning theory contributed to content validity. The ELC may also want to examine the relationship between scores on their evaluation instrument and scores on other, related instruments, and/or compare evaluation scores to performance measures, such as grades, in order to assess predictive validity. ss RQ1 asked, “How do participants rate key aspects of experiential training?” Descriptive statistics (N = 495) indicated that participants were quite satisfied, overall, with their experience in the ELC (4.0 overall evaluation mean). Subjects were most satisfied with the degree to which they were able to share their activity experiences and observations (4.1) and the positive atmosphere in the center (4.1). Items rated lower by participants concerned concept understanding (3.8), business application (3.8), and learning enhancement (3.8). It should be noted that application and learning are areas of typical concern regarding experiential exercises. See Table I for more on items, means and standard deviations. More should be said about the impact of the data transformation procedure that was used in this study. By compressing ratings on a five-point scale into three groups, the unacceptable degree of data skewness was eliminated, thereby allowing for the use of standard parametric tests. Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 170 However, such a transformation is not without costs. The effect of re-coding responses 1-3 into a single group was to move from an interval scale to an ordinal scale. The disadvantage of using an ordinal scale in this case, versus an interval scale, is that a great deal of information about the participants’ degree of concern in a given area was lost in the transformation. Note, however, that Pedhazur and Schmelkin (1991) concluded that ordinal scales can normally be used successfully in the same statistical tests as interval scales. RQ2 asked, “How do ratings of aspects of experiential training relate statistically?” Correlations between predictor items were all significant at the .001 level. (As such, the correlation matrix is not included here.) Degree to which participants felt comfortable expressing opinions was least related to other predictors and connection between exercise/discussion and enhancement of learning were most related to other predictors. A factor analysis was attempted, however, only one factor was able to be extracted. One might ask, ‘Why would all predictor items be so highly correlated?’, and ‘Why would SPSS be unable to extract more than one factor?” One possible (and preferred) explanation might be that the instrument was so we/l designed that the items all related very highly to each other and grouped to form one factor (overall evaluation of experiential training). An examination of the data, however, reveals a more likely cause. Most participants tended to provide similar ratings for all ten-survey items. Pedhazur and Schmelkin (1991) identified this response pattern as an “acquiescence” response style, in which subjects provide high ratings on all items, regardless of content. Jones (1990) noted that this response pattern is typical in training evaluations because subjects have a desire to hurry through the items. (See Recommendations section for ideas on combating acquiescence.) Other possible causes of the failure to extract more than one factor include: 1) there were a very limited number of items (10); and 2) the N should have been higher. (A common rule of thumb is that there should be 100 subjects per item to perform a factor analysis.) Finally, in response to RQ3 (Can a model of “drivers” of experiential training be constructed?), the answer was “Yes.” Stepwise regression generated a five-item model explaining a significant amount of variance in overall evaluation scores (N = 446; R2 = .54; F(5,440) = 103.40; p. <.0001). The five items in the model of experiential training effectiveness concerned: learning enhancement, interest in concepts, connection between exercises/discussion, topic relevance, and comfort in expressing opinions. Other researchers are encouraged to further test the nature and validity of this model. Table 2 reveals that items related to learning enhancement and interest in concepts explained noticeably more variance than other items in the equation (meaning practitioners should work especially hard to assure that participants learn something new and that material is presented in an interesting manner). Table 2 also reveals that each of the five items that remained in the equation were significant at the .01 level. (Note: N = 446; R2 = .54; F (5,440) = 103.40; p. < .0001. Regression was computed using transformed data. See Methods section for explanation of data transformation procedure. Items concerning concept understanding, business application, sharing of experiences, and positive atmosphere were not entered into the equation. See Norusis, 1990, pp. B102-B104, for information on variable selection criteria. See Table I for complete item text.) RECOMMENDATIONS FOR EVALUATION OF EXPERIENTIAL TRAINING The present study provides several useful ideas for those engaged in the design, administration, and analysis of experiential training evaluations. Regarding design, evaluators of experiential training are encouraged to: I) Use other methods in combination with reaction questionnaires. (Consult sources such as Arvey, Maxwell, & Salas, 1992; and Sackett & Mullen, 1993, for more regarding design approaches.), 2) Use items, similar to those used in the present study, that are reflective of theoretical assumptions about experiential learning; 3) Carefully consider item wording (this study has led the ELC to reword several “double-barreled” or otherwise ambiguous items); and 4) Consider steps, including instructions to subjects to carefully consider each item separately and/or alternating positive and negative item stems, to combat response styles. (See Gable, 1986, for more about response styles.) Regarding administration, evaluators of experiential training are encouraged to: I) Consider distributing separate evaluations to assess facilitation, activity effectiveness, and overall center/training department effectiveness; 2) Carefully consider verbal instructions given when distributing forms (ask participants to consider each item separately); and 3) Be sure to distribute several of each type of evaluation form, to iron out differences in subject/class moods, Finally, regarding analysis, evaluators Developments In Business Simulation & Experiential Exercises, Volume 22, 1995 171 of experiential training are encouraged to: I) Consider multiple statistical methods, including those used in the present study; 2) Be prepared to combat skewness in data that result from positive evaluations of training and/or acquiescence; 3) Don’t use the numbers alone (also carefully consider written comments); and 4) Don’t stop with an evaluation study (be sure to take corrective action). CONCLUSION The purpose of this paper was to further explore the process of evaluating experiential business training. First, an overview of theoretical and applied perspectives was presented for both experiential learning and training evaluation. Research questions related to a specific case study of experiential training evaluation were posed. Next, details of study methods and outcomes were presented and discussed. Finally, recommendations for others involved in the process of evaluating experiential training were made. It is hoped that this paper will contribute to a better understanding of the process of evaluating experiential training, as well as to the overall betterment of management development efforts. AUTHOR NOTE The author wishes to thank the staff of the Experiential Learning Center, at the School of Business Administration, University of Southern California, for their assistance in the completion of the present study. REFERENCES Asgar, J. (1990, July). Give me relevance or give me nothing. Training, pp. 49-51 Arvey, R. D., Maxwell, S. E. & Salas, E. (1992) The relative power of training evaluation designs under different cost configurations. Journal of Applied Psychology, 77, 155-160 Ban, C., & Faerman, S. R. (1990) Issues in the evaluation of management training. Public Productivity and Management Review, 13, 271-286 Bolt, J. F. (1990, May). How executives learn: The move from glitz to guts. Training & Development JournaI, pp 83-87. Boud, D., Cohen, R., & Walker, D. (Eds.) (1993). Using experience for learning. Buckingham, England: Society for Research into Higher Education & Open UP. Boverie, P., Mulchahy, D. S., & Zondlo, J. A. (1994) Evaluating the effectiveness of training programs. In J Pfeiffer (Ed.), The 1994 annual: Developing human resources (pp. 279-294). San Diego: Pfeiffer & Company. Cudworth, A. L. (1994, March). Encouraging participant responses in experiential situations to develop key learning concepts. Paper presented at the annual meeting of the Association of Business Simulation and Experiential Learning, San Diego, CA. Dev, C. S (1990) Measuring the value of experiential learning. Cornell Hotel & Restaurant Administration Quarterly, 31, 105- 107 Gable, R. K. (1986) Instrument development in the affective domain Boston: Kluwer-Nijhoff. Geber, B. (1994, April, Suppl.). Let the games begin. Training, pp. 10-15 Goodstein, J., & Goodstein, L. D (1991) A matrix for evaluating training In J. W. Pfeiffer (Ed.), The 1991 annual: Developing human resources (pp. 267-285). San Diego: University Associates. Hemmasi, M., & Graf, L. A. (1991) Educational perspectives of business simulation gaming: A comparative study of student and practitioner perspectives. Developments in Business Simulation & Experiential Exercises, 18, 53-56 Hoberman, S., & Mailick, 5. (1992). Experiential management development. New York: Quorum. Jones, J. E (1990, December). Don’t smile about smile sheets. Training and Development Journal, pp. 19-21 KoIb, D. (1984) Experiential learning: Experience as a source of learning and development. Englewood Cliffs, NJ: Prentice Hall. May, L. S., Moore, C. A., & Zammit, S. J. (Eds.). (1987). Evaluating business and industry training. Boston: Kluwer. Medsker, K. L., & Roberts, D. G. (1992) ASTD trainer’s toolkit Alexandria, VA: American Society for Training and Development. McEvoy, G. M., & Buller, P. F. (1990) Potholes along the road to evaluating learning outcomes: The case of outdoor management training. In J. Wingender & W. Wheatly (Eds.), Developments in Business Simulation & Experiential Exercises, 1 7, 202. Stillwater, OK: Association for Business Simulation and Experiential Learning. Norusis, M. J. (1990) SPSSJPC+ statistics 4.0 for the IBM PC/XT/AT and PS/2 Chicago: SPSS Inc. Pedhazur, E. J., & Schmelkin, L. P. (1991) Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum. Pfeiffer, J. W. & Jones, J. E. (Eds.) (1980). The 1980 annual handbook for group facilitators. San Diego: University Associates. Phillips, J. J. (1983) Handbook of training evaluation and measurement methods Houston: Gulf. Sackett, P. R., & Mullen, E. J. (1993) Beyond formal experimental design: Towards an expanded view of the training evaluation process. Personnel Psychology, 46, 613-627. Tabachnick, B. G., & Fidell, L. 5 (1989). Using multivariate statistics (2nd ed). New York: Harper-Collins. Teach, R. D., & Govahi, G. (1993). The role of classroom techniques in teaching management skills Simulation & Gaming, 24, 429-445 Wolfe, J., & Roberts, C. R. (1993) A further study of the external validity of business games: Five-year peer group indicators. Simulation & Gaming, 24, 21-33 Table of Contents Volume 22, 1995 Simulation Performance, Learning and Struggle Are Good Simulation Performers Consistently Good? Cognitive and Behavioral Consistency in a Computer-Based Marketing-Simulation-Game Environment: An Empirical Investigation of the Decision-Making Process Chalk & Cheese: Executive Short-Course vs. Academic Simulations Revisiting Personality Bias in Total Enterprise Simulations Are Good Strategies Consistently Good? Investigating the Use of a Computer Simulation as an Effective Pedagogical Tool for the Application of a Strategic Model The Problem of determining an Individualized Simulation's Validity as an Assessment Tool A Simulation Based Analysis of the Value of Information in the Hrebiniak Joyce Typology of Adaptation Relative to Porter's Generic Strategies The Impact of Sales and Income Growth on Profitability and Market Measures in Actual and Simulated Industries A Comparison of a Stand Alone Version of a Simulation with the Traditional Competitive Version Computer-Assisted Gaming of International Business Analyzing Simulations with Computer-Based Programs and Applying the Experience to a Real-World Business A Preliminary Investigation of the Use of a Bankruptcy Indicator in a Simulation Environment Graduates' Views on the Use of Computer Simulation Games Versus Cases as Pedagogical Tools An Analytical Advertising Model Approach to the Determination of Market Demand Dealing with the Complexity Paradox in Business Simulation Games A Prototyping Approach for Incorporating Large Data Bases into Media Planning Simulations: An Example Using Magazine Media Through the Looking Glass, Inc: Superior-Subordinate Personality Type and the Leniency effect Evaluating the Effectiveness of Role Playing Simulation and Other Methods in Teaching Managerial Skills Student and Teacher Perceptions of a Management Simulation Course Performance Evaluation: The Effect on the Propensity to Create Budgetary Slack Management Team Formation for Large Scale Simulations Comparative Static Analysis with the Complete PPA Package: A Strategic Market Planning Tool Consistency in Intent: Learning Objectives at the 1994 Intercollegiate Business Policy Competition A New Twist on an Old Game: The Business Strategy Game: A Global Industry Simulation 3ed Evaluation of Performance in Management Simulation: A Management Coefficients Model Building SimuWorlds: Strategic Management Games of the Future Bulls and Bears: A Stock Market Simulation A Systems Thinking Paradigm and Think Computer Simulation Model of Broadcast and Cable Television Industry Competition Jacket Factory The Sales Management Simulation The Marketing Management Simulation A Demonstration of Promodel Demonstrating A New, Cross-Functional Business Simulation: Vision+ A Cost Chain For The Business Strategy Game Simulation Special Session On Experiential Teaching Compensation Dilemmas: An Exercise In Ethical Decision-Making Organizational Storytelling: Telling Tales In The Business Classroom Evaluating Experiential Training: Case Study And Recommendations The Internship Portfolio: An Innovative Tool For Experiential Learning, Critical Thinking, And Communication An Ethnographic Analysis Of The Pedagogical Impact Of Cooperative Communicating Consumer Behavior: A Long-Term Integrated Exercise Using Personal Consumption Journals And Consumer Analysis Papers An Experiential Paradigm For Teaching Business Problem Solving Developing Leadership Skills The Spss® Student Assistant: The Integration of A Statistical Analysis Program Into A Marketing Research Textbook Negotiating With Your Students Using TQM Principles To Transform Accounting Systems Into An Experiential Exercise Enhancing The Effectiveness Of Outdoor-Based Experiential Training Using Virtual Reality Concepts Case Writing In A Developing Country: An Indonesian Example Experiential Learning Using Focus Groups How Real Should Experiential Pedagogy Be? A Viewpoint From Our Students Reengineering The Internship: A New Approach To Experiential Learning Utilizing The Cosmopolitan/Local And Marginal Man Constructs To Measure Students' Propensity For Creativity Developing Experiential Processes For Teaching Quantitative Techniques For Business Team Learning Roles: A Cooperative Learning Technique Creating the Ultimate Small Business Student Experience: Melding Score/Ace with SBI The Development of Trust in Work Teams: The Impact of Touch Some Outcomes of Experiential Learning: How the Cultural Dynamics of Different Countries are reflected in Workplace Norms & Values Incorporation of Job Analysis Results in Various Forms of Selection Interviews Chudesno, Inc.: An Evaluation of an Experiential Training and Development Simulation An Exercise for Exploring the Relationship between Jungian Psychological Types and Organizational Dynamics Partnership: A Radical Approach to Experiential Learning Partnership: A Nice Idea, But How Do I Get Started? Recognizing Discrimination at Work Using Critical Incident Skills Questions to Help Students Become More Successful at Job Interviewing The Video Project Introduction to Psychological Type Theory Come On Down Understanding Facilitation for Development and Continuous Learning: A Micro-Workshop Leadership and Empowerment: An Experiential Exercise in Decision Making Experiential Exercises and Pedagogy Track Workshop: Selecting a Manager for Maquiladora, Inc. Experiential Training In Multi-Cultural Corporate Settings The Role of Facilitation as an Aid to Complete Learning An Experiential Exercise to Illustrate Difference in Information Processing Behaviors and Styles How to Deliver Accessible Survey Results Age Diversity in the Workplace- Family Feud Style Nafta Standoff: A Cross-Cultural negotiating Role-Play Three Strikes and You're Out!: A Downsizing Experiential Exercise