SIMULATION PERFORMANCE AND ITS EFFECTIVENESS AS A PBL PROBLEM: A FOLLOW-UP STUDY Developments in Business Simulation and Experiential Learning, Volume 34, 2007 43 SIMULATION PERFORMANCE AND ITS EFFECTIVENESS AS A PBL PROBLEM: A FOLLOW-UP STUDY Philip H. Anderson University of St. Thomas phanderson@stthomas.edu Leigh Lawton University of St. Thomas l9lawton@stthomas.edu ABSTRACT This study continued the exploration of the relationship between the students’ success on a simulation exercise and their perceptions of its value as a PBL problem. This study found no significant relationship between financial performance on a simulation and student perceptions of its ability to achieve the benefits derived from a good PBL problem. Limitations and directions for future research are explored. WHAT IS PROBLEM-BASED LEARNING? Problem-Based Learning (PBL) is a pedagogical process that begins by presenting the learner with an engaging problem, question, or puzzle. Learners then discover course concepts for themselves as they explore the problem. PBL grew out of findings of the cognitive sciences regarding how we learn. It originated in medical education and has gone on to gain acceptance as an effective pedagogy in such diverse disciplines as physiology, food production, and geology (Allen and Duch, 1998; Bereiter and Scardamalia, 2000; Duch, Gron, and Allen, 2001; Lieux and Luoto, 2000; Mierson, 2001). PBL is founded on the idea that problems should precede answers. It is designed to give students the opportunity to identify the ideas and skills they need to work through problems. This process helps students recognize their knowledge deficiencies about a discipline, motivates them to understand course concepts, and facilitates their application of those concepts to real problems (Miller, 2004; Brownwell and Jameson, 2004). Spence (2001) argues that PBL provides students with opportunities to examine and experiment with what they already know; to discover what they need to learn; to develop the people skills they need for improving their performance in a team setting; to improve their writing and speaking abilities (to state and defend their own ideas with sound arguments and evidence); and to become more flexible in their approach to problems. This pedagogical process, Spence contends, dramatically improves learning. DEFINING PROBLEM-BASED LEARNING Problem-Based Learning has been defined as “a method of instruction that uses problems as a context for students to acquire problem-solving skills and basic knowledge” (Banta, Black, and Kline, 2000, p1). It also has been described as a “range of educational approaches that give problems a central place in learning activity” (Bereiter and Scardamalia, 2000, p185). Universal to all definitions of PBL is (a) the approach to learning utilized by the instructor and (b) the use of a problem as the central focus of attention in the course (Sherwood, 2004). We will discuss each of these elements in turn. Locus of Learning Barrows (1986) and Spence (2001) make a distinction between subject-based learning (i.e., traditional learning) and problem-based learning. Subject-based learning is teacher-centered; the teacher provides the subject (i.e., student) with the correct answer for various circumstances. The subjects are taught how to use this information as the teacher assigns problems applicable for these “answers”. By contrast, problem-based learning is student-centered; the teacher expects the students to take responsibility for their own learning as they search for answers to the problem assigned. The Importance of the “Problem” Since the problem that learners are asked to solve plays such a critical role in PBL, much attention has been devoted to a discussion of what constitutes a “good” problem. Duch, et al., (2001) argue the quality of the “problem” used in large part determines whether the implementation of the pedagogy is successful. They state that PBL problems need to meet two criteria to be effective for a PBL design. The problems should (1) engage student interest and (2) require the students to develop and implement the principal concepts of the course in order to successfully solve the problem. They contend that establishing a good problem can mailto:phanderson@stthomas.edu mailto:l9lawton@stthomas.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 44 require creativity because the material for good PBL problems is not found in traditional textbooks. Lohman (2002) expands the elements of a good PBL problem, stating that it should have three “structural features”. One, the exact nature of the problem should be unclear and the information needed to solve the problem should be incomplete. Two, there should be more than one way to solve the problem. And three, the problem should not have a single right answer. Barrows (1986), Edens (2000), and Sherwood (2004) contend that the context of the problem must be considered in a PBL designed course, as it provides the circumstances that give meaning to the problem for the students. Sherwood (2004) notes the importance of both organizational and social context in the accomplishment of PBL objectives and offers vignettes, cases and simulations as examples of context problems for use in management education. SIMULATIONS AS PBL PROBLEMS Given their widespread use in business programs (Faria and Nulsen, 1996), simulation exercises could provide instructors with a familiar vehicle for introducing PBL into the business curricula to reap the pedagogy’s benefits. However, before using a simulation exercise as the problem in a PBL designed course, it is useful to consider whether it meets the requirements of a good PBL problem. Our review of the literature found support that simulation exercises meet the three criteria needed to be a good PBL problem identified above. The simulation’s ability to engage students’ interest is supported in reviews of the literature by Wolfe (1985), and later by Washbush and Gosenpud (1991). Further, the linkage between a simulation exercise and the application of course concepts has been demonstrated in multiple studies (Anderson and Lawton, 1997, Green and Faria, 1995; Hemmasi and Graf, 1992, Miller, et al., 1998, Schellenberger, et al., 1989, Teach and Govahi, 1988, Wolfe, 1990). In addition, Anderson and Lawton (2004b) point out that simulations fit the three criteria outlined by Loman (2002) stated above. Finally, Anderson and Lawton (2004a) found support for students’ perceptions of simulations as meeting the characteristics of good PBL problems. PURPOSE OF THE STUDY This study continued the exploration of whether students’ perceptions of a simulation’s effectiveness as a PBL problem are influenced by their financial success on the simulation. Earlier research by Anderson and Lawton (2006) reported mixed results on this question that may have been influenced by the timing of when student opinion was solicited. The current research changed when measurement was undertaken to address this timing issue. It also sought to address questions raised by Anderson and Lawton (2005) on the effectiveness of simulations as PBL problems throughout the duration of a course by assessing student perceptions of a simulation’s effectiveness at the beginning and end of a course. The hypotheses for this study were: H1: There will be a positive correlation between performance on the simulation and the students’ attitudes toward the simulation experience. H2: There will be a positive correlation between performance on the simulation and the students’ perception of how much they know about the discipline of management. H3: There will be a positive correlation between performance on the simulation and the students’ perception of how well the simulation reflects the discipline of management. H4: There will be a positive correlation between performance on the simulation and the students’ perception of how much they learned from the simulation experience. RESEARCH METHODOLOGY The Subjects of the Study Subjects for the study were seniors at a medium-sized, university located in the Midwest. All the students were traditional, college-aged students enrolled in a senior-level strategic management capstone course. The course is required of all business management majors. A total of 39 students participated in the study. After exclusion for missing data, the final number included in the analysis was 25. The Simulation The simulation used was Threshold Competitor (Anderson, et al., 2003). Threshold Competitor is a moderately complex total enterprise simulation requiring students to make approximately 40 decisions involving elements of the marketing mix (e.g., price, quality, promotion), operations (e.g., hire and fire workers, order raw materials, set production levels), and finance (manage cash flow, borrow long-term funds) for each period of play. Each decision period represented three-months (i.e., one quarter). Threshold Competitor has a Team version (in which student-managed companies compete against other student- managed companies) and a Solo version (in which one student-managed company competes against 15 computer- managed companies, not other student-managed companies). The Solo version allows students to process their decisions and move to the next quarter of operation at their own pace, without need for instructor involvement. There are two versions of the Solo program; Solo Practice and Solo Exam. The Solo Practice version allows students to restart the simulation as often as they wish. That is, if students are not satisfied with their performance, they can quit that particular simulation run and initiate a new round of competition from the beginning (Quarter 1). This allows Developments in Business Simulation and Experiential Learning, Volume 34, 2007 45 them to restart the simulation repeatedly until they achieve results they find acceptable. The Solo Exam version is identical to the Solo Practice version with the exception that the students are allowed only one pass through the quarters. They cannot go back and reprocess new decisions for Quarter 1 once they have processed that quarter. Assessment Measures Five measures were used in this study. They were (1) performance on the simulation, (2) student attitudes toward the simulation, (3) students’ perceived knowledge about the discipline of management, (4) students’ perception of how well the simulation reflected the discipline of management, and (5) how much the students thought they learned from participating in the simulation exercise. The first three measures were scales consisting of multiple items; the last two were single item questions. Table 1 shows that for the multiple item scales; none had particularly high levels of internal consistency as measured by Cronbach’s alpha. The low alpha scores were disappointing and surprising since they were inconsistent with the high alphas found in previous work (Anderson and Lawton, 2006). Consequently, we analyzed the data both on how the “collective” measures related to financial performance, and also on how each of the items related to financial success. Research Design In order to assess the relationship between the students’ performance on the simulation and their evaluation of various aspects of the simulation, we collected data on three separate occasions: (1) after the students completed their individual Solo Practice experience in Week 3 of the semester; (2) after the students completed their experience (in groups) with the Team version; (3) at the end of the semester after the students completed their individual Solo Exam experience in Week 13 of the semester. Table 1 Study Scales Scale Name Description of Items Number of Items Cronbach’s alpha Performance Three measures of performance on the simulation (all three of the following were standardized to make their magnitudes comparable): 1) Sales 2) Net income 3) “Points” awarded by the simulation to reflect the relative performance of each company 3 .770, .938, .827 Attitude Six semantic differential scales. The simulation was… unpleasant 1 2 3 4 5 6 7 enjoyable frustrating 1 2 3 4 5 6 7 satisfying dreadful 1 2 3 4 5 6 7 engaging simplistic 1 2 3 4 5 6 7 challenging dull 1 2 3 4 5 6 7 stimulating overwhelming 1 2 3 4 5 6 7 manageable 6 .753, .553, .651 Perceived knowledge How knowledgeable do you feel you are about the discipline of management? (4 point scale from “Not at all knowledgeable” to “Very knowledgeable”) How much do you feel you would have to learn about management before you would be able to perform competently in your first job in a management position? (Reverse scored 4 point scale from “Nothing” to “An extreme amount”) How much do you feel you would have to learn about management before you would be able to perform competently as a manager? (Reverse scored 4 point scale from “Nothing” to “An extreme amount”) If you landed a job as a mid-level manager of an area business, how capable would you be of handling the job? (4 point scale from “Not at all capable” to “Very capable”). 4 .519, .673, .606 Reflected discipline How well do you think Threshold Competitor reflects the discipline of management? 1 na Learning How much do you think you learned from participating in the Threshold Competitor simulation? 1 na * Note: there are three values of Cronbach’s alpha for each of the scales shown above, because the questionnaire was administered on three separate occasions. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 46 Assessment #1. The early introduction of a problem for students to solve is at the core of the PBL pedagogical model. To accomplish this, we introduced students to the simulation in the third class meeting of the course. The first class meeting dealt only with class organizational issues (e.g., course requirements, testing, formation of groups). The second class meeting was limited to a general overview of course topics and concepts and a brief introduction to the simulation. At the next class, the students were given the assignment to use the Solo version of the simulation to run their company as an individual player for four decision rounds. This meant that the students operated their companies prior to a discussion of a framework for decision-making and prior to instruction on how course concepts applied to the simulation exercise. Following their completion of the Solo Practice exercise, the students completed a questionnaire on their perception of the Solo exercise, providing feedback on the exercise’s merits as a PBL problem. Assessment #2. Over the next 10 weeks of the course, the students participated in the Team version of Threshold. This exercise consisted of 12 decision rounds. Following completion of this Team exercise, the students again completed a questionnaire on their perception of the simulation. Assessment #3. Following the completion of the Team exercise, the students again were given a second Solo assignment. They had to run their company as an individual player for eight decision rounds during a 3½ hour exam period. Following their completion of this Solo exercise, the students again completed a questionnaire on their perception of this Solo exercise. The timing of Assessments 2 and 3 in this study differed from those administered by Anderson and Lawton (2006) in their study of the influence of students’ financial success on their assessment of the simulation’s value as a PBL problem. In that study, Assessment #2 was taken after completion of the Team exercise and a second Solo exercise. Assessment #3 was done at the final class session; students were instructed to respond based on their total experience with the simulation over the duration of the term. They were explicitly directed not to fill out the questionnaire based solely on their recently completed Solo Exam experience. This assessment pattern raised questions regarding the students’ ability to separate their Team performance from their Solo Exam performance, when completing the questionnaire. The current study sought to avoid this possible confusion while still assessing the students’ perceptions over the duration of the semester. RESULTS Item Averages Over Time Table 2 shows the averages and standard deviations for each of the assessment measures. Consistent with earlier studies by Anderson and Lawton (2005, 2006), students’ ratings of the simulation on how it met the criteria of a good PBL problem were quite favorable. The attitude measures ranged from 4.44 to 6.32 on a seven-point scale. The average rating for Assessment #1 was 5.38, for Assessment #2 was 5.39, and for Assessment #3 was 5.51. Contrary to prior studies, students’ ratings for the attitude measures tended to improve over the duration of the course. While most changes did not reach the .05 level of significance, the ratings at Assessment #3 were higher than at Assessment #1. Only “Challenging” showed a decline – by the end of the course, students found the simulation to be less challenging. As one would hope, students’ perception of their knowledge was higher at the end of the course than at the beginning. While “amount to learn to perform competently in a first management job” fell just short of statistical significance, all changes were in the expected direction. There was little evidence that students’ opinion of the realism of the simulation changed over the course of the semester. There also was little evidence of a change in the students’ perception of how much they learned from the simulation. Hypothesis Results While we found support that the students’ credited the simulation with the characteristics of a good PBL problem, a primary concern of this study was to assess whether these assessments were related to the financial success they achieved during the exercise. One would hope that students would be able to separate their results from their evaluation of the simulation, but anecdotal evidence suggests otherwise. Table 3 shows the results of a test of the hypotheses stated above, constructed to assess the relationship between student perceptions and their financial success. Hypothesis 1. There was no support for the hypothesis that there is a positive correlation between performance on the simulation and students’ attitudes toward the simulation. While all three correlation coefficients were positive, they were very small and none was statistically significant. Hypothesis 2. No support was found for an association between performance on the simulation and the students’ perception of how much they know about the discipline of management for any of the assessment periods. In fact, only one of the three correlation coefficients was in the expected direction. Hypothesis 3. No support was found for a relationship between performance on the simulation and the respondents’ perception of how well the simulation reflected the discipline. Hypothesis 4. Finally, there no support for the hypothesis positing a relationship between performance and the students’ perception of how much they learned from the simulation. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 47 Table 2 Averages and Standard Deviations for Assessment Measures Assessment#1 Ave / St Dev Assessment#2 Ave / St Dev Assessment#3 Ave / St Dev Attitude Measures (7 Pt Scale) †† 5.38 / 0.703 5.39 / 0.673 5.51 / 0.632 Enjoyable 5.32 / 0.900 5.52 / 1.085 5.80 / 1.000 * Satisfying 4.56 / 1.294 4.44 / 1.446 4.80 / 1.225 Engaging 5.72 / 0.792 5.64 / 0.907 5.92 / 0.862 Challenging 6.32 / 0.627 6.08 / 0.909 5.76 / 1.234 * Stimulating 5.68 / 0.988 5.76 / 1.012 5.68 / 0.802 Manageable 4.68 / 1.464 4.88 / 1.691 5.08 / 1.077 Perceived Knowledge Measures (4 Pt Scale) † 2.39 / 0.382 2.53 / 0.441 2.73 / 0.414 ** Knowledgeable about management 2.56 / 0.583 2.56 / 0.651 2.80 / 0.577 * Amount to learn to perform competently in first management job 2.32 / 0.557 2.48 / 0.653 2.12 / 0.666 Amount to learn to perform competently as a manager 3.04 / 0.611 2.76 / 0.597 2.48 / 0.586 ** Capable of handling job as a mid-level manager 2.36 / 0.638 2.80 / 0.577 2.72 / 0.614 * Simulation reflects management discipline †† 2.80 / 0.645 2.76 / 0.523 2.84 / 0.138 Learned from participating in simulation †† 5.44 / 0.651 5.40 / 0.816 5.56 / 0.142 Change from Assessment #1 to Assessment #3: * p < .05, ** p < .01. † (1-tailed tests); †† (2-tailed tests) Table 3 Hypothesis Test Results Hypotheses “Clustered” Measures Performance 1 R-sq / p-value Performance 2 R-sq / p-value Performance 3 R-sq / p-value H1 Attitude .108 / .606 .096 / .650 .123 / .559 H2 Perceived knowledge -.068 / .747 .233 / .262 -.206 / .323 H3 Simulation reflected discipline -.039 / .853 .073 / .729 .000 / .998 H4 Perceived learning -.260 / .209 .117 / .576 .106 / .614 Table 4 Hypothesis Test Results – Individual Measures Hypotheses Individual Measures Performance 1 R-sq / p-value Performance 2 R-sq / p-value Performance 3 R-sq / p-value H1 Attitude Enjoyable Satisfying Engaging Challenging Stimulating Manageable .068 / .745 .205 / .229 -.204 / .328 .368 / .070 .144 / .492 -.095 / .652 .385 / .057 .388 / .055 -.003/.988 -.581 / .002 ** -.209 / .317 .089 / .674 -.135 / .521 -.212 / .309 -.114 / .589 .348 / .088 .075 / .722 .435 / .030 * H2 Perceived knowledge Of discipline For 1st job management position For job as manager For mid-level position -.087 / .679 .141 / .501 .044 / .833 -.249 / .230 .120 / .568 .345 / .092 .080 / .703 .132 / .529 -.280 / .175 .128 / .541 -.296 / .151 -.099 / .637 * p < .05, ** p < .01. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 48 Further Analysis Given the weak Cronbach alphas reported in Table 1, analysis was performed on individual attitude items to see if they were related to financial success. These results are shown in Table 4. Only two of the possible 30 measures showed a significant relationship with financial success. Finding no relationship with any of the measures for the Assessment #1 was not surprising since the students’ familiarity with the simulation at that time was very low. The students were confronted with the simulation with virtually no preparation, so they probably had low expectations for success at that stage of the exercise. There was one significant relationship and two nearly significant relationships for the measures in Assessment #2. Students financially less successful rated the simulation as more challenging than students who were successful. And there is some weak support that the students found the simulation exercise more enjoyable and satisfying if they were financially successful. For Assessment #3, student who performed well found the simulation to be somewhat more manageable. The relationship between how challenging students found the simulation to be and performance is difficult to interpret. For Solo Practice and Solo Exam, there was a positive (though not quite significant) correlation between level of challenge and performance. The more successful they were, the more challenging they rated the simulation. But there was a statistically significant negative relationship between level of challenge and performance for the team experience. As noted above, they see the simulation more challenging if they struggled to achieve financial success. Why this exists for the Team experience, but not the Solo experience is unknown. DISCUSSION A previous study (Anderson and Lawton, 2006) failed to find a relationship between performance on a simulation and attitude toward the simulation. Given the authors’ anecdotal experience accumulated over 20 plus years of working with simulations and conversations with other simulation users, we suspected that these results were an aberration. As reflected by our hypotheses, we had expected to find that those students who did well on the simulation would rate it more favorably, would have a greater belief that the simulation reflected the discipline, and would perceive that they learned more from participating in the exercise. We found virtually no support for a relationship between financial performance on the simulation exercise and students’ attitudes toward the simulation. Nor was there support for a relationship between performance and their perception of how much they learned from participating in the simulation, how well the simulation reflected the discipline, or their perception of their managerial abilities. While unexpected, it was reassuring to discover that the students were able to assess the merits of their simulation experience regardless of their performance on the exercise. It appears they were able to separate their success on the simulation from their assessment of its educational value. If these findings can be replicated more broadly (e.g., other simulations and other teachers, as noted in the Limitations section, below), and if the students’ perceptions are at all related to reality, then assessing student learning based on financial performance on a simulation exercise needs to be done cautiously. Using financial performance as the sole or primary proxy for learning may be ill-advised. While the assessment measures used in this study were based on student perceptions, the results still call into question the merits of placing too much importance on financial success to assess learning that occurred on a simulation exercise. LIMITATIONS There are two principal limitations of this study. One is the relatively small sample size. These findings are based on a sample of only 25 students. The sample size was hindered by the need to eliminate 14 participants who failed to submit all data elements involved in the study. Clearly there is a need to replicate this study with a larger number of students. A second limitation of the study is that it is based on a single instructor using a single simulation. It is conceivable that students would respond differently to other instructors or to other simulations. Again, this points up the need for replication of this study by other instructors using different simulations. CONCLUSIONS As stated above, the results of this study were contradictory to our expectations. Conventional wisdom would suggest that students achieving financial success on a simulation exercise would translate that success into more positive assessments of the value of the simulation than would those experiencing weaker financial performance. This did not occur during any of the three assessment periods. However, these results are consistent with those found earlier (Anderson and Lawton, 2006). The findings of this study suggest that the absence of any significant relationship between financial performance and the items measured may, in fact, be the rule rather than the exception. A clear need for replication exists if we are to understand the results of this study. Different simulations and different instructors must address this issue to ensure that these results are not artifacts of this particular set of circumstances. Further research in this area should include assessments of student learning that are not based on their self-perceptions. If, in fact, student learning on a simulation exercise is independent of their performance on that exercise, assessment of their knowledge and learning must Developments in Business Simulation and Experiential Learning, Volume 34, 2007 49 be based on evidence other than the results of the simulation itself. REFERENCES Allen, Deborah and Barbara Duch, Thinking Toward Solutions: Problem-Based Learning Activities for General Biology, Harcourt College Publishing (1998). Anderson, P.H. and L. Lawton (2006) “The Relationship between Students’ Success on a Simulation Exercise And Their Perception Of Its Effectiveness As a PBL Problem.” Developments in Business Simulations and Experiential Exercises. Vol. 33 pp 41-47. Reprinted in the Bernie Keys Library, 10th edition [Available from http://ABSEL.org] Anderson, P.H. and L. Lawton (2005) “The Effectiveness of a Simulation Exercise For Integrating Problem-Based Learning in Management Education” Developments in Business Simulations and Experiential Exercises. Vol. 32. pp 10-18. Reprinted in the Bernie Keys Library, 9th edition [Available from http://ABSEL.org] Anderson, P.H. and L. Lawton (2004a) “Simulation Exercises and Problem-Based Learning: Is There a Fit?” Developments in Business Simulations and Experiential Exercises. Vol. 31 pp 68-73. Reprinted in the Bernie Keys Library, 8th edition [Available from http://ABSEL.org] Anderson, P.H. and L. Lawton (2004b) “Applying Problem Based Learning Pedagogy to a Simulation Exercise.” Proceedings of the Applied Business Research Conference. March, 2004. Reprinted in the Bernie Keys Library, 8th edition [Available from http://ABSEL.org] Anderson, P.H. and L. Lawton (2003) “Simulation Exercises and Problem-Based Learning: An Exploratory Study”. Proceedings of the Western Decision Sciences Institute. Vol. 32. Anderson, P.H., D.L. Beveridge, T.W. Scott, and D. L. Hoffmeister, Threshold Competitor: A Management Simulation, 3rd Ed. Prentice Hall, Inc. January, 2003. Anderson, P.H., and L. Lawton (1997) “Demonstrating the Learning Effectiveness of Simulations Where We Are and Where We Need to Go”. Developments in Business Simulations and Experiential Exercises. Vol. 24 pp 68- 73. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] Banta, Trudy, Karen Black, and Kimberly Kline (2000) PBL “2000 Plenary address offers evidence for and against Problem-Based Learning”, PBL Insights, 3, pp.1-7. Barrows, H.S. (1986). “A Taxonomy of Problem-Based Learning Methods.” Medical Education, 20, pp. 481- 486. Bereiter, C. and W.R. Scardamalia (2000). “Commentary on Part I: Process and Product Problem-Based Learning (PBL) Research.” In D.H. Evensen and C.E. Hmelo (Eds.), Problem Based Learning: A Research Perspective on Learning Interactions (pp. 185-195). Mahwah, NJ: Lawrence Erlbaum. Bigelow, John (2004), “Using Problem-Based Learning to Develop Skills in Solving Unstructured Problems.” Journal of Management Education, October 2004, 28, pp. 591-609. Brownwell, Judi, and Daphne Jameson (2004), “Problem- Based Learning in Graduate Management Education: An Integrative Model and Interdisciplinary Application”. Journal of Management Education, October 2004, 28, pp. 558-577. Duch, Barbara, Susan Gron, and Deborah Allen, editors (2001) “The Power of Problem-Based Learning, A Practical ‘How To’ For Teaching Undergraduate Courses in Any Discipline”, Stylus Publishing, LLC. Edens, K.M. (2000). “Preparing Problem Solvers for the 21st Century Through Problem-Based Learning.” College Teaching, 48 (2), pp. 55-60. Faria A. J. and R. Nulsen (1996), “Business Simulations: Current Usage Levels: A Ten Year Update” Developments in Business Simulations and Experiential Exercises, 23, 22-28. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] Green, D. and A. J. Faria (1995), “Are Good Strategies Consistently Good?” Developments in Business Simulations and Experiential Exercises, 22, 31-37. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] Hemmasi, M. and L.A. Graf (1992) “Managerial Skills Acquisition: A Case for Using Business Policy Simulations,” Simulation & Gaming 24, 4: 298-410. Lieux, Elizabeth and Patricia Luoto (2000) “Exploring Quantity Food Production and Service through Problems”, 2nd ed., Prentice Hall. Lohman, Margaret C. (2002) “Cultivating Problem-Solving Skills through Problem-Based Approaches to Professional Development” Human Resource Development Quarterly, 13, 3: 243-261. Mierson, Sheella (2001) “A problem-based learning course in physiology for undergraduate and graduate basic science students” Advances in Physiological Education, Vol. 275, Issue 6, S16-S27, 12-15-98 Miller, H.E., P.L. Schumann, P.H. Anderson, & T.W. Scott (1998). Maximizing Learning Gains in Simulations: Lessons from the Training Literature. Developments in Business Simulation and Experiential Exercises, 25: 217–223. Reprinted in the Bernie Keys Library, 2nd edition [Available from http://ABSEL.org] Miller, Janice (2004) “Problem-Based Learning in Organizational Behavior Class: Solving Students’ Real Problems”. Journal of Management Education, October 2004, 28, pp. 578-590. Schellenberger, R.E., J.A. Hill, and R.B. Keusch (1989), “An Exploratory Study of the Effect of Strategic Emphasis in Management Games on Attitudes, Interest, and Learning in the Business Policy Course.” Developments in Business Simulations and Experiential Exercises, 16, 178. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] http://ecatalog.thomsonlearning.com/150L/lpext.dll?f=XMLHitList&qf=DCQuery&ht=catalog.xml&q=%5bfield,ISBN:0030250331%5d&xsl=productdescription.xsl http://ecatalog.thomsonlearning.com/150L/lpext.dll?f=XMLHitList&qf=DCQuery&ht=catalog.xml&q=%5bfield,ISBN:0030250331%5d&xsl=productdescription.xsl http://ecatalog.thomsonlearning.com/150L/lpext.dll?f=XMLHitList&qf=DCQuery&ht=catalog.xml&q=%5bfield,ISBN:0030250331%5d&xsl=productdescription.xsl http://www.harcourtcollege.com/ http://www.styluspub.com/getpage.cfm?file=books/book5231.html&userid=71525717 http://www.styluspub.com/ http://vig.prenhall.com/catalog/academic/product/1,4096,013083534X,00.html http://vig.prenhall.com/catalog/academic/product/1,4096,013083534X,00.html http://vig.prenhall.com/catalog/academic/product/1,4096,013083534X,00.html http://vig.prenhall.com/ Developments in Business Simulation and Experiential Learning, Volume 34, 2007 50 Sherwood, Arthur (2004), Problem-Based Learning in Management Education: A Framework for Designing Context.” Journal of Management Education, October 2004, 28, pp. 536-557. Spence, Larry (2001), “Problem Based Learning: Lead to Learn, Learn to Lead”, Problem Based Learning Handbook, Penn State University, School for Information Sciences and Technology. Teach, R. and G. Govahi, (1988) “The Role of Experiential Learning and Simulation in Teaching Management Skills,” Developments in Business Simulation & Experiential Exercises, eds. P. Sanders and T. Pray 15: 65-71. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] Washbush, J.B. and J.J. Gosenpud (1991), “Student Attitudes about Policy Course Simulations”, Developments in Business Simulations and Experiential Exercises, 18, 105-110. Reprinted in the Bernie Keys Library, 1st edition [Available from http://ABSEL.org] Wolfe, J. (1985). “The Teaching Effectiveness of Games in Collegiate Business Courses”, Simulation and Games, 16, 251-288. Wolfe, J. (1990). The Evaluation of Computer-based Business Games: Methodology, Findings, and Future Needs. In J.W. Gentry (ed.), Guide to Business Gaming and Experiential Learning (pp. 279–300), New York: Nichols/GP Publishing. Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues