ASSESSMENT AND SIMULATIONS: MEASURING THE ACADEMIC LEARNING COMPACTS WITHIN Developments in Business Simulation and Experiential Learning, Volume 34, 2007 215 ASSESSMENT AND SIMULATIONS: MEASURING THE ACADEMIC LEARNING COMPACTS WITHIN Martin J. Hornyak University of West Florida mhornyak@uwf.edu E. Brian Peach University of West Florida bpeach@uwf.edu Stephen Snyder University of West Florida ssnyder@uwf.edu ABSTRACT A question posed for institutions of higher learning today is, how can the outcomes it desires in graduates be assessed to ensure compliance with educational standards? In summer 2004, our State legislature directed universities and colleges to develop key Student Learning Objectives (SLO) that meet newly established Academic Learning Compacts (ALC) in curriculums. One State-designated learning domain is for students to demonstrate the SLO, Project Management (PM), and this requires it to be measured and evaluated. As an activity completed daily by students, as well as professional managers in the working world, task completion and assessing associated project management activities can involve examining the many complex processes required to successfully complete a project. At our university, we view everyone (i.e. students, faculty, and family) as a project manager with obligations to complete to graduate. For example, our College of Business (COB) students must successfully complete a required business policy analysis and formulation course that engage student teams in a unique learning opportunity by participating in a business strategy simulation. Students are responsible for PM activities that structure, organize, and assess the simulation effort throughout the semester. Rubrics have been developed to measure and evaluate teams and individual students on sound PM practices. Each competing team is able to become the simulations best performing group by applying good PM skills. Teams balance a myriad of performance factors using methods and techniques learned in the COB’ core requirement courses that assist in crafting and executing selected business strategies. A newly developed rubric to measure PM is being evaluated to assess SLO accomplishment. Team members, rating each other on performed PM activities, and final team simulation performance standings are used to assess if the PM SLO has an effect final simulation performance. INTRODUCTION Evaluating and assessing academic engagement of students and their studies is a top priority at academic institutions today including our own. With recent success with the Southern Association of Colleges and Schools (SACS) accreditation process and our pending Association to Advance Collegiate Schools of Business (AACSB) re- accreditation efforts have resulted in several university- and college-level directives concerning the academic engagement of students. One directive has led our State’s universities and colleges to mandate a series of Academic Learning Compacts (ALC) including one to assess students’ project management (PM) capabilities. A “PM” rubric is under development that assists assessment of a PM learning outcome within our COB’ capstone course: MAN4720 Business Policy and Formulation. Goals of this paper include analyzing whether a SLO like PM can be assessed using experiential exercises like simulations. This goal is accomplished in four ways: 1) explaining how a PM Academic Learning Compact (ALC) became established at our school, 2) summarizing the course, the simulation and a Fall 2005 PM rubric initially used to assess PM, 3) discussing the development and use of a new PM rubric in Spring 2006, and 4) using this PM rubric analyze Spring 2006 team simulation performance and the PM peer evaluation conducted by each team. ESTABLISHMENT OF ACADEMIC LEARNING COMPACTS (ALC) The State Board of Governors in summer 2004 adopted policy resolutions requiring all State universities to mailto:mhornyak@uwf.edu mailto:bpeach@uwf.edu mailto:ssnyder@uwf.edu Developments in Business Simulation and Experiential Learning, Volume 34, 2007 216 implement Academic Learning Compacts (ALC) for baccalaureate and graduate degree programs. At a minimum, an ALC must identify the expected core Student Learning Objectives (SLO) that graduating students must achieve to demonstrate their learned content/discipline knowledge and skills, communication skills, and critical thinking skills. This directive also requires that corresponding assessments or rubrics be developed to determine how well student learning is matching articulated expectations (State Board of Trustees – Academic & Student Services Committee Meeting, August, 2004). At our school, both the baccalaureate and graduate degree programs are expected to present program-level ALC and core SLO for the following domains: 1) Content* – concepts, theories, and frameworks of the discipline, 2) Critical Thinking* - information management, higher-level cognitive skills, problem solving, and creativity, 3) Communication*/Literacy – written (reading and written), spoken (listening and speaking), quantitative, technological, and other communication skills as appropriate to the discipline, 4) Integrity/Values – decision making, academic integrity, professional standards for discipline integrity, and 5) Project Management – project planning and execution pertinent to the discipline. Degree programs can also present SLO representing discipline specific skills or special outcomes to distinguish program graduates not identified within the five domains listed above. The above asterisks (*) areas represent domains now being required by our school’s new Board of Governor’s policy (Quality Enhancement Plan, January 2005). THE COURSE, SIMULATION, AND PROJECT MANAGEMENT ALC THE COURSE The COB’ capstone course, MAN4720’s placement in the curriculum provides the ideal venue to assess student mastery of the State-designated ALC. Elements within the capstone policy course are basic strategic management theory, a simulation, and the college’s directed ALC learning domains. MAN4720’s key evaluation components include 40% exams & quizzes, 30% case discussion and analysis, and 30% experiential learning using a business simulation. The course’s simulation element is completed in groups of 3-4 team-selected members. Each team has complete control over making and submitting of simulation decisions. A course schedule identifying key simulation due dates is provided in the semester syllabus. The COB’ learning domains of content, critical thinking, and communication ALC are measured within the course by individual student written case analysis. In addition, MAN 4720 includes a Total Enterprise Simulation called Capstone Business Simulation (Capsim) by Management Simulation, Incorporated as the key experiential learning course element. THE SIMULATION Capsim simulates the yearly activities of a sensor manufacturing organization. Each team must formulate and implement strategy and tactics for a $100 million company. Early in the semester, student teams learn the simulation’s environment by using rehearsals rounds. Practice rounds are then completed to reinforce proper strategies, and Capsim is culminated by each team’s management of a simulated eight years of firm operations. Student teams make all research and development, production, marketing and financial decisions for their firm. Teams are ranked based on their combined performances in Return on Equity, cumulative profit, market share, and market capitalization over the final eight decision periods. The instructors selected these four Capsim variables as the best gauges of course SLO. Each student team selects their strategy, tactics, and approaches to secure marketplace success. Using a simulation to capture SLO in a strategic management course has been articulated previously in simulation and experiential learning literature. Peach (1996) uses Wellington & Faria’s (1995) research findings to show that a positive relationship exists between simulations and strategic management. Basic tenets of strategic management are seen when simulations are used and the participating teams develop clear goals, perform external and internal environmental analysis, introduce clear strategies, monitor performance, and take corrective action (Peach, 1996). Simulations can represent teams managing a firm in a competitive environment. Competitive environments require students to exercise basic PM skills ensuring all activity elements are accomplished in an effective and timely manner. PROJECT MANAGEMENT The PM SLO is still one ALC needing more formal definition since becoming a State Board of Governor’s directive. Specifically, the corresponding assessments or rubrics must be identified and developed to determine how well student learning matches the articulated expectations. A deeper definition of successful PM needs to be developed because success is more than just simply completing the course work. Presently, the COB is working to improve the embed PM skills throughout its entire curriculum with a PM SLO being developed for every course. A project is defined as “a one-time only set of activities with a definite beginning and ending.” (Robbins & DeCenzo, p. 415, 2004). PM in this course, MGT 4720, is defined as “the task of getting the activities done on time, within budget, and according to specifications” (Robbins & DeCenzo, p415, 2004). Typically, PM includes three phases: planning, scheduling, and controlling (Heizer & Render, p56, 2004). Based on these definitions, a PM rubric was developed in Fall 2005 (Figure 1) and used to measure those PM components for student teams (Hornyak, Peach, Bowen, Moes, & Wheeler, 2005). Results from this first Developments in Business Simulation and Experiential Learning, Volume 34, 2007 217 study captured the PM activities of 89 students with 57% being rated as exemplary, 35% rated as acceptable, and 8% unacceptable. Afterwards however, instructors identified major problems and offered key interventions to improve this rubric. Instructors found the Fall 2005 rubric did not adequately capture a student’s PM skills and recommended that the rubric be re-worked. It was also found that instructors varied on how the rubric was used to capture measures of student performance. A follow-on recommendation was made to develop a common method for data collection when using a PM rubric. A WORK-IN-PROCESS: TOWARD A PM RUBRIC FOR FALL 2006 Project management skills are critical to successfully compete in a simulation but unfortunately a PM rubric used in Fall 2005 did not capture many strong team PM activities because of unclear administration procedures, poorly defined rubric inputs, PM areas being evaluated inconsistently by varying raters, and team members not finishing the PM rubric completely. As a result, this study’s findings may be categorized as being questionable. The Figure 1 Pilot Study: Fall 2005 Assessment of Project Management Skills in the Capstone Course Name of Student: __________________________________ Input Points: 0 – 1 2 – 3 4 – 5 Project Planning Fails to meet expectation Meets Expectation Exceeds Expectation 25 Points Appropriate team contract written & signed /5 Decision work breakdown structure & timeline /10 Using acceptable models (M&O, Strategy, I&CA) /10 Instructor Input Process Points: 0 – 1 2 – 3 4 – 5 Project Process (Scheduling/Controlling) Fails to meet expectation Meets Expectation Exceeds Expectation 25 Points Attends group meetings /5 Arrives on time for group meetings /5 Arrives prepared for group meetings /5 Participates in group meeting discussions /5 Works effectively as a group member /5 Individual Student Input Output Points: 0 – 1 2 – 3 4 – 5 Project Delivery (Controlling) Fails to meet expectation Meets Expectation Exceeds Expectation 50 Points Delivers complete project decisions on time /10 Delivers complete project write-ups on time /10 Effective professional content & well-written /15 Effective professional content & well-written /15 Instructor Input Final Rating (Circle the rating based on total points) Point range Rating Point range Rating Point range Rating 90 -100 Exemplary 73 – 89 Acceptable less than 73 Unacceptable Developments in Business Simulation and Experiential Learning, Volume 34, 2007 218 COB is next scheduled to formally administer a new PM rubric in Fall 2006. Since our university is tasked to meet the State-directed ALC, a university Quality Enhancement Program (QEP) task force was formed between the three university colleges (Professional Studies, Art and Science, and Business) in Fall 2005 to design a PM rubric. This task force organized the PM learning domain into four broad outcomes each with specific SLO. First is project conceptualization where students select problems to be solved, identify relevant resources & obstacles, instigate execution planning and develop criteria for a successful completion of a quality project. The second involves assessment of self-regulation within individuals. This outcome is measured by goal completion, managing timeframes, executing appropriate timeframes, and demonstrating flexible and quality contributions. Third, the teamwork skills needed in PM activities are to be assessed. Completed responsibilities, practicing ethical judgment, managing conflicts, and contributing positively to task completion and quality are teamwork activities in this SLO. Finally, the task force wants project deliverables assessed. This includes delivering an acceptable product on time, effective result presentations (i.e. oral, written and/or visual means), effective response to feedback, and makes valid suggestions for improvement. To address the Fall 2005 PM pilot study’s suggested interventions and include recent University task force recommendations, the COB decided to conduct a second PM pilot study with the simulation. A modified PM rubric version using the recommendations from the University task force recommendations was developed in Spring 2006. The COB officially measured the revised PM rubric again in Spring 2006 to get ready to meeting a PM QEP suspense date in Fall 2006. The proposed improved rubric (Figure 2) places measures on the recommended task force project management areas of project planning, self-regulation, teamwork, and deliverables to describe PM team performance within Capsim. This new Spring 2006 rubric is part of a follow-on study to assess if students have an ability to capture meaningful PM skill information and improve the rubric’s administration. Input factors in this version were rearranged to better measure PM needs of simulation teams. The modified simulation PM rubric is directly applicable to bettering the overall COB curriculum by measuring student PM skills. Using a simulation as an experiential learning exercise directly supports the university’s QEP goal of active learning and student engagement. By participating in the simulation, students are intimately involved with PM from beginning to end. To make this PM evaluation rubric effective, raters evaluate basic PM skills of their team and teammates in an attempt to measure the team’s intense simulation learning experience that reinforces a student’s learned strategy and management skills. PM student learning objectives are achieved by participating in Capsim. Teams must divide up the labor and assign specific tasks to individual members. By not identifying all critical tasks, performance may adversely be affected. A team’s ability to recognize unassigned activities and reassign critical performance tasks becomes more difficult as the simulation progresses. Capsim demands teams manage and schedule group meetings and group work. Doing so can ensure team members complete their individual duties and assignments before getting together. Weekly decisions requires group members balance individual preferences and goals to arrive at team decisions. For example, team members may become set on a personal agenda or goal for their products or in managing a Capsim operation. Settling situations like these require using learned negotiation and compromise skills to develop in order to settle these team-level issues. Each weekly decision demands that team members close out complex decisions in a specific time schedule. Teams must make decisions about multiple products in multiple competing segments and make compromises between product managers as well as between Marketing, Production, R&D, and Finance managers. Two major deliverables are required of teams: 1) the weekly decisions and 2) written reports assessing student’s success at situational analysis, issue identification, and strategy development. The above paragraph describes simulation-driven PM skills and activities that students evaluate using the new rubric to determine if our courses are developing mandated student PM skills. SECOND PILOT STUDY SPRING 2006: LOOKING FOR PM AGAIN The largest difference between the Fall 2005 PM rubric and the Spring 2006 PM rubric is the incorporating views of PM across the university and its colleges. The COB Fall 2005 rubric was based on the basic defining concepts of PM: planning, scheduling, and controlling (Heizer & Render, p56, 2004) and strategy (Thompson, Gamble, & Strickland, 2004). The task force project management areas of project planning, self-regulation, teamwork, and deliverables developed out of very long discussions about what various colleges and their departments interpret PM to be. Using the task force developed PM areas as a guide; the Spring 2006 PM rubric was developed. Adjustments were made to eliminate repeating concepts such as “assess quality” and “develop plans” in every learning outcomes as suggested by the task force. The Spring 2006 rubric attempts to define and quantify what PM entails by focusing on the key PM activities to be assessed. If effective PM activities are not examined, team members and academic courses may never see what they must do to be a success. The second PM pilot study (Figure 3) shows results of team PM activities for 105 students. PM scores show 33% were ranked exemplary, 48% were rated as acceptable, and Developments in Business Simulation and Experiential Learning, Volume 34, 2007 219 Figure 2 Project Management Rubric Pilot Study Spring 2006 Pilot Study: Spring 2006: Assessment of Project Management Skills in the Capstone Course Name of Student:_______________________________________Team: _________________ Place an X to identify level of self & your team members project management efforts Input Points 1 2 3 4 5 A. Project planning/ conceptualizations team task Unsatisfactory Below Expectation Meets Expectation Exceeds Expectation Well Exceeds Expectation 20 Pts Identifies relevant resources & obstacles /5 Integrate discipline concepts appropriately /5 Develops strategies for execution in constraints /5 Identifies criteria for successful completion /5 Input Points 1 2 3 4 5 B. Self Regulation Unsatisfactory Below Expectation Meets Expectation Exceeds Expectation Well Exceeds Expectation 15 Pts Sets appropriate goals for completing individual tasks /5 Manages timeframe and schedule well /5 Executes appropriate priorities /5 Input Points 1 2 3 4 5 C. Team-Work Skills Unsatisfactory Below Expectation Meets Expectation Exceeds Expectation Well Exceeds Expectation 15 Pts Contributes positively to task completion as team member /5 Completes responsibilities as a team member /5 Manages conflict among team members /5 Input Points 1 2 3 4 5 D. Project Delivery Unsatisfactory Below Expectation Meets Expectation Exceeds Expectation Well Exceeds Expectation 50 Pts Delivers acceptable product on time /10 Effectively presents results using appropriate means /10 Responds effectively to feedback /10 Makes valid suggestions for improving process & product /10 Assesses quality of contribution accurately /10 Final Rating (Circle the rating based on total points) Point range Rating Point range Rating Point range Rating 90 -100 Exemplary 73 – 89 Acceptable less than 73 Unacceptable Developments in Business Simulation and Experiential Learning, Volume 34, 2007 220 19% unacceptable. After the PM rubric was scored, instructors again voiced concerns. Every student was asked to fill out a PM rubric for each member of the team including themselves. Unfortunately, only one class (Instructor 3 Class 2) was able to have every student comply. Other classes had a few teams with individuals not evaluating everyone on their team or not turning in all their completed PM evaluations. Teams having evaluations for every member but the rater are used along with the referenced class above to formulate Figure 3. ASSESSING ELEMENTS OF RUBRIC RESULTS SIMULATION PERFORMANCE AND PM RATINGS Team PM scores are generated by summing individual team members PM ratings scores for each learning objective area. The team member scores are then summed together and averaged for team PM results. The exemplary, acceptable, and unacceptable scores are generated from summed averages of team members PM outcome categories. Every team member’s individual project planning, self- regulation, teamwork, and project delivery rater’s score is averaged and summed into a PM skill rating category: exemplary: 90-100, acceptable: 73-89, and unacceptable: less than 73. In general, the PM rubric does indicate top simulation performing teams may have a relationship to PM activities. Also, data suggests some top performing teams can assess low performers critically. At the same time, some teams could not separate the rubric from individual course grades deciding that every team member gets a maximum assessment. This suggests that stronger, clearer instructions need to be drafted before the next PM assessment is conducted. INTERVENTIONS Before this rubric is used again, some key interventions need to occur to improve administering this rubric. Instructors found the current rubric did appear to capture student PM skills that may relate to final simulation performance. Instructors need to review the rubric elements and what is being reported in a continued effort to make the rubric user-friendly. The hope is by making the rubric easier to use and understand may eliminate collecting of contaminated data. Stronger rules of engagement (ROE) such as, 1) raters completing the rubric individually, 2) instructors administering the rubric the same (e.g. in class, 20 minutes timeframe, no conversations, etc.), and 3) raters completing one rubric for each person on their team and one for themselves must be established. The ROE are being formalized now between instructors to accomplish stronger rubric implementation instructions for the Fall 2006 PM rubric’s administration. Presently, individual team members fill out one (1) PM rubric for each team member in class. For a class of 30 students, this means instructors must collect 120 rubrics. If instructors have multiple classes, managing rubric collection can become quite the chore. Future semesters may find on- line data collection a good way to collect and to reduce PM information. DATA COLLECTION When common data collection methods are established for the Fall 2006 PM assessment, broader analysis of the findings may be run. For example, correlation analysis between the final simulation score, team PM scores and each PM rating category. Correlation analysis may indicate relationships between certain PM activities and team performance in the simulation. Also, the data may also offer insights into how group work is arranged and conducted. Learning to work in teams during school can be one of the most valuable things students can take into their future work situations. The PM rubric is administered at the end of the semester. Instructors measuring the course’s ALC must be able to use the information gained to improve future student performance. Instructors may be able to work on organizing and instructing teams on what business behaviors provide the most value to team efforts. This may be accomplished by mandating team contracts assigning specific tasks to team members. MAN4720 instructors are currently working on designing approaches for this to happen. CLOSING THOUGHTS This paper begins trying to answer the paper’s opening question, “How can the outcomes academic institutions desire in graduates be assessed to ensure compliance with educational standards?” Our experience can offer thoughts on the importance and application of using rubrics to measure State-directed learning outcomes like PM. If used appropriately, rubrics can confirm what PM activities may be identified as “best practices” for students to use in organizations pre- or post-graduation. Knowing the best PM practices may provide powerful tools that promote operating excellence and better execution for team members. If a best practice is to be valuable and transferable, it must demonstrate success over time, deliver quantifiable and highly positive results, and be repeatable (Thompson, Gamble, & Strickland, 2004). Whether these identified PM activities are repeatable at other times and by other teams is a question that requires further study. The answer seems to depend on whether PM activities can continue to be more clearly defined and measured appropriately and this is exactly where our analysis of the PM ALC is. Assessment of the PM SLO continues as further modifications of the PM rubric are being made. Official reporting of the latest PM measurements will be provided to our university QEP organization in early 2007. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 221 Capsim is an experiential learning exercise that embeds strategic management theories and frameworks students use when making strategic decisions and preparing business reports. Thus, the simulation offers students the chance to apply in a realistic competitive environment the discipline specific skills and knowledge acquired throughout the COB curriculum. This curriculum is driven by a State-directed SLO requiring graduating students be able to demonstrate learned skills in PM. Using simulations as a pedagogical tool has been supported in a variety of empirical research efforts (Gentry, 1990). This paper supports the use of simulations as a means to assess the development of student PM skills. Active learning, student engagement, and development of PM skills come directly out of a simulated competitive environment between teams. Figure 3 PM’s Simulation Assessments 4720 Instructor 1 Class 1 (Rater self evaluation NOT included) Class size: 30 Teams Sim. Final Score Team P.M. Score Exemplary Scores Acceptable Scores Unacceptable Scores SensetheWorld (4) 83.2 83.8 2 1 1 Hungry Hippos (4) 78.3 85.5 1 3 0 Sense-Us (4) 72.6 77.7 0 3 1 Sensible Designs (4) 71.5 100 4 0 0 Team Ramrod (3) 68.3 79.5 0 2 1 Sixth Sense, Inc. (4) 62.0 89.5 1 3 0 Roadrunner (4) 61.4 84.9 1 3 0 SAAB (4) 54.8 77.8 0 3 1 Simulation Project Management Assessment 4720 Instructor 2 Class 1 (Rater self evaluation NOT included) Class size: 23 Teams Sim. Final Score Team P.M. Score Exemplary Scorers Acceptable Scores Unacceptable Scores Bronze Balls (4) 71.4 87.1 2 1 1 Venus, Inc. (4) 70.1 81.8 0 4 0 Synergy (4) 63.5 99.5 4 0 0 Fantastic Four (4) 62.3 71.6 0 3 1 No Name (3) 56.4 89.1 1 2 0 MAJK (4) 35.2 73.4 2 0 2 Simulation Project Management Assessment 4720 Instructor 3 Class 1 (Rater Self evaluation NOT included) Class size: 19 Teams Sim. Final Score Team P.M. Score Exemplary Scorers Acceptable Scores Unacceptable Scores Liger, Inc. (4) 85.5 96.6 4 1 0 VIP’s (5) 68.9 80.3 2 1 2 5th Quarter (4) 66.9 99.5 4 0 0 MoJo Inc. (4) 53.5 77.5 0 3 1 Best of the Best (2) 51.2 43.8 0 0 2 4720 Instructor 3 Class 2 (Rater’s Self evaluation IS included) Class size: 33 Teams Sim. Final Score Team P.M. Score Exemplary Scorers Acceptable Scores Unacceptable Scores Money Makers (5) 71.6 84.9 3 0 2 PJKJW (5) 63.1 70.8 0 3 2 Sensors Plus (5) 46.1 90.9 3 2 0 Team Fox (4) 38.4 86.6 1 3 0 Purple Gorillas (5) 36.6 67.2 0 1 4 4:1 Ratio (5) 35.2 85.2 0 5 0 Apprentices (4) 26.2 77.6 0 2 2 Developments in Business Simulation and Experiential Learning, Volume 34, 2007 222 REFERENCES Gentry (1990) - Editor, Guide to Business Simulation and Experiential Learning/Association for Business Simulation and Experiential Learning. Heizer & Render (2004), Principles of Operations Management. 5th Ed,, Upper Saddle River, NJ: Pearson/Prentice Hall. Hornyak, Peach, Bowen, Moes, & Wheeler (2005), “Conducting A Business Simulation: Faculty And Student Views of Project Management.” Developments In Business Simulation & Experiential Learning, 33, pp. 107-117. Fritzsche & Cotter (1990) “Guidelines for Administering Business Games.” Guide to Business Gaming & Experiential Exercises, Chapter 6, pp.74-89. Peach (1996) “Enhancing Simulation Learning Through Objectives and Decision Support Systems.” Developments In Business Simulation & Experiential Learning, 23, 61-67. Robbins & DeCenzo (2004) Fundamentals of Management: Essential concepts & application. Columbus, OH: Prentice-Hall. Schemerhorn (2003) Management. 8th ed., New York NY: John Wiley & Sons, Inc. Thompson, Gamble, & Strickland (2004) Strategy - Winning in the Marketplace, New York, NY: McGraw- Hill/Irwin. University of West Florida Board of Trustees. (2004). Academic & Student Services Committee Meeting Minutes, August 2004. University of West Florida, College of Business (1997) Academic Planning 1997-2002. September 1997. University of West Florida Online Course Catalog www.uwf.edu/catalog. 2005. Course Listing and Descriptions. 23 September 2005. University of West Florida Quality Enhancement Plan. (2005). Enhancing Student Learning: Creating a community of learners through active learning and student engagement. January 2005. Wellington, W. & Faria, A. (1995) “Are good simulation performers consistently good?” Developments in Business Simulation & Experiential Exercises, 22, 5-11. http://www.uwf.edu/catalog Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues