HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP... Developments in Business Simulations and Experiential Learning, Volume 32, 2005 HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP ACADEMICALLY SOUND COURSES THAT ALSO MEET STAKEHOLDER NEEDS Gretchen Vik San Diego State University (Panel Chairperson) Douglas Micklich Illinois State University Monique Forte Stetson University ABSTRACT Topics we will cover: • Each person will discuss his/her school’s current assessment requirements and implementation plans. Accreditators, legislatures, and other stakeholders are all interested in assessment in our universities today. This panel discussion will bring together three people from different areas of the country who teach at state universities and a private school. Rather than focusing on accreditation, this panel will focus on how to assess course and programs, and how our experiential learning and simulation projects can be evaluated. The end goal of assessment is curriculum improvement, which serves all stakeholders. • Monique Forte: AACSB assessment seminar, developing direct measures, curriculum mapping • Gretchen Vik: using a survey (indirect measure) to assess a simulation project, developing rubrics for experiential assignments • Doug Micklich: how AACSB assessment compliance with the guidelines, how he assesses the strategy class project PANEL OVERVIEW Other topics of interest to us and our audience: All three participants have experience implementing various assessment systems at their universities. SDSU has recently had AACSB accreditation renewed under the new system. Many schools have implemented indirect measures such as student, alumni, employer, and stakeholder surveys (Attachment 1: Sample Indirect Assessment Plan), but the AACSB advice on assessment is to include direct measures, use grading rubrics shared with the students ahead of time, and list learning outcomes on each syllabus. • What methods (exams, surveys, grading rubrics, learning outcomes, external reviewers, employer/stakeholder evaluations, etc.) are effective in different types of courses? • How can courses meet both the needs of academic accrediting bodies AND local employers who hire our students? • What are the advantages/disadvantages of numeric evaluations and assessments? Developing grading rubrics that make explicit what each assignment is planned to accomplish is a time- consuming task, but helps students know what is expected of them (Attachment 2: Grading Rubric). One way of ensuring that courses do not overlap is to create a curriculum map showing which general and specific competencies are embedded in each course business students take (Attachment 3: Stetson University Undergraduate Curriculum Map). • How do you write effective learning outcomes? OUTCOMES OR GOALS OF THE SESSION Attendees will know more about how to write up an assessment plan, to create grading rubrics and surveys, to use numeric evaluations, and to write learning outcomes. We anticipate that the audience will have stories to share about developing their own plans. Simulations and Experiential Learning assignments also need rubrics and learning outcomes, and we will talk about how these may differ from those for more traditional assignments. Handouts dealing with sample plans, outcomes, rubrics, etc. will be distributed. 315 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 REFERENCES Information and Decision Systems Department Assessment Plan, College of Business Administration, San Diego State University, 2001. This indirect assessment plan has since been modified and strengthened. Short Report Grading Rubric, Information and Decision Systems Department, College of Business Administration, San Diego State University , 2001. This sample rubric is one of many used in a sophomore business communication class to show students ahead of time what aspects of the paper will be evaluated most and how they will be evaluated. Stetson University School of Business Administration Undergraduate Curriculum Map, 2004. This sample curriculum map shows how course competencies are distributed over undergraduate courses. Attachment 1: Sample Assessment Plan for a Department We will continue existing and ongoing measures currently in place: • student outcomes assessment for undergraduate and graduate programs; • external reviews of undergraduate and graduate programs; • students' evaluation of their undergraduate and graduate experience; • employer evaluations; and/or • graduates qualifying for professional licenses and certificates. We also integrate information from the following new sources: • Alumni Advisory Board Survey Results • Questions to be added to student evaluations • AACSB Review of program • Other review or evidence from professional associations that our curriculum meets national standards and is being revised for current trends in education. • National questionnaire results about importance of the department’s four disciplines to graduates. Implementation We will use information gathered by these various methods to keep courses current, add new courses, and drop courses no longer useful to students and employers. Particularly in information systems, we will maintain rotating topics courses to help us react to technological changes more quickly than the current process allows. Feedback on curriculum or program changes may be evaluated by faculty, students, alumni, and/or employers as appropriate. 316 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Attachment 2: Sample Grading Rubric SHORT REPORT GRADING SHEET NAME___________________________________ SECTION_______ TEAM_______ CONTENT One paragraph introduction—shows organization Well–organized information Summary GRAPHIC CHOICE OF MATERIAL Quality Introduced MECHANICS Sentence structure WORD CHOICES SPELLING/DICTION Punctuation Other Agreement Other CITATIONS (10%) In text Source list Chicago style Annotated FORMAT—MEMO (10%) Date To From Subject Appropriate headings COMMENTS GRADE Content 60% Graphic 20 Citations 10 Format ______ 10____ Total 100 317 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 ATTACHMENT 3: STETSON UNIVERSITY SCHOOL OF BUSINESS ADMINISTRATION UNDERGRADUATE CURRICULUM MAP 1st Year General Education Business Foundation Common Body of Knowledge BN109 EBT191 DS280 DS350 ATG211 ATG212 BN209 BN301 MGT305 MKT315 FIN311 BN495 General Competencies: Teamwork + + + + + + + Communication + + + + + + + + Creative Thinking + + + + + + + + Adapting to Change + + + + + + + Specific Competencies: Workforce Diversity + + + + + + Ethics + + + + + + + + + + + + Technology + + + + + + + + Global Perspective + + + + + + + + Analytical Skills + + + + + + + + + + + Course Titles: BN109: Special Topics in Business (First Year Business Experience) EBT191: Introduction to Information Technology DS280: Introduction to Statistics DS350: Quantitative Methods for Business Decisions ATG211:Introduction to Financial Accounting ATG212:Introduction to Management Accounting BN209: The Legal, Social, and Ethical Environment of Business BN301: Business Communication MGT305: Principles of Management MKT315: Principles of Marketing FIN311: Business Finance BN495: Strategic Management 318 Table of Contents Volume 32, 2005 LEARNER BEHAVIOR IN THE ONLINE CLASSROOM EXPERIENCE THE EFFECTIVENESS OF A SIMULATION EXERCISE FOR INTEGRATING PROBLEM-BASED LEARNING IN MANAGEMENT EDUCATION DEMONSTRATION OF FOUR WEB-BASED SIMULATIONS THRESHOLD COMPETITOR: A MANAGEMENT SIMULATION ENTREPRENEUR: A NEW VENTURE SIMULATION MERLIN: A MARKETING SIMULATION MICROMATIX: A STRAGETIC MANAGEMENT SIMULATION LEARNING STYLES INFLUENCES ON SATISFACTION AND PERCEIVED LEARNING: ANALYSIS OF AN ONLINE BUSINESS GAME INTERNATIONAL INTERNSHIPS: DESIGN AND EXPERIENCES SIM MAP: TURNING ACTION-BASED LEARNING INTO SIMULATED CONSULTING PROJECTS NOTEL HEALTH SERVICES: A ROLE-PLAYING SIMULATION TEACHING EXPERIENTIALLY WITH THE MADELINE HUNTER METHOD: AN APPLICATION IN A MARKETING RESEARCH COURSE SIMULATING CUSTOMER LIFETIME VALUE: IMPLICATIONS FOR GAME DESIGN AND STUDENT PERFORMANCE VIRTUAL PROGRESS: SIMULATING ECONOMIC DEVELOPMENT ONLINE STRATEGIC MANAGEMENT: AN EVALUATION OF THE USE OF THREE LEARNING METHODS IN CHINA ADOPTION OF DISCUSSION-BASED TEACHING AND ASSESSMENT IN TEACHING STRATEGIC MANAGEMENT IN CHINA STUDENTS' VIEW ON THE USE OF CASE METHOD IN CHINA CHINESE STUDENTS' PERCEPTIONS OF BUSINESS GAMING EXPERIENCECSR - A CORPORATE SOCIAL RESPONSIBILITY SIMULATION CREATING DYNAMIC INTERACTION IN A VIRTUAL WORLD: ADD VALUE TO ONLINE CLASSROOMS THROUGH LIVE ELEARNING AND COLLABORATION: A DEMONSTRATION CAPABILITIES OF EXPERIMENTAL BUSINESS GAMING A COMPARISON BETWEEN SOLUTIONS AND DECISIONS IN A BUSINESS GAME VALIDATING BUSINESS SIMULATIONS: DOES HIGH PRODUCT QUALITY LEAD TO HIGH PROFITABILITY? ALIGNING ART AND EPISTEMOLOGY: ILLUSTRATIONS TO DISTINGUISH DISCOVERY FROM KNOWLEDGE BUILDING TUTORIALS USING WINK STUDENTS AS LAB RATS: THE ETHICS OF CONDUCTING NON-PEDAGOGICAL RESEARCH IN THE CONTEXT OF CLASSROOM SIMULATIONS AND EXPERIENTIAL LEARNING THE EFFECT ON GAME PERFORMANCE OF DIFFERENT MEASURES AND UNITS OF ANALYSIS IN QUANTITATIVE ANALYSIS ANALYZING AND THINKING WHILE PLAYING A SIMULATION COMPUTER BUSINESS SIMULATION DESIGN: THE ROCK POOL METHOD EXPANDING THE ROLE OF E-ROOMS IN DISTANCE LEARNING APPLICATIONS TO MANAGEMENT EDUCATION APPLICATION OF TRADITIONAL AND ONLINE JOURNALING AS PEDAGOGY AND MEANS FOR ASSESSING LEARNING IN AN ENTREPRENEURIAL SEMINAR DEVELOPING MANAGERIAL EFFECTIVENESS: ASSESSING AND COMPARING THE IMPACT OF DEVELOPMENT PROGRAMMES USING A MANAGEMENT SIMULATION OR A MANAGEMENT GAME INTERNATIONAL MANAGEMENT GAME Œ AN INTEGRATED TOOL FOR TEACHING STRATEGIC MANAGEMENT INTERNATIONALLY STUDENT EXPECTATIONS OF SIMULATIONS DISTANCE EDUCATION DELIVERY OF AN INTENSIVE SIMULATION BASED COURSE TEACHING SERVICE LEARNING USING A BUSINESS GAME ROLE-PLAY SIMULATION EDUCATIONAL PERSPECTIVE OF COLLABORATIVE VIRTUAL COMMUNICATION AND MULTI-USER VIRTUAL ENVIRONMENTS FOR BUSINESS SIMULATIONS SIMULATION PERFORMANCE & PREDICTOR VARIABLES: ARE WE LOOKING IN THE WRONG PLACES TO MEASURE THE RIGHT LEARNING? VIDEO CASE: JET-A-WAY INC. Œ FOCUSING ON DIVERSITY AND ENTREPRENEURIAL LEADERSHIP ACTIVE LEARNING: WHAT IS IT AND WHY SHOULD I USE IT? FACILITATING THROUGH COLLABORATIVE REFLECTIONS TO ACCOMMODATE DIVERSE LEARNING STYLES FOR LONG-TERM RETENTION ONLINE CUMULATIVE SIMULATION TEAM PERFORMANCE PACKAGE WHEN PROPHECY FAILS: A SMALL SAMPLE, PRELIMINARY STUDY USING EXPERIENTIAL LEARNING TO INTEGRATE THE BUSINESS CURRICULUM FORECASTING STOCK VALUE EMPLOYING PROGRESSIVE PRACTICES AND PRINCIPLES TO FACILITATE SEMINAR ROOM LEADERSHIP AMONG LEARNERS: SHARED POWER AND COLLECTIVE ACCOUNTABILITY INDIVIDUAL ACHIEVEMENT DOES NOT GUARANTEE TEAM PERFORMANCE: AN EVIDENCE OF ORGANIZATIONAL LEARNING WITH BUSINESS GAMES DECISION MAKING IN BUSINESS SIMULTION DESIGN ZUG UM ZUG 2015: COLLECTIVE BARGAINING AS A TWO-LEVEL GAME A NEW METHOD FOR MODELING INNOVATION AND R&D IN BUSINESS SIMULATIONS: ILLUSTRATED WITH A SIMULATION OF A NEW PRODUCT DEVELOPMENT PORTFOLIO DEVELOPING A MICRO SIMULATION EFFECT OF MARKET SHARE AND PRODUCTION EXPERIENCE ON COMPANY PROFITABILITY RE-DESIGNING A CURRICULUM THAT VALUES A WORK-INTEGRATED APPROACH TO STUDENT LEARNING HOW SIMULATIONS AND EXPERIENTIAL LEARNING FIT AS WE COMPLY WITH LEGISLATIVE AND AACSB ASSESSMENT GUIDELINES: HOW TO DEVELOP ACADEMICALLY SOUND COURSES THAT ALSO MEET STAKEHOLDER NEEDS EVALUATING SERVICE LEARNING: REFLECTION AND ASSESSMENT FROM THE STUDENT POINT OF VIEW SIMPLIFYING AND ENHANCING FINANCIAL ANALYSIS IN CASES AND SIMULATIONS OVERCOMING THE BUSINESS GAME COMPLEXITY PARADOX EXPLORING THE PREFERENCE IN LEARNING APPROACH AMONG THE HONG KONG UNIVERSITY STUDENTS: CASE STUDY, PROBLEM-BASED OR TRADITIONAL TEXTBOOK QUESTION AN EXERCISE FOR EXPLORING THE RELATIONSHIP BETWEEN JUNGIAN PSYCHOLOGICAL TYPES AND POLITICAL STYLE IN THE WORKPLACE EVALUATING THE DIRECTION OF RESEARCH IN ONLINE EDUCATION: ARE WE GOING ANYWHERE? AIS RAIL SYSTEM: A COMPUTER-BASED JOB-ORDER COST SIMULATION BLOOM BEYOND BLOOM: USING THE REVISED TAXONOMY TO DEVELOP EXPERIENTIAL LEARNING STRATEGIES DELIVERING A TECHNOLOGY-BASED CASE IN A TECHNOLOGICAL WAY: THE SMARTCART CASE USING THE INTERNET TO ENHANCE COURSE PRESENTATION: A HELP OR HINDRANCE TO STUDENT LEARNING TEACHING PRACTICES: A CLUSTER ANALYSIS OF STUDENTS IN HONG KONG TEACHING PRACTICES: A CLUSTER ANALYSIS OF TEACHING STAFF IN HONG KONG EVALUATING A SIMULATION WITH A STRATEGIC EXPLORATION TOOL A SYMBOLIC MODEL OF THE SIMULTANEOUS ACHIEVEMENT OF CONCRETE BENEFITS AND LEARNING BY PARTICIPATING GROUPS IN EXPERIENTIAL ACTIVITIES: ‚THE SPHERE OF EXPERIENTIAL LEARNING