USING FOCUS GROUPS AS A TOOL TO ASSESS LEARNING: A DEMONSTRATION AND A DISCUSSION Developments In Business Simulation & Experiential Learning, Volume 24, 1997 USING FOCUS GROUPS AS A TOOL TO ASSESS LEARNING: A DEMONSTRATION AND A DISCUSSION Louise S. Barden. Centers for Disease Control and Prevention and University of Georgia Ronald S. Barden, Georgia State University ABSTRACT This interactive session includes a focus group dealing with the theory and measurement of learning from simulations and experiential exercises. A discussion of the "how to’s” of this technique is provided along with evaluation of its potential as a assessment tool in this area. BACKGROUND At the conclusion of the 1996 ABSEL meeting, there was a panel discussion of the need to research learning in order to advance the fields of business simulation and experiential learning. Unfortunately, most of the conference’s participants had already departed. One of the authors was present at that panel discussion and began to contemplate possible research methodologies to address the issues raised. One potential methodology involves qualitative as opposed to quantitative research methods. The other author has researched and used focus groups to study communication. As a team, the authors will present a forty-five minute session on how to use focus groups as a tool to research learning. This session will begin with a demonstration focus group in which six attendees will participate as members and all other attendees will participate as observers using the fishbowl technique. A discussion of the experience and a summary of the “how-to’s” of the focus group technique and an evaluation of its potential as an assessment tool in the area of learning from simulations and experiential exercises will follow. FOCUS GROUP OUTLINE The following outline is provided as background for the focus group you will experience. I. Introduction A. Definition: 6-12 individuals, who interact freely in a candid discussion about the issue of interest. B. Characteristics: 1. Everyday conversation 2. Relaxed 3. Semi-structured C. History 1 Robert Merton (1956), Krueger (1994) 2. Commercial product development and packaging 3. Political campaigns 4. Health II. Applications A. Formative research 1. Survey/questionnaire design 2. Identify important issues 3. How people think and feel 4. How to motivate B. Evaluation of programs, products, materials C. Enrich quantitative findings III. Advantages A. Short “up-front” time B. Unexpected findings C. “Puts faces on statistics” D. Good with children and low literacy populations IV. Disadvantages A. Time consuming analysis B. Limited generalizability C. Sources of bias I. Preconception 2. Self-selection 3. Domination of a few V. Tasks A. Develop discussion guide 1. Open-ended, neutral questions 2. Include probes 3. Funneling technique 35 Developments In Business Simulation & Experiential Learning, Volume 24, 1997 4. Pilot test for clarity B. Select moderator 1. People skills 2. Knowledgeable about topic 3. Experienced with identified population 4. Remain neutral 5. Summarize with member checks for validity 6. Attentive, gives positive body language 7. Gives feedback to researcher C. Select observers D. Recruit participants 1. Recruiting strategies a. Volunteer recruiter b. “Snow ball” c. Random sample . Screen for homogeneity E. Collecting and managing data 1. “Organization is NOT optional.” 2. Recording a. Video b. Audio c. Transcripts d. Field notes F. Analysis 1. Field reports 2. “Cut and paste” 3. Computer software G. Interpretation REFERENCES Krueger. R. A. (1994). Focus groups A practical guide for applied research (2nd ed). Thousand Oaks: Sage. Merton, R. K. (1956). The focused interview, The Free Press. Glenco, Illinois. 36 Table of Contents Volume 24, 1997 Incorporating Computer Telephony into the MIS Course A Learner Oriented Infrastructure for videoconferencing Based Distance Education Courses The Identification of Temporally Related Structural Elements of the Experiential Component of Electronic Spreadsheet Tasks Does Involvement Influence Learning from Simulation Participation? Some Relationships with Helpfulness and Performance Outcomes Musings on Business Garne Performance Evaluation Performance on a TE Simulation: What does it represent? The Business Policy Game: An International Simulation - An Assessment Tool Students in Free Enterprise as Experiential Learning The Role of Computer Models in Wargames: A Practitioner's view Using Focus Groups as a Tool to Research Learning: A Demonstration and a Discussion Using Medical Simulations to Teach Multi-Cultural Diversity Using the World Game to Internationalize International Exchange Game Community Services Needs Assessment: An Innovative Approach Introducing Students to Potential Total Quality Management Ethical Dilemmas Privacy in the Workplace: A Situational Analysis Antecedents of Learning in Simulations Demonstrating the Learning Effectiveness of Simulation: Where we are and Where we need to go The Use of Computer Simulations as a Pedagogical Aid in Teaching Management Information Systems Evaluating Simulation Learning in a Distant Learning Instructional Model Financial Engineering of Global Investments The Labor History Game: Playing with the Past Perks Participation Assessing Negotiator's Proficiency with a Negotiation role-play The Art of Negotiating Enhancing Learning and Employee Development through the Assessment of Learning Pedagogy Preferences across selected Dimensions of Culture: A Preliminary Investigation Cooperative Learning: What are we Learning? Effective Use of Mastery Based Experiential learning in a Project Course to improve skills in system analysis and Design Simulations and Learning: Can we prove a relationship? (Seminar) Predicting and Reviewing NYSE Stock Prices by use of Basic Statistical Analysis and Logic Toy Car Depreciation Exercise ABSEL as Home Community: An Interactive Exploration Service Learning: Linking Academic Study to Community Development and Business Enhancement Ability of Efficient Evaluation of Knowledge-Based Management Strategies Corporate Ethics Training Programs Modeling Attributes in Demand Functions of Computerized Business Simulations: An Extension of Teach's Gravity Flow Algorithm Expert Systems Combined with Neutral Networks: Tools to Benefit the Marketing Researcher Computer Game Design: New Directions for Intercultural Simulation Game Designers Consistency in Simulation Performance over Time and Across Simulation Games The Impact of an Artificial Market Leader on Simulation Competitor's Strategies Business Plans, Case Studies, and Total Enterprise Simulations: A Natural Co-existence An Exploration into the Non-Use of Business Simulations The Market Game: Interactive Learning Through Market Simulation Plotting Brand Trajectories with the COMPLETE PPM package: A Market Segmentation Analysis and Positioning Tool Me and Mine Inc: An Exercise in Management Theory Learning to Differentiate Leadership from Managerial Position Business Ethics Survey: A Perspective from the Retail Industry Experiential Learning in Demand Analysis for an Agricultural Commodity Marketing on the Internet: A Pedagogical Exercise College Students Need Simple Computer Simulations, Especially for International Business Courses An Analysis of Student Attitudes, Performance, and Strategies in a Simulation Competition Based on a Controlled Product-Market-Entry Game Structure The Crystal Enterprise: Application of a Non-Computerized Simulation Model in a Process of Organizational Change Threshold: A Windows-Based Behaviorally Oriented Total Enterprise Simulation Coaching Business Game Teams Using a Decision Variable Optimizer Rock and Roll is here to Stay: Enhancing Experiential Pedagogy with Musical Exercises An Example of Business Process Analysis Simulation for Customer Software Support The Use of Business Gaming in Hong Kong Academic Institutions Using the Integrative International Simulation INTOPIA Mark 2000 in a Concentrated MBA Curriculum A Group Experience as an Integrated Part of the Core Management Course Empowered Learning in the Classroom The Inter-Group Interaction: An Innovative Approach to Cooperative Learning Courses that Utilize Student Team: An Approach to enhancing their Effectiveness Thoughts about the Measurement of Learning: The Case for Guided Learning and Associated Measurement Issues Measuring Student Learning Using Business Simulations: A Theory Based Perspective On the Use of PC Fingame in an Undergraduate Finance Course How Managers get Things Done: A Virtual Soundbite Internet Experiential learning in the Principals of Marketing Classroom: A Pedagogical Approach Current Student Perceptions Relative to Business Simulations Beyond Capitalism: Designing Business Policy and Social Justice The Use of Boards of Directors to Evaluate Reports and Presentations in an Undergraduate Business Policy Course Beacon Lumber: An Experiential Introduction to Financial Accounting The Application of Organizational Motivation Principles: The Experiential Business Simulation Motus Manufacturing Avoiding a Bogey: Grading Case Discussions Scientifically Communicating Consumer Behavior II: A Modified Exercise Using Personal Consumption Journals in Condensed Courses Designing Instruments for Assessing the Effectiveness of Simulations, (Seminar) 360º Performance Feedback: Appraisal vs. Assessment The Cafeteria Approach to Managing an Academic Career: Remaining Non-Perishable while Doing Your Own Thing An Experiential Exercise Related to Person-Organization Fit and it's Consequences For Today's Dynamic and Changing World, Business Simulations need to be expanded to encompass Much Greater Complexity: A Demonstration The Incident Process: A Case in Reverse Contextually-Anchored Business Simulations The Energy Factor: Building Motivation in the Simulation Gaming Environment The Design of an Internet Game