Developing Participant Satisfaction Models of Experiential Exercises in Business Education Developments in Business Simulation and Experiential Learning, Volume 26, 1999 DEVELOPING PARTICIPANT SATISFACTION MODELS OF EXPERIENTIAL EXERCISES IN BUSINESS EDUCATION Gregory H. Patton, University of Southern California Daniel Cochece Davis, Marist College ABSTRACT Previous business education research establishes predictive relationships between student satis- faction levels and both "utility" and "under- standing" oriented measures. The present study extends this line of research by also examining value relationships. A replication of recent AB- SEL research indicates stable findings, and an extension identifies that utility and understand- ing may actually be better predictors of how valuable students perceived a business course to be, rather than how satisfied they were with the course. INTRODUCTION The present paper focuses on the effect of inte- grating a program of experiential exercises into an introductory business course. This article of- fers an extension of previous empirical research and theoretical development in the area of expe- riential learning and pedagogical praxis by building on preliminary work presented at the 1998 annual meeting of the Association for Business Simulation and Experiential Learning (ABSEL). The present study seeks to confirm the extent to which large lecture, discussion sec- tion, and experiential formats contribute to overall student understanding, as well as student evaluations of satisfaction and value in business education. Drawing on Albertson's (1995) con- cept of "learning enhancement," two competing models appeared capable of predicting major portions of variance in students' overall evalua- tion of their educational experience. The first model is grounded in students' perceived "util- ity" of their educational experience, and focuses on the degree to which the student learns how to apply business information. The second model is based on the student's perceived "mental un- derstanding" of the educational experience and focuses on the degree to which information im- proves understanding of the material. Explora- tory research by Patton, Davis and Govahi (1998) compared these two competing models of "learning enhancement" across three catego- ries of participant learning: large lectures, dis- cussion sessions, and experiential exercises. On a macro level, "nearly equal amounts of partici- pant variance were accounted for across the two models ("mental understanding": r2 = 82%; "utility": r2 = 79%)" (Patton et al., 1998, p.73). On a micro level, it was discovered that "in each of the three categories, the predictor variables accounting for the greatest variances of overall satisfaction were consistently related to "mental understanding" rather than 'utility'" (p. 73). PURPOSE The purpose of the present study is to expand present knowledge of the unique ways in which experiential exercises contribute to overall par- ticipant learning. The present effort reexamines the two primary prediction models of student satisfaction ("mental understanding" and "util- ity"), seeking to replicate earlier findings. Repli- cation of empirical research is foundational and indispensable in ascertaining the stability of sta- tistical findings. In addition, previous research was limited to the relationship between partici- pants' understanding and satisfaction. The pre- sent expansion of this research involves an in- vestigation of the perceived value of the three educational methods, and is based on sugges- tions for future research generated at ABSEL 1998 when Patton et al (1998) was initially pre- sented. Specifically, along with satisfaction, the value participants place on the three instruc- 165 tional categories is also explored, as well as the satisfaction of participants compared to the value participants place on the three categories of instruction. Thus, the present study seeks to examine the contribution of two different per- ceptual measures of students' educational ex- perience. Patton, Davis, and Govahi (1998) suggest par- ticipants' perception of improved conceptual un- derstanding of management concepts is a better predictor than participants' perception of their improved ability to apply these concepts when determining overall satisfaction of a core man- agement course. The present study begins with a replication to verify the stability and reliability of these past findings across all three learning categories. Following this is an exploratory ef- fort to better understand satisfaction and value as two different perceptual measures of class- room success. The best case to be made for a learning methodology is when participants view it as both valuable and satisfying. Although there is a downside to utilizing per- ceptual measures when conducting research into experiential learning and business simulations (Wolf, 1998), the present study does not offer definitive claims of learning gains as the result of participation in simulations and exercises. Such validation claims are best done though other data collection methods. The present re- search focus is on exploring the components leading students to perceive experiential exer- cises as satisfying, as well as valuable. By better understanding participants' perceptions, those components making experiential exercises satis- fying for participants may be better utilized. The creators of experiential exercises can use this understanding to develop activities that partici- pants view as more satisfying and valuable. Ul- timately, this type of inquiry necessitates per- ceptual data collection. This research program is also guided by a con- sumer orientation, resulting in different claims from research involving validation of learning methodologies. As business schools battle for top students, national rankings are increasingly determined by surveys of students, recruiters, faculty and industry executives. Current and former students' evaluations of their educational experience are highly influenced by levels of perceived satisfaction and value. To the extent that experiential learning and business simula- tions can improve students' perceptions of their education, the political justification for inclusion of experiential components in courses becomes far easier, if not critical for the success of some courses. These claims of user satisfaction are not meant to supplant the broader issue of learn- ing effectiveness, rather to provide additional justifications that are, or may be, unique to ex- periential methodologies. METHODOLOGY Survey data was collected at the conclusion of an introductory, upper-division, management course containing lecture, a discussion labora- tory, and experiential components. The particu- lar management course in organizational behav- ior is a core requirement for all business majors being taught at a large, southwestern, private university. During the sixteen-week class, stu- dents of the four-unit management course are provided fifteen lectures, twelve discussion ses- sions, and engage in four experiential exercises. A total of 308 participants voluntarily com- pleted the survey at the semester's conclusion. The survey, building on the earlier research ef- forts of Patton, Davis, and Govahi (1998) fo- cused on student's perceived understanding and utility of organizational behavior concepts. Spe- cifically, this survey sought quantitative feed- back to 21questions, divided among three cate- gories (lecture, discussion lab, and experiential learning sessions). Each of the three categories contained seven standard questions, measured on seven-point scales, relating to the students' perceived abilities in using and understanding course material, as well as their perceptions of 166 the overall value and satisfaction of the educa- tional category. Data analysis was completed using SPSS for Windows (version 7.5). RESULTS A monotonic trend (e.g., Braver & Sheets, 1993) exists among the means of the "overall satisfac- tion" measures of the three categories (e.g., Ex- periential program = 6.42; Discussion lab =5.47; Large lecture = 5.35). Skewness across the three categories varies, showing minimal to severe skews, directed toward the positive evaluations of the learning experiences in each of the learn- ing method categories. Students often evaluated their experiences at the highest evaluation level possible (7.00), resulting in a data set displaying a strong "leaning" or skewness toward the posi- tive end of the scale. The skewness in the par- ticipant's responses was most pronounced in the experiential category (Experiential program = - 2.82; Discussion lab = -1.08; Large lecture = - 0.62). The consistency of participant responses, particularly within the experiential category, led to a highly peaked statistical distribution of re- sponses. Similarly, the kurtosis of the experien- tial program was 9.92, indicating a very strong consistency across students' evaluations of their experiences. The distribution of the other two categories did not demonstrate the same level of peakedness (Discussion lab = .692; Large lec- ture = -.19). Stepwise multiple regression analyses facilitated testing of the two competing satisfaction models of business education. Survey instrument items supporting a "mental understanding" model and a "utility" model functioned as predictor vari- able clusters in the analyses. The initial crite- rion variable used was overall satisfaction with the course experience. Nearly equal amounts of participant variance were accounted for across the two models ("mental understanding": r2 = 43%; "utility": r2 = "41%). A second series of regression analyses was used to determine micro satisfaction level predictors. Four questions (two "mental understanding" and two "utility") within each category served as predictor variables in a stepwise multiple regression for each of the three methods of participant learning (large lec- ture, discussion lab, and experiential program). The overall satisfaction measure within each category of learning served as the criterion vari- able. In each of the three categories (experiential program, discussion lab, and large lecture), the predictor variables accounting for the greatest variances were consistently related to "mental understanding" (r2 =.63 ; r2 = .65; r2 = .30 respectively) rather than "utility" (r2 = .57; r2 = .63; r2 = .24 respectively). Stepwise multiple regression analyses, similar to those used to determine predictive ability of sat- isfaction variance, were used to predict variance in how valuable students perceived their busi- ness management experience. This was con- ducted on a macro level and with each respec- tive learning category. In almost every case, the "mental understanding" and "utility" models functioned as superior predictors of response variance across perceptions of how valuable the educational experience had been for the stu- dents. Though superior, the two models showed similar patterns to the empirical results for satis- faction. On the macro level, nearly equal amounts of participant variance was accounted for ("mental understanding": r2 = 46%; "utility": r2 = "42%). Across the three categories (experi- ential program, discussion lab, and large lecture) nearly equal amounts of values was also ac- counted for: "mental understanding" (r2 =.62 ; r2 = .70; r2 = .57 respectively) and "utility" (r2 = .60; r2 = .67; r2 = .43 respectively). DISCUSSION Though the present study is part of a larger data collection effort, results from an initial data analysis provide several intriguing findings. De- spite the consistency with which the "mental understanding" predictor model accounts for greater variance in participant satisfaction than 167 the "utility" variable, differences between the variances are slight and may not be of practical use. Though the replication of Patton, Davis and Govahi's (1998) initial findings should not be ignored, the most important findings from this line of research may not be that a "mental un- derstanding" model shows slightly superior pre- dictive abilities, but with the recognition that both "mental understanding" and "utility" com- ponents appear to be integral parts of all three learning categories. For experiential programs, these results indicate that participant satisfaction does not lie solely with the skill-building "util- ity" component, but also with the cognitive learning "mental understanding" component. Too often, experiential exercises have been relegated within learning models to a post knowledge retention, post knowledge organiz- ing, skill building phase (see the discussion of learning models in Patton et al, 1998). Research on the multiple phases of the learning process by Hsu (1989), for example, concluded that the home of gaming and simulation is in the third learning phase of skill building and "experienc- ing" that occurs after two previous cognitive phases. The current research confirms that expe- riential exercises span many learning phases. A program of experiential activities in an introduc- tory management course are as likely to assist in the cognitive learning of core concepts as they are to assist with the application of the concepts and the building of applied skills. For the crea- tors of experiential exercises, this understanding can aid in the development of activities that par- ticipants view as more satisfying and valuable. Specifically, exercise should be developed that include both strong cognitive “mental under- standing” elements and the traditional skill building “utility” components. Additionally, the present results suggest that perceptions of how valuable an educational ex- perience is may not be synonymous with how satisfying the experience is perceived by stu- dents. As such, each term probably represents a dimension of students' educational experience. From a consumer orientation, it would certainly be best if the experiential learning components were seen as highly successful in both dimen- sions. The results from 1998 and the 1999 data sets appear to provide strong support along these lines. While confirming the foundational aspects of Patton, Davis, and Govahi (1998), the mono- tonic trend among the satisfaction measure means was different across the three learning categories then that which was reported in the 1998 exploratory study. While the experiential component was stable and was seen as the most satisfying in both data sets, the lecture category dropped from second to third in the ranking in the current data set. Focusing on the experiential component, the mean of the satisfaction with this learning component rose from 6.14 in the 1988 study to 6.42, well above the levels re- ported for the other two learning categories, while the skewness also increased from –1.96 to -2.82. IMPLICATIONS FOR FUTURE RESEARCH Building on the consumer orientation approach, it would be informative to expand the scope of the present research into areas that allow more definitive claims as to the value and satisfaction experienced by students after engaging in expe- riential programs. Additional inquiries also ap- pear necessary into the perceptual changes that may occur in students towards experiential ac- tivities before and after they engaging in a pro- gram of such activities. For example it may be far more effective when conducting a recruit- ment program to put perspective students through an experiential activity rather than to explain or discuss how such activities may be used within a school’s business program. From a methodological standpoint, the percep- tions of how valuable and how satisfying an educational experience is, although different, 168 may have tremendous overlap in terms of pre- dictor variables. This presents empirical re- searchers with the task of separating out co- linearity, or what Pedhazur & Schmelkin (1991) term "multicollinearity" (p. 448). This problem of correlation among variables can unduly di- minish the variance accounted for in regression analyses, as well as other statistical analyses. Future researchers should seek to develop pre- dictor variables that account for unique variance when regressed on criterion variables. REFERENCES Albertson, D. S. (1995). Evaluating experien- tial training: Case study and recommenda- tions. Developments in Business Simulation and Experiential Learning, 22, Braver, S. L., & Sheets, V. L. (1993). Mono- tonic hypotheses in multiple group designs: A Monte Carlo study. Psychological Bulle- tin, 113(2), 379-395. Hsu, E. (1989). Role-event gaming simulation in management education. Simulation & Games, 20(4), 409-438. Patton, G. H., Davis, D. C., & Govahi, G. (1998). Predictive models of learning: Par- ticipant satisfaction of experiential exer- cises in business education. Developments in Business Simulation and Experiential Learning, 25, p.69-75. Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, Design, and Analysis: An In- tegrated Approach. Hillsdale, NJ: Lawrence Erlbaum Associates. Wolfe, J. (1998). Business games teach: Thoughts on the sources of conflicting con- clusions on their effectiveness. Develop- ments in Business Simulation and Experi- ential Learning, 25, p. 151. 169 Table of Contents Volume 26, 1999 ABSEL's Historical Research Interests Back From the Future: An ABSEL Merlin Exercise for the Year 2005 The Contributions of ABSEL During the 1980's ABSEL's Contributions to Experiential Exercises in the 90's ABSEL's Contributions to Experiential learning/Experiential Exercises: The Decade of the 1970's Business Simulations - Algorithms and Model Enhancements: A 25 year Review A Study of the ETS General Field Test as an AACSB Assessment Tool and the Impact of Experiential Exercises and Simulation on Learning A Framework for Assessing the Competencies Reflected in Simulation Performance Developing A Learning Culture: Assessing Changes in Student Performance and Perception Financial Simulation Using Distributed Computing Technology Analyzing Managers' Judgements and Decisions with an Educational Business Simulation Understanding Your Business through Home-Made Simulator Development An Examination of a Reanalysis of the Impact of a Market Leader on Simulation Competitors' Strategies Applying Shocks to TE Simulations: A Demonstration Increasing Efficiency of Management Skill Assessment A Testbank for Measuring Total Enterprise Simulation Learning Developing Leadership Skills - Video Live! LEADSIMM: Collaborative Leadership Development for the Knowledge Society A Team Approach to Producing Multi-Media Laptop and Video Formatted Presentation Tinkertoys Revisited: Exploring Trust Based Relationships Overall Dominance in Total Enterprise Simulation Performance Success or Bankruptcy: The Relationship between Personal and Goal Orientation and Simulation Performance Assessing the Effects of Feedback: Muti-method and Muti-directions in Multi-pedagogical Courses The Missing Ingredients in Experiential Learning Purpose and Learning Benefits of Business Simulations: A Design and Development Perspective Building Capabilities for Change through Laboratory Simulations Modeling Innovation as a Process of Design in Educational Business Simulation The Need to Measurer variance in Experiential Learning and a New Statistic to do so Assessing Effectiveness of an Experiential Oriented Course Over Time Developing Participant Satisfaction Models of Experiential Exercises in Business Education Perceptions of Learning in TE Simulations Students' View on the Use of Business Gaming in Hong Kong Management Gaming's Lost Opportunity: Meditations about the Russian Experience and Prospects for the Future Unanticipated Enhancements in the Business Strategy and Policy Game when Running in Windows 95 Using a Business Game to Demonstrate Broad Business Concepts A New Model for Business Courses (Getting the Student Connected) Seeing the Forest and the Trees: Integrating Knowledge Using Large Scale Simulations in Capstone Business Strategy Classes Is It Here To Stay? A Roundtable Discussion of the Inter-Group Interaction Interactive Tools Used in Applying Financial Concepts Strengthening Essential Skills through a Finance Exercise: Calculation of Beta A Model of Currency Exchange Rates Understanding Currency Exchange Rates: A three-part Exercise Student Experiences in the World Intercollegiate Business Game Competition Student Experiences in the International-Collegiate Business Policy Game Competition Using Boards of Directors in Simulation Environments: Comments From Board Members Sharing Best Practices: Teaching Smarter, Not Harder Star Power: A Simulation for Understanding Power and Empowerment The Marketing Game: A Marketing Principles Simulation Alexander Islands: GSSM Tiny Business Simulator on the WWW Putting Strategy into Strategic Business Games Industry Analysis, Porter's Five Forces Model and Strategic Group Maps in the Business Strategy Game Simulation The Use of Concept Mapping to Improve Student Performance and Understanding of Strategic Management Concepts: A Comparison of Techniques Transformational thinking in the Organizational Behavior Course: The Use of Metaphor as an Assessment Tool Creation of a Virtual Learning Community for the Global Virtual Enterprise Project Inter Institutional use of Educational Resources: Joint Use of Management Simulation Games Business Insights: Theory and Practice with the Aid of a Business Simulation Progress: An Experiential Exercise in Development Marketing The ABC's of teaching the Theory of Constraints to Undergraduate Business Students Demonstrating Principles of Organizational Purchasing Behavior through an Experiential Exercise The Virtual Manager: A Different Simulation for Managing Complexity When the Rules are changing and Chaos Breeds Innovation: Recapturing the Value of Constructive Thinking and Play in Simulation Training The Pitfalls, and Potential, of Actual Events as Problem Drivers Using Business Games to teach Environmental Awareness and Green Management: The International Experience with the ENSIM Game A Review of my ABSEL-Related Work Simulation of Government: A Workshop Using GEO Creating Internet-Based Games Using Perl and JavaScript Who's on First? Exploring the Concepts of Problem-Based Learning, Experiential Learning, and Lifelong Learning Students Learn Customer Service and Selling while Conducting Research So You Want to Run an NFL Football Team–An Honors Interdisciplinary Project Supervised Internship: The Employer's Perspective Using Computer Assisted Simulation to Teach International Business Strategy: A Case of the Multinational Management Game (MMG) Multi-Cultural Experiential Learning: A Computer Simulation in China An Appreciative Stance on Diversity as We Move into the 21st Century: A Timeline Exercise to Identify Key Experiences in Good Work Relationships between Black and White Peers A Day in the Life of an Interactive, Real Time, and Internet Delivered Course: A Demonstration Comparing Internet Search Engines: An Experiential Learning Exercise Total Enterprise Simulations and the Internet: Assessing Student Perceptions and Preferences Teaching Accounting Information Systems in a Practicum Format Providing Experiential Learning in Accounting through a Field Study Payroll Project A Spreadsheet Based Business Simulation Game Computer-Behavioral Simulations Training for Project Managers Simulation Scenarios - Rationale and Illustration