Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players Page 383 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 ABSTRACT The “free rider” syndrome plagues many group efforts. This is an especially harmful phenomenon for those using business games as team-based learning experiences. Be- cause of this lack of engagement naturally associated with players in groups, this paper tested a set of predictors of a player’s engagement or unengagement in the simulation being played. Engagement varied widely when measured by the proportion of online screen visits made by each team’s members. Of the study’s 76 subjects, 18.4% were highly engaged with 38.2% of them being almost com- pletely unengaged. Those who actively participated had significantly higher pre-college aptitude scores and cumu- lative college grade-point-averages. Despite the group’s bi -modal nature, there were weak but significant predictive relationships between player aptitudes and collegiate grade-point-averages and engagement levels. These weak correlations indicate there are other more important fac- tors associated with a player’s level of game engagement. Educators, and especially those preparing students for business careers, have endorsed the use of active group learning environments (Becker & Dwyer, 1998). More im- portantly, there has been an increase in their use (Brooks & Ammons, 2003; Lejk & Wyvill, 2001) based on their sup- posed ability to deliver the benefits of high cognitive in- volvement and greater activation and motivational levels for the materials and tasks being presented (Peterson & Miller, 2004). For those being trained for the business world, whose practitioners must bring about instrumental effects, group-learning methods help the student to learn about group dynamics, develop their interpersonal skills, expose them to diverse perspectives and produces more well-rounded assignments and other task-related outputs (Mello, 1993). Despite these often-documented benefits, there are still factors that make many group-learning efforts frustrating to their participants and sub-optimal in the results they produce (Ashraf, 2004; Gatfield, 1999; Kerr, 1983). One of these factors is the presence of “free riders” or those who engage in social loafing (Williams & Karau, 1991). These individuals do not bear their fair share of the group’s work while obtaining all of its benefits (Albanese & Van Fleet, 1985; Jones, 1984). This imbalance frequently leads to dominance by a few and less learning for others, bitterness, claims of prejudice by those negatively evaluated and a lowering of the quantity and quality of the group’s output (Brooks & Ammons, 2003; McElhinney & Murk, 1994; Schoenecker, Martell & Michlitch, 1997). The problem of getting the free rider to put forth a requisite level of effort is exacerbated in student-related projects. In the workplace, employees are more-directly rewarded for their efforts with paychecks, promotions and other entitle- ments such as employee benefits (Piezon & Donaldson, 2004). Students, however, do not have these rewards and punishments at their disposal other than by awarding group-dispensed grades and peer evaluations. These may not be meaningful given the rewards of free-ridership (Murphy, Wane, Liden & Erdogan, 2003). Because of the presence of social loafing in all group pro- jects, and possibly those associated with business games that depend on the joint efforts of their players, this paper attempts to quantitatively document its presence. It does so by not relying on anecdotal and post hoc player evalua- tions that may not capture the amount of free ridership manifested on their teams. In doing so, the paper will at- tempt to identify the degree a player’s pre-game scholastic aptitudes and academic achievement levels are associated with their degree of engagement in a business game learn- ing environment. LITERATURE REVIEW This paper is concerned with documenting the actual presence of free riders on business game teams. Therefore, it does not cover the vast literature on why there are free riders and social loafers. For those topics the reader is re- ferred to the pioneering work of Ringelmann (1913), the meta-analytic study by Karau & Williams (1993) that documents its pervasiveness and Latane, Williams and Har- kins‘ (1979) summary of the theories on its causes. In this case, the paper is interested in the degree to which a game THOSE WHO DO AND THOSE THAT DON‘T: A STUDY OF ENGAGED AND DISENGAGED BUSINESS GAME PLAYERS Joseph Wolfe Experiential Adventures LLC Jwolfe8125@aol.com Robin McCoy University of San Diego mailto:Jwolfe8125@aol.com Page 384 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 player pays attention to the institution‘s requirements for engaging in a particular game‘s online participation activi- ties in preparation for their team‘s decision-making cycle. Anderson & Lawton (2004) have characterized the playing of a business game as being a large, continuous problem-solving exercise. With this perspective, a business game is the application of McMaster University‘s problem- based approach to education (Koh, Khoo, Wong, & Koh (2008). In this vein, the requisites for finding a series of continuous solutions to the problems players face in a game are related to those associated with successful problem- solving activities simultaneously involving physical move- ment, navigating the game‘s interface, and cognitive/ mental thought processes. The analogy used here is that of solving the Rubik‘s cube riddle. The task is to reassemble a scrambled set of six differently colored faces so that only one of those colors appears on each of the cube‗s six faces. The riddle‘s solution entails a number of moves of the faces, a sequential solving each of the cube‗s three layers without destroying any of the previously constructed lay- ers, and also having the patience to make the number of moves needed to reach the cube‘s solution. Many do not have the patience or endurance to go through this process as there are 519 quintillion possible arrangements of the cube‘s pieces. Continuing the Rubik‘s analogy to the play of an inter- net-served business game, its decision makers must physi- cally manifest the following actions: 1. Variety—Pursue a number of different screens (turn a number of cube faces). 2. Depth—Intensely examine the screen‘s informa- tion (drill down through the cube‘s three layers). 3. Endurance—See the task to its end (keep turning cube faces until the riddle has been solved). Accordingly, this paper defines the general concept of game engagement as consisting of three attributes or activi- ties. If a game‘s company is to be successful, it is believed players must vigorously pursue the information the game provides and they should maintain this pursuit on a long- term period-by-period basis. This would entail visiting EXHIBIT 1 SUBJECT DEMOGRAPHICS Descriptor Value Population size 76 Course drop rate 13.2% Age 21.0 ACT/SAT Percentile 56.6 High School Grade-Point-Average 3.25 Gender: Male 64.5% Female 35.5% Ethnicity: White 72.4% Black 7.9% Hispanic 5.3% Asian 2.6% American Indian 2.6% Foreign 3.9% Unknown 3.9% Other 1.3% Page 385 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 most, if not all the game‘s screens each quarter, even if some degree of functional specialization was being used by the company. In addition to visiting the game‘s screens, each player would study each screen‘s information and decision-making inputs with a high level of depth, i.e., de- vote in depth rather surface depth to the screen‘s task. Fi- nally, each player should consistently pursue both depth and variety throughout the game‘s duration. If all these conditions have been met, such a player would be consid- ered to be highly engaged in the game. A player could be considered unengaged with the game if few screens were visited, each visit was very short and there were many gaps in the number of visits made on a periodic basis. METHODOLOGY The study‘s subjects were introductory-level business students at a campus-based liberal-arts university. Its Fall 2009 enrollment was 4,262 full-time students. Its business administration major is granted by its College of Arts and Sciences. The demographics associated with this study‘s participants (n=76) are presented in Exhibit 1. They played The Global Business Game, Business Basics Edition (www.onlinegbg.com) on self-selected three -member teams for its six rounds of play. The game (GBB) is an internet-delivered computer-based simulation de- signed for introductory-level business students. One of the game‘s features is the instructor‘s ability to monitor and otherwise record the time, date, and duration of all internet activities engaged in by its players. The fea- ture also divulges the screens viewed and how they were used such as scanning, editing or printing them. A review of all the screen activity logs revealed that no screens were printed. This was interpreted as being an indication that the only source of information a player was able to obtain for each decision-making session came from a viewing of the game‘s screens rather than viewing output printed by one player and distributed to the team‘s other players. EXHIBIT 2 PLAYER PARTICIPATION BY ENGAGEMENT LEVEL EXHIBIT 3 PARTICIPANT CHARACTERISTICS BY GENERAL ENGAGEMENT Note: Tests conducted as one-tailed tests. Engagement Level Number Percent Disengaged—Free Riders 29 38.2% Indifferent—Easy Riders 33 43.4% Engaged—Hard Chargers 14 18.4% Total 76 100.0% Participant Characteristic Game Engagement Significance Disengaged Engaged Male 58.6% 57.1% n.s. Female 41.4% 42.9% n.s. White 43.8% 57.1% n.s. Black 26.9% 21.4% n.s. Hispanic 11.5% 14.3% n.s. Asian 3.8% 7.1% n.s. Aptitude 43.9 77.1 0.001 HSGPA 3.06 3.54 0.063 CGPA 2.69 3.31 0.016 Page 386 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 At the game‘s end, the subjects were asked via a five- minute, one-page questionnaire, to rank their teammates regarding their economic and social contributions to their company‘s results. Each player‘s engagement level was initially determined by the number of screens each player visited as a proportion of the team‘s total screen visits. The number of screen visits was considered a reasonable and objective measure of grossly defined game engagement as logging screen time was the only way a company‘s players could prepare themselves for any face-to-face meetings that might occur. The log review found the student population fell into three engagement groups. Those classified as being en- gaged, termed ―Hard Chargers‖, visited the game‘s website for at least five of the game‘s six required decision-making rounds. They also made at least 75.0% of their company‘s total visits. Those classified as being disengaged, termed ―Free Riders‖, visited their game‘s website less than 15.0% of their company‘s total screen visits. In-between or indif- ferent groups termed ―Easy Riders‖ were inconsistent in their number of visits, the times spent by screen and their participation throughout the game. Their inconsistencies across all the dimensions of engagement made them an amorphous entity lacking any identity or characteristics. Exhibit 2 indicates the number and proportions of players involved. An examination of those who were either Disengaged or Engaged produced the bi-polar results presented in Ex- hibit 3. Each had unique aptitudes and academic achieve- ment levels but similar demographics. Based on this infor- mation it would appear the engaged group brought superior aptitude levels and college achievement results, and almost significantly superior high school achievement levels, to the game and therefore might demonstrate different playing behaviors. RESULTS The results presented in Exhibit 4 indicate there were distinct differences in how each player type engaged in the game. Those in the Engaged group visited 218.0 or 42.1% more times than the Disengaged group. Those in the En- gaged group also demonstrated more Depth by spending 61.1% more time during each screen visit. Most impor- tantly, from a long-term participation perspective, those who were Disengaged did not visit their company‘s screens 81.0% of the time while the Engaged players rarely missed an opportunity to interact with the game‘s online system. The lack of endurance, or ―staying power‖ of the Dis- engaged group, is highlighted in Exhibit 5. The graph indi- cates that both groups fell in the number of times they vis- ited their company‘s website toward the end of the game‘s final decision round. This is to be expected as players quickly narrow their search routines once their strategies have been put into place. They also become more selective as to what needs to be viewed. The differences between the two lies in the Engaged group‘s higher average starting position, which was 393.4 screens per player and a high degree of continuity with their visits. In the game‘s first period an average of only 20.2 visits were made by the Dis- engaged group, and they made hardly any visits between the game‘s third and fifth periods. DISCUSSION Based on these results it appears this study‘s students engaged in behaviors that did and did not maximize what they could learn from the game‘s learning environment. Many of the unengaged students did not attempt to prepare themselves for their decision-making sessions and did not examine their company‘s results. If they did so, it was done inconsistently at a very superficial level. The Engaged players did what game designers, instructors and gaming advocates hoped players would do. They conducted a wide EXHIBIT 4 GAME ENGAGEMENT BY ENGAGEMENT DIMENSION Notes: 1. Variety = The number of screens viewed per decision period. 2. Depth = The average number of minutes spent per screen viewed. 3. Endurance = The number of sessions accessed as a percent of all sessions available. Engagement Dimension Average Game Engagement Score Significance Disengaged Engaged Variety 5.3 screens 223.3 screens p < 0.001 Depth .18 minutes .29 minutes p = 0.015 Endurance 19.0% 97.0% p < 0.001 Page 387 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 search of the screens offered by the game, they spent a number of minutes per screen and they did this throughout the simulation‘s run. Given these associations Exhibit 6 indicates the degree a particular student‘s engagement level could be predicted based on measured scholastic aptitude scores and past and current academic achievements. The player‘s aptitude, when measured by either SAT or ACT scores, explains between 17.0%-32.0% of the vari- ances in the variety, depth and endurance that was exhib- ited. A player‘s high school grade-point-average was statis- tically significant only in accounting for 15.0% of the vari- ance in the endurance they demonstrated over the game‘s run. Each player‘s current college grade-point-average fared better as a predictor of two engagement attributes. The coefficients of determination for variety and endurance ranged from 0.10-0.13. While this study found a degree of statistical signifi- cance there is the larger question of whether these correla- tions are strong enough to make a difference in either pre- dicting a player‘s engagement level or their performance in the game. With r-squares ranging from 0.17-0.32, or aver- aging 0.23, about 0.77% of the variance in their engage- ment attributes were not explained. Based on this assess- ment there appears to be other factors that determine a player‘s engagement level. Because of either the actual or the possible lack of engagement in group projects, many have suggested the use of peer evaluations as a way to obtain additional measures of an individual‘s efforts and more accountability from all the team‘s members (Leijk & Wyvill, 2001; Magin, 2001; Ryan, Marshall, Porter & Jaia, 2007; Wen & Tsai, 2006). This study examined this aspect by asking each player to rank order the economic and social contributions their teammates made over the game‘s run and then comparing those rankings with each player‘s true within-team engage- ment rank. The results presented in Exhibit 7 indicates there is a weak but significant relationship between an individual's perceived value to attending to the team's social needs and the individual's level of engagement. Moreover, there is a fairly strong relationship between the individual's perceived value to attending to the team's eco- nomic needs and the individual's level of engagement. Interestingly there is a nonsignificant relationship between a player‘s assessed economic versus social contribution. This result indicates the assessors were using different criteria to judge the value of social concerns EXHBIT 5 PERCENT CHANGE IN SCREENS VIEWED -100.0% -90.0% -80.0% -70.0% -60.0% -50.0% -40.0% -30.0% -20.0% -10.0% 2 3 4 5 6 Period P er ce n t C h an g e Engaged Unegaged Page 388 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 with economic concerns. This finding also is in conflict with the findings of Wolfe & Roberts (1993) where there were strong correlations between a player‘s assessed social contribution and that player‘s economic contribution. In that study, however, the players were near-graduation busi- ness school seniors. Perhaps after four years of business schooling game players and business school students in general, may recognize that being ―social‖ and ―economic‖ are twin and not separate virtues. CONCLUSION Group-related business games are a popular pedagogy in business schools. They help initiate students into the dynamics of group relations and group decision-making skills while providing a realistic context for participants. Unfortunately, oftentimes there are ―free riders‖ that afflict many group efforts. This serves to create imbalances be- tween participants not only in terms of learning but also in the quality of the learning outcomes. This research sought to understand the contributing factors that created engaged versus non-engaged group participants. In general, it ap- pears that some of the factors that contribute to an engaged participant are aptitude scores such as higher college GPA and SAT or ACT scores. However, this paper‘s research emphasis was not to advocate against using group-related projects as a business teaching methodology but to instead recognize the extent to which free-ridership can exist and that the instructor needs to be pro-active with measures to insure that true collective learning is possible. The simula- tion and group project used in this study were intended to initiate an experiential learning exercise of managerial con- cepts. As with all team-based work in academia as well as the work-place, there will be members more engaged in the project than others. This research provides a starting point in which to identify the relationship between engaged and unengaged team members. Other factors beyond this re- search offer opportunities for further predictive group par- ticipant engagement research. REFERENCES Albanese, R., & Van Fleet, D.D. (1985). Rational behavior in groups: The free-riding tendency. Academy of Man- agement Review, 10(2): 244-255. Anderson, P.H., & Lawton, L. (2004). The effectiveness of a simulation exercise for integrating problem-based learning in management education. In Developments in Business Simulation and Experiential Learning, 32: 10 -18 Ashraf, M. (2004. A critical look at the use of group pro- jects as a pedagogical tool. Journal of Education for Business, 79: 213-216. Becker, D., & Dwyer, M. (1998). The impact of student verbal/visual learning style preference on implement- ing groupware in the classroom Journal of Asynchro- nous Learning Networks, 22(2): 61-69. EXHIBIT 6 PREDICTIVE POWER OF SCHOLASTIC APTITUDE AND ACADEMIC ACHIEVEMENT Note: All significant p < 0.038. *Non-significant. EXHIBIT 7 ASSESSMENTS OF SOCIAL AND ECONOMIC CONTRIBUTIONS Engagement Dimension Predictor r² Aptitude HSGPA CGPA Variety 0.19 0.01* 0.10 Depth 0.17 0.01* 0.00* Endurance 0.32 0.15 0.13 Contribution r² Significance Social 0.09 p = 0.008 Economic 0.29 p < 0.000 Social/Economic 0.04 n.s. Page 389 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Brooks, C.L., & Ammons, J.L. (2003). Free riding in group projects and the effects of timing, frequency, and specificity of criteria in peer assessments. Journal of Education for Business, 78(5): 268-272. Cooper, B., & Foy, J.M. (1969). Students‘ study habits, attitudes and academic attainment. Higher Education Quarterly. 23(2): 203-212. Gatfield, T. (1999). Examining student satisfaction with group projects and peer assessment. Assessment and Evaluation in Higher Education, 24: 365-378. Jones, G.R. (1984). Task visibility, free riding, and shirk- ing: Explaining the effect of structure and technology on employee behaviors. Academy of Management Re- view, 9(4): 684-695. Karau, S. J. & Williams, K. D. (1993). Social loafing: A meta-analytic review and theoretical integration. Jour- nal of Personality and Social Psychology, 65(4): 681- 706 Kench, P.L., Field, N., Agudera, M., & Gill, M. (2009). Peer assessment of individual contributions to a group project: Student perceptions. Radiography, 15(2): 158- 165. Kerr, N.L. (1983). Motivation losses in small groups: A social dilemma analysis. Personality and Social Psy- chology, 45: 819-828. Koh, G.C.-H., Khoo, H.E., Wong, M.L., Koh, D. (2008). The Effects of Problem-based learning during medical school on physician competency: a systematic review. Canadian Medical Association Journal 178(1):34-41 Latane, B., Williams, K., & Harkins, S. (1979). Many Hands Make Light The Work: The Causes and Conse- quences of Social Loafing, Journal of Personality and Social Psychology, 37(6): 822-832. Lejk, M., & Wyvill, M. (2001). The effect of inclusion of self-assessment with peer assessment of a group pro- ject: A quantitative study of secret and agreed assess- ment. Assessment and Evaluation in Higher Education, 2(6): 551-561. Magin, D. J. (2001) A novel technique for comparing the reliability of multiple peer assessments with that of single teacher assessments of group process work, As- sessment and Evaluation in Higher Education, 26(2): 139–152. McArdle, G., Clements, K.D., & Hutchinson-Lendi, K. (2004). The Free Rider and Cooperative Learning Groups: Perspectives from Faculty Members. In Pro- ceedings, Academy of Human Resource Development International Conference, Estes Park, CO, pp. 529-535. McElhinney, J., & Murk, P.J. (1994). Using small learning groups in graduation education. Paper presented, American Association for Adult and Continuing Edu- cation, Nashville TN. Mello, J.A. (1993). Improving individual member account- ability in small group settings, Journal of Management Education, 17(2): 253–259. Murphy, S.M., Wane, S.J., Liden, R.C. & Erdogan, B. (2003). Understanding social loafing: The role of jus- tice perceptions and exchange relationships. Human Relations, 56(1): 61-84 Peterson, S.E., & Miller, J.A. (2004). Comparing the qual- ity of students‘ experiences during cooperative learn- ing and large-group instruction. Journal of Educa- tional Research, 97(3): 123-133. Piezon, S. and Donaldson, R. (2005). Social loafing and free riding in online learning groups. Online Journal of Distance Learning Administration, 9, 6. Accessed August 2010 from http://www.westga.edu/~distance/ ojdla/winter84/piezon84.htm. Ringelmann, M. (1913). Recherches sur les moteurs animés: Travail de l'homme. Annales de l'Institut Na- tional Agronomique, 12(1): 1-40. Ryan, G. J., Marshall, L. L., Porter, K., & Jia, H. (2007). Peer, professor and self-evaluation of class participa- tion. Active Learning in Higher Education: The Jour- nal of the Institute for Learning and Teaching, 8(1): 49 -61. Schoenecker, T.S., Martell, K.D., & Michlitch, J.F. (1997). Diversity, performance, and satisfaction in student group projects: An empirical study. Research in Higher Education, 38(4): 479-495. Struyven, K., Dochy, F., & Janssens, S. (2008). The effects of hands-on experience on students' preferences for assessment methods. Journal of Teacher Education, 59 (1): 69-88. Thompson, M.E. (1976). The prediction of academic achievement by a British study habits inventory. Re- search in Higher Education. 5(4): 365-372. Wen, M. L., & Tsai, C. (2006). University students' percep- tions of and attitudes toward (Online) peer assessment. Higher Education: The International Journal of Higher Education and Educational Planning, 51(1): 27-44. Wolfe, J., & Roberts, C. (1993). A further study of the ex- ternal validity of business games: Five-year peer group indicators. Simulation & Gaming, 24(1): 21-33. Table of Contents Volume 38, 2011 Simulated Tabletop Exercise for Risk Management - Anti Bio-terrorism Scenario Simulated Tabletop Exercise From Business Games to Simulations - Simuworlds & Microworlds Demand Equation Redux: The Design and Functionality of the Gold/Pray Model in Computerized Business Simulations Managing Client-Based Learning: Insights from Successful Teaching Project Courses in Marketing Tracking Forecast Error Type, Frequency and Magnitude with the Forecast Error Package Responding to Facilitate Collaboration Simulating Sudden Change and the Value of Timely Information Managing Human Resources Simulation A Study on Collectivism and Group Decision-Making: An International Comparison of Japan, China, and Russia Using a Gaming Simulation The Use of Management Games in the Management Research Agenda Gaming On-Line: A Simulation Application Positioning and Performance in Simulated Networks Supply Chain Management: A Simulation Application Simulation as a Teaching Method in Strategic Management Distance Studies Entrepreneurship: A Game of Risk and Reward Phase II: The Start-Up Return to the Paradise Islands: From Confrontation to Cooperation Effect on Market Performance of Displaying Supply and Demand Curves in a Business Simulation Appreciating Complexity: The Chief of Staff of the Army Game Managing Organizations: Experiential MBA Course Teaches Alternatives to the Machine Model A Simulation Model for Analyzing the Night-Time Emergency Health Care System in Japan The Continuing Evoluation of Assessing Project Management as an Academic Learning Outcome (ALO) Should College Instructors Change Their Teaching Styles to Meet the Millenial Student? The Mouse Game and its Effects on Team Interdependence Learning from the Gulf Oil Spill to Prepare for a Brighter Future: A New Game Engaging Stake Holders in Triple Bottom Line Accounting & Strategic Planning Video Killed the Biblio Star: The Impact of Digital Media on Student Learning Outcomes Exploring Motivation: Using Emoticons to Map Student Motivation in a Business Game Exercise An Alternative to PC and Internet Based Simulations: The Internet Integrated Mode MiddleState University -- A Crisis in Education Complexity Avoidance, Narcissism and Experiential Learning Examining the Cognitive, Affective, and Psychomotor Dimensions in Management Skill Development Through Experiential Learning: Developing a Framework A Situational Leadership Exercise Based on the Biology of a Starfish JOGAI CEFET -- The Industrial Administration Undergraduate Game Would You Take a Marketing Man to a Quick Service Restaurant? Modeling Corporate Social Responsibility In A Food Service Menu-Management Simulation Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met A Team Based Information Literacy Exercise ABSEL Marketing Communications Plan An Interdisciplinary Study of the Impact of Playing a Marketing Simulation Game on Student Knowledge of Management Accounting/Finance Principles Analyzing Construction Planning of Interiro Finish Work of Apartment Building by Simulation Doing Murder One Again The Simple Business Game and Simulation Transfering the Knowledge of Middle Management to Novices Infectious Disease Simulation Model for Estimation of Spreading Understanding the Relative Influence of Several Factors in ERP Simulation Performance: An Exploration of Ecological Validity Tragedy of the Commons: An Exercise Using Clickers to Illustrate and Teach a Key Concept in Negotiations The Meaning of Firm Demand in Business Simulations If the Games Work, Why Aren't More Faculty Willing to Play? Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players