The Use of Management Games in the Management Research Agenda Page 83 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 ABSTRACT The main objective of this manuscript is to theoretically discuss the pros and cons related to the use of management games in basic empirical research and to help answer the following question “can games be used in the same way as … wind tunnels for testing designs and strategies?” (Bass 1964: 546). This manuscript is structured in three parts that will basically (a) define the term „management games‟ as a laboratory research instrument; (b) analyze realism and validity issues, with an intent to identify the benefits of using games for research purposes; and (c) present a sum- mary of the main issues covered. INTRODUCTION Of the multitude of fundamental problems that come to mind to those who do research in the field of organization/ management is that of the scientific methodology that is most appropriate to use given a certain organizational phe- nomenon. According to Harrison, Lin, Carroll & Carley (2007: 1230) there are two options for researchers: (a) a theoretical analysis or deduction or (b) an empirical analy- sis or induction. With theoretical analysis or deduction, research con- clusions are generally an expectation and are dependable of further empirical analyses. On the other hand, with empiri- cal analysis the following problem is faced: how does a researcher access data from real firms to empirically test theories? If data is available then the difficulty is deciding which empirical methodology is the most appropriate to analyze the information (Schwenk 1982). Saunders & Thompson (1980: 125) refer to five sources of data for em- pirical research in the field of management (a) interviews, (b) company reports, (c) government documents, (d) ques- tionnaires, (e) laboratory experiment and (f) observation. In an attempt to help researchers select from these options, Keys & Wolf (1990: 323) refer to McGrath (1982) who suggests making the decision based on three broad dimen- sions: “the ability to generalize from the sample to the population, the control and precision with which to evalu- ate the behaviors, and the realism of the setting in which the actors behave”. To study strategic decision-making, a specific area of strategic management, we could argue that, ideally, data ought to be directly measured by interviewing „in loco‟ and „on line‟ those responsible for planning strategies - top managers. This „ideal‟ situation rarely occurs as Nees (1983: 176) illustrates in her study about divestment. She writes, “…divestors in Europe were reluctant to invite re- searchers on board, making the access to field data very difficult.” When company reports and government docu- ments are considered, the problems are related to the qual- ity of information provided (whether the information is reliable) and the quantity of qualitative information (Harrison et al. 2007). Another choice is the questionnaire survey which has several strengths and weaknesses. For THE USE OF MANAGEMENT GAMES IN THE MANAGEMENT RESEARCH AGENDA José Henrique Dieguez-Barreiro Catholic University of Brasilia (Brazil) joseh@ucb.br henrique.barreiro@gmail.com Javier González-Benito University of Salamanca (Spain) javiergb@usal.es Jesús Galende University of Salamanca (Spain) jgalende@usal.es Edson Kenji Kondo Catholic University of Brasilia (Brazil) kondo@ucb.br mailto:joseh@ucb.br mailto:henrique.barreiro@gmail.com mailto:javiergb@usal.es mailto:jgalende@usal.es mailto:kondo@ucb.br Page 84 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 instance, decisions must be made regarding „what to ask‟ and „to whom to send‟ the survey to. The last option for empirical research listed by Saunders & Thompson (1980) is the laboratory experiment. According to Keys & Wolfe (1990), one of the main problems is the difficulty to repro- duce the real world in an artificial manner. That is, there is a „trade off‟ among all three dimensions, as suggested by McGrath (1982). But what exactly is a laboratory experiment? We could define laboratory research simply as an experiment in which a certain environmental condition is recreated in an artificial manner, and where a phenomenon can be ob- served, studied, and easily repeated. By laboratory re- search we can observe the use of limited replicas of real organizational conditions, created basically by three means: (a) games, not necessarily made by computer, created to investigate individual and team issues (e.g. the Prisoner‟s Dilemma Game as described by Schlenker & Bonoma, 1978); (b) case studies; and (c) management games. This study hypothesizes that management games can be used as a means to carry out laboratory experiments in the field of management. Cohen & Rhenman (1961: 158), for instance, stated that: “The success of natural sciences like physics, chemistry, and biology in the use of laboratory experimentation has always been a challenge to social scientists. But the equipment of the latter for performing laboratory experiments has been meager, and this has often been thought to be one of the major reasons preventing more rapid pro- gress in the social sciences.” We believe that part of the equipment referred to by Cohen & Rhenman (1961) for the social scientist is already available, albeit misunderstood. As mentioned above, the equipment we refer to is management games. The reason we believe management games are a useful research instrument is related to the enhanced level of accu- racy, accessibility, and complexity that business games have nowadays. This is the result of years of development since the 50‟s as well as advances in technology. The evo- lution of computational and software systems has trans- formed the technology of simulations used in management games. This transformation has led to a set of maneuver- able programming tools for social scientists. In the follow- ing sections we analyze management games as instruments for research on management/organizational issues by pre- senting their advantages and disadvantages and providing suggestions for their use in research. MANAGEMENT GAMES THE CONCEPT OF MANAGEMENT GAMES We begin by proposing a definition for the term „management games‟. In the literature terms that are used include (a) management simulations; (b) business simula- tions; (c) business games; (d) game simulation; and (e) game or simulation. We define management games as did Keys & Wolfe (1990), two experienced authors on the de- sign and use of management games. The term management games encompass at least two central concepts discussed below. Simulation For a definition of simulation Nees (1983: 176), re- ferred to Abelson (1968) who asserted that simulation is “the exercise of a flexible imitation of process and out- comes for the purpose of clarifying or explaining the under- lying mechanisms involved.” Keys & Wolfe (1990: 308) provide us with another definition “a simulated experiential environment is a simplified and contrived situation that contains enough verisimilitude or illusion of reality to in- duce real world–like responses by those participating in the exercise.” A management game, in simulation terms, is a simpli- fied replication of a business observed reality. In other words, it is a „relaxed‟ reality, as reality is represented in a simplified manner, despite simulations being designed upon well known theoretical foundations. As an example we cite economic theory (e.g. microeconomics) which helps demand modeling in business simulations (Gold & Pray 1990: 119). As Nees (1983) observed, “the objective of a management game is not to duplicate reality „in vitro‟ but to create and observe a system that complies to the same behavioural pattern” (p. 176). In sum, management games use simulation techniques to replicate the economic, and/or industrial environment (Cohen & Rhenman 1961). At this time, it is important to highlight the difference between management games and computational modeling. Both rely on mathematical and computational simulation to achieve their objectives. Management games use simula- tion to artificially create a business environment to which „real‟ subjects (players) will be exposed and their behaviors will be observed. Computational modeling, on the other hand, uses simulation to generate models which try to ex- plain the relations between firm data inputs and outputs, in the absence of subjects (players) except the researcher. As Harrison et al. (2007) claim (the insertion is ours), “while simulation can be distinguished from deduction and induc- tion (recognized methods to do science), it does have simi- larities to these other methods” (p. 1230). Game „Game‟ is the other concept linked with management games. This could be easily observed when we include the subjects (players) in the simulation. Keys & Wolf (1990: 308) made this link between simulation for the users and their behavior stating that “management games are used to create experiential environments within which learning and behavioral changes can occur and in which managerial be- havior can be observed.” Babb, Leslie & Van Slyke (1966) were more specific and related management games to a Page 85 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 complex context where players compete among each other. In their words, “business games are decision–making exer- cises in which teams compete in satisfying specified objec- tives … players make sequential management-type deci- sions which affect their current and future positions” (p. 466). In a similar way Larréché (1987: 559) defines game as “… a tool that allows individuals to use and develop their decision-making skills in a fictitious competitive envi- ronment.” Management games, in „players language‟, is a place where players (subjects) can express their behaviors and exercise their skills in a competition, making sequential decisions, individually or in teams, and where the decisions of each individual or group affect simultaneously their re- sults and those of others. An additional difference between „simulation‟ and „game‟ is related to game theory and management games. Game theory is considered an important instrument to in- vestigate the conflict between cooperation and defection (particularly in the case of the prisoner's dilemma). Cohen & Rhenman (1961), for example, argue that the mathemati- cal approach of game theory helps clarify concepts such as strategy, coalition, game value, and game solution. At the same time, they believe that management games are more effective than game theory by stating that “… game theory offers very little for the analysis and nothing for the solu- tion of the very complex situations involved in many busi- ness games” (p. 134). Babb et al. (1966) state that manage- ment games have a different approach from game theory. In their words game theory: “… is normative or prescribes how „rational‟ peo- ple ought to behave under specified conditions … these games are generally characterized by deci- sion making on only one variable and by players knowing in advance the „payoff matrix‟ or possi- ble results of specified decisions … [and conclude that] … by comparison, these experimental games are not nearly so complex or comprehensive as business games (p. 466).” Management games can also be analyzed by the following dimensions: (a) Scope. Keys & Wolfe (1990: 308) classify manage- ment games as Total Enterprise Games and Functional Business Games. They define the former as “simulations that deal with the entire organization, provide a balanced number of decisions variables in marketing, production and finance, and thus require the strategic integration of several subunits for organizational performance … (Horn 1977; Keys 1987),” and the latter as “simulations that concen- trate on a single subunit of the firm.” (Keys & Wolfe, 1990: 309). (b) Role of the game administrator. The game adminis- trator plays an important role by fine tuning the conditions of the game environment, and by encouraging subjects through different types of incentives (Larréché 1987). (c) Level of information provided to player. By chang- ing the level of information available to players, the game administrator affects the level of uncertainty allowing play- ers to experience situations that might take decades to ex- perience in real life (Babb et al. 1966; Nees 1983; Keys & Wolfe 1990; and Hambrick 2007). (d) Fast feedback of the consequences of decisions made. Players are quickly provided with the results and feedback about their decision-making, normally a few min- utes after conveying the decision to the game administrator. (e) Interdependence among team (players) decisions. In the „real world‟ the decisions of major firms have some degree of interdependence with each other, depending on the level of concentration of the industry, and jointly affect the firm and industry results. This feature allows the game administrator to define different levels of interdependence among firm and industry-related variables. (f) Sequential decision making and the longitudinal aspect of decision making. This is useful in studying a phenomenon for which time is an important variable to be considered. Since decision makers are exposed to sequen- tial decision-making, their decisions and their results can be followed over time. This facilitates, for example, experi- ments where the environment could be modified, during a certain period of time by some circumstances. In this case, results can be compared to ulterior and posterior conditions and results. (g) Existence of decisions influencing immediate or future results. This issue leads us to more „real world‟ situations where in the present and future, for instance, advertising efforts influence firm and industry demand. A researcher could include in his/her research design, for ex- ample, disturbance effects over player decisions and re- sults. To conclude this section, we are reminded of Biggs (1990: 24) who observed “in computerized business games, game players (participants, students) assume the role of decision-makers in organizations.” In our understanding, it is important to highlight the role of the game administrator in facilitating the process of experiencing the complexity of the reality of firms, and the role of computers, in generating faster and more reliable results, if compared to games with- out computer assistance. THE ROLE OF MANAGEMENT GAMES ON RE- SEARCH The primary objective of management games when they were first created in the late 1950s was for educational purposes. As part of this objective, researchers who work in this field state that the main goal of this methodology was to produce a „dynamic environment‟ through the use of computer programming. They believe that this provides a perfect environment to exercise „complex strategic man- agement‟. Over this „dynamic‟ and „complex‟ stream there Page 86 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 is a secondary objective identified by researchers who used it: to employ the management game as a research instru- ment (Cohen & Rhenman 1961). The Dickinson, Gentry & Burns (2004) study identified research related to manage- ment which used business games to acquire data for empiri- cal tests, the earliest of which was conducted by Cangelosi & Dill (1965). More recently we have identified two other contributory articles: one by Mathiew & Schulze (2006) where they used a business simulation to test team process- performance relationships, and the other, theoretical in es- sence, written by Hambrick (2007) which advocates that management games could be an important research instru- ment to gather data and build on the „upper echelons the- ory‟. According to Keys & Wolfe (1990: 307) “business games arrived on the scene in the late 1950s, spawned by the fusion of developments in war games, operations re- search, computer technology, and education theory.” It is possible that this apparently chaotic and complex genesis affected the perceived value attributed by researchers to the use of management games as a research tool. To illustrate this we refer to Hambrick (2007) who confessed with great honesty that he had been considering the use of manage- ment games to clarify the „upper-echelons theory‟, but had “… been intimidated by the technical challenges of design- ing the simulation (management game)” (p. 338) (the inser- tion is ours). In addition, several issues have risen in the literature questioning the use of management games for research purposes. The main problem seems to be that because a game does not provide a real-life firm environ- ment, it would yield little improvement in practice (Dickinson, Gentry & Burns 2004; Keys & Wolfe 1990 only to list the more recent ones). On the other hand, other authors consider management games as an important in- strument for social research, like Schlenker & Bonoma (1978) who argued: “… [management] games could serve as a skeletal analogy of many social situations and contexts. In constructing a game analogy, an attempt is made to dissect from the complexities of real social in- teractions some fundamental structural aspects that can be employed to facilitate our understand- ing of the actual situations (p. 09).” REALISM AND VALIDITY One of the most questionable aspects of business simu- lations is the generalizability of the results gathered from using this research method. There are two consistent prob- lems in this respect: experiment realism and experiment validity. EXPERIMENT REALISM Keys & Wolf (1990: 324) through arguments made in prior external research (Lant 1989; McGrath 1982) state that “business simulations have often provided a realistic group decision-making context, but not a realistic organiza- tional context.” This dichotomy of realism could be a problem when we try to generalize the findings made in laboratory research. To the list of disadvantages we could add (a) firm/industry conditions that are not consistent with real life; (b) that laboratory experiments are artificial and their results are not representative of the real world; and (c) that laboratory experiments are not as adequate field re- search in identifying and defining variables (Schwenk 1982). Extending the discussion, Gentry et al. (1984) who, referring to Aronson & Carlsmith (1968), delineate two types of realism: (a) mundane realism which refers to how likely the experiment would occur in the real world; and (b) experimental realism which refers to the degree to which the subjects (players) who are involved in laboratory re- search take the experiment seriously. Likewise, Dickinson, Gentry & Burns (2004) assert that in a game there is “limited mundane realism, i.e., face validity” (p. 346). Cohen & Rhenman (1961) warned about some short- comings related to the level of reality of management games. According to these authors, games do not include all the challenges that managers find in a real business such as personnel, psychological, and organizational problems. Furthermore, the easy (i.e. cost free and quick) way with which the players receive the information (mainly gener- ated by the computer) keep them oblivious of how difficult it really is in the real world. A final warning is that players might mistakenly “feel … that they really know how to run a business as a result of their experience in playing man- agement games” (Cohen & Rhenman 1961: 152). In our experience with management games these are still valid statements today in 2010. Another aspect of realism is related to game complex- ity. We argue that the more a game follows a real world appearance, the higher the number of variables, which can increase exponentially. Consequently, the complexity of the game (including mathematical modeling and computer programming) is directly proportional. This complexity could affect two other important practical questions when considering laboratory research: (a) the time available to do the experiment, which could be divided into two: first, the available time for class meetings; and second, the time available for the subjects (players) to perform the decision making and supplemen- tary tasks as required by the research question; and (b) the capacity of the subject (players) to manipulate information created in the „complex‟ situation on which the firm and industry environment were designed. The main problem of the realism issue could be de- fined as a game/research designer‟s paradigm: to find equi- librium between game complexity-reality and research pro- posals. The problem to be addressed by the game/research designer concerns the adequate limits of subjects (players) Page 87 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 to manipulate/process information and the time available to perform the experiment to guarantee the closest reproduc- tion of the real situation faced by managers. This problem of „equilibrium‟ is linked with the game „designer‟s dilemma‟ (Teach 1990) which states that there are three aspects which a game designer must deal with: (a) the true simulation, the complexity to represent indus- trial systems in mathematical formulae; (b) the game, a set of rules that govern the game, the level of acceptance among players and the limits imposed on players; and (c) the context, the competition the game evokes among play- ers or between players and nature. It is reasonable to assume that the complexity of simu- lating an environment is limited to two aspects: (a) know- ing perfectly the firm and industry environmental models, the variables involved in both environments and the interre- lations among these same variables; and (b) developing the game on paper (as a classic game based on table, cards etc.) or even in a computational way (programming software and capacity to process it on a hardware). This second as- pect is no longer as challenging a problem since computa- tional technology has become very accessible. As observed by Cohen & Rheman (1961) more than four decades ago, “the use of computers has provided an opportunity for the designers of games to incorporate in them a great deal of realistic complexity while still keeping their administration relatively simple” (p. 134). Games which are played on computers permit an in- creasing number of variables in more complex relations (Biggs 1990; Keys & Biggs 1990) to be processed quickly, thus reducing the time spent by the game administrator on data input, processing and releasing the results and infor- mation to the subjects. Keeping the realism in mind, Cohen & Rhenman (1961: 134) alert us to the advantage of using computers in game play stating that, “an electronic computer also adds considerably to the drama of game play.” The same authors also reinforce the ease that com- puters provide to use stochastic or random variables, which add more reality to the management game. Some characteristics that we think are relevant to con- sider in an analysis of the level of reality of a management game for research purposes are that (a) it provides interrela- tions between functional areas; (b) it recreates a similar dynamic situation found in real life; (c) it provides some level of risk and uncertainty; (d) it provides a systematic collection of information, for the players, and the game administrator-lecturer-researcher; (e) it provides opportu- nities for players to learn and reinforce a variety of analyti- cal tools in a sequence of events (dynamic environment); (f) it provides a place where organizational problems (at least some of them) could be illustrated; and (g) it could demonstrate the value of planning and policy-making. (Cohen & Rhenman 1961; Biggs 1990; Keys & Biggs 1990). Finally, in a comparison of other laboratory experi- ments using management games, and highlighting the level of reality provided by the latter, Gentry et al. (1984), states: “… it should be remembered, though, that one of the advantages of the simulation game over the laboratory experiment is its increased realism. „Realism‟ can be viewed as a continuum, and just how much more closely the simulation is to the field study than the laboratory experiments de- pends upon the nature of the game itself and the manner in which it is administered” (p. 2). This statement confirms the importance of the game administrator who has an important role in creating and maintaining the level of realism and dynamism in a game. This is also highlighted in the follow section. EXPERIMENT VALIDITY When examining the concept of validity, two impor- tant dimensions are internal and external validity. Other dimensions are also discussed in the literature such as face validity (Keys & Wolfe 1990) and ecological validity (Schlenker & Bonoma 1978: 23), however in our research, we have chosen to focus on the first two. According to Schlenker & Bonoma (1978: 22) internal validity: “… refers to whether an effect produced in a study resulted from the experimental manipulations or whether the effect might have been coincidentally produced by such factors as subject manipulation, history, prior testing, or any of the host of artifacts to which the experimental endeavor is prone.” In other words, internal validity is concerned with the possible negative influence that laboratory simplification of reality and subject manipulation could create in defining and correlating variables. This influence could create arti- ficial (and unreal) measures or relationships among vari- ables resulting in a lack of internal validity. On the other hand, the laboratory experiment can be used to confirm (or refute) variables previously investigated on the field. (Schwenk 1982). External validity, according to Schlenker & Bonoma (1978: 22) refers to “the ability to generalize the findings obtained from an experiment toward (a) different subject populations, (b) different ways of measuring the same variables, and (c) different situations and settings.” In other words, external validity is concerned with how an experiment could be replicable in another experiment with different sets (i.e. subjects and place) and ultimately, how it replicates the real world ipsis litteris. Whether in the field or in the laboratory, the main problem with research is related to validation in that (a) is the identification and description of variables reliable? (b) is the interdependence found among these variables reli- able? and (c) are there any other experiment and researcher biases? Page 88 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 From another perspective, we could analyze this prob- lem evoking the „control‟ introduced over the variables in experiments. If we „control‟ an experiment, in the field or in a laboratory, by omitting a confounding variable in an intentional manner, we ensure a better internal validity. But if we manipulate the internal validity we influence the external validity in the sense that the former is a prerequi- site of the latter (Schwenk 1982). The lack of external va- lidity leads to a lack of generalizability. In another situa- tion, if we do not „control‟ the experiment but omit an im- portant variable in the experiment we are lacking external validity. Schwenk (1982) states that researcher biases may occur when “the researcher makes his own guesses about the relationship between critical variables” (p. 215). An important conclusion made by Schwenk (1982) illustrates this problem of external validity on field experiments: “… it has been claimed that field research of the sort advocated by Mintzberg (1977) and others has at least two major weaknesses. First, that it is dif- ficult to control confounding variables in the field settings and second, that the results may be col- oured by experimenter bias” (p. 215). On the other hand, some advantages of management games, as laboratory research, were stated by Key & Wolf (1990: 323): “… (a) simulation provides more precise measure- ments of behaviour than field research because decisions are made in a closed organization/ environment system, and the similar decision re- sponses are made repeatedly over time.” and…” (b) further, the environment, though com- plex and realistic, is a known entity to the re- searcher. Thus the causal relationships between the organization and environment can be deter- mined by the researcher in a way that is impossi- ble in field research (Lant, 1989; Lant & Mont- gomery, 1989; McGrath, 1982).” CONCLUSIONS ABOUT REALISM AND VALIDITY An important conclusion in the discussion of realism and validity is that management games have an increased degree of firm-reality when compared to other kinds of laboratory experiments. The complexity of the game deter- mines the level of reality in the experiment. The level of validity can be strengthened by the variables chosen by the researcher and the model developed for the study. This may be aligned with an organization‟s environment to the degree desirable for a particular study. Taking into ac- count the latter arguments, and evoking Schwenk (1982), why are we still sacrificing laboratory research by using field research? CONCLUSIONS FIELD VERSUS LABORATORY RESEARCH To choose between field research and laboratory re- search is not a decision without challenges. It is hardly preferable to conduct research in the field, where the reality of a firm is available without any fortuitous interference by the researcher or any other artificial laboratory conditions. It is known that some conditions of the firms are difficult, impossible or eventually dangerous to be accessed. Infer- ence from variables studied in the field may produce prob- lems related to the validity of an experiment. The main point here is that the objective of the research and the ade- quacy to the kind of experiment should be aligned. For instance, a research objective that considers direct access to the TMT (top management teams) of firms, visualizing a sequence of strategic decision made over a period of time (once or twice a year), poses enormous challenges to carry out in the real world, due to the difficulty of obtaining ac- cess to such people. But the main limitation of only ana- lyzing reality as it has turned out is the inability to get in- sights from alternative realities that might have happened. Our first conclusion comes in line with Schwenk (1982) and Nees (1983) who state that management games are not adequate in exploratory studies, where variables must be identified and described. These authors also agree that management games could be used together with field research, in a sense that (Nees 1983: 182) “once developed into testable format, these hypotheses could be subjected to a „laboratory‟ experiment where the independent and de- pendent variable are closely controlled and then manipu- lated.” Schwenk (1982: 214) states, “the laboratory re- search helps to refine the researcher‟s understanding of the nature of and the relationship between the variables.” In other words, once the variables are identified and well de- fined, laboratory research, with a special feature of experi- ment control, could provide other important and useful evidences on the interrelationships among the variables. A second conclusion is that, on some occasions, games might be the „only option‟, perhaps with issues related to the behavior and work of TMT. Studies which attempt to understand the nature and the essence of the decision- making process must access the decision makers in close proximity, that is, in their natural state. Management games could provide a complex environment, enough to be appreciated by the subjects and considered a „very-near- real‟ experience. Despite this, some other issues must be considered, for example, (a) the subjects used; (b) the time available for an experiment; (c) the costs related to the process; (d) time and cost of software/simulator develop- ment; and (e) the time needed by the research designers and subjects. FINAL REMARKS Page 89 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 Cohen & Rhenman (1961), in an attempt to illustrate the usefulness of laboratory experiments in social research argue that: “… when collecting empirical data to test his fun- damental theories, the scientist accepts the artifi- cial test tube experiment. But when he wants to test, e.g., a complex production process, these simple laboratory experiments are not regarded as reliable. He knows that work in a laboratory might cause him considerable trouble in the full scale plant. This is why he wants to test the process in a pilot plant designed to make experimentation pos- sible… considering its size, cost, and purpose, a laboratory for experimental games … really is a pilot plant test station. But even with this limita- tion, organizational „pilot plant test‟ should be very valuable. A simple test which shows that an organization works in a tolerable way provides valuable knowledge” (p. 164). But one conclusion provided by Festinger (1959: 10) is important to consider: that “a laboratory experiment need not, and should not, be an attempt to duplicate a real life situation.” Along a similar line, for Gentry et al. (1984, p. 01), a management game is a middle range between field and laboratory research. They also state that “in general, the hope is that simulation games can allow sufficient con- trol so as to ensure internal validity while at the same time being sufficiently realistic so as to have some external va- lidity.” Bass (1964) argues that a management game “is not the tool with which to test specific individual cognitive processes, one-by-one, any more than a pilot plant is usu- ally necessary to test a specific chemical reaction, or a wind tunnel is necessary to test the tensile strength of a particular alloy” (p. 546). Bass (1964) likewise believes that manage- ment games are a recommended experimental procedure to examine questions related to “organizational mix, particu- larly of real men, processes and materials as they inter- act” (p. 546). Babb et. al. (1966) claim that “some real-life compari- sons may be necessary to validate findings based on gam- ing experiments” (p. 468) but “the gaming method may even provide further empirical evidence on the theoretical issues of the controversy” (p. 469). The authors conclude that “management games become a desirable device for obtaining research data which would not be possible using conventional techniques” (p. 472). These authors express concern on the subject choices, “the objectives of the ex- periment should be considered in the selection of sub- jects” (p. 471). For those who intend to design a manage- ment game, Larréché (1987) warns that (the insertion is ours) “the development of the simulation should be driven by the theoretical knowledge of market and competitive mechanism and not by the pedagogical (or research) con- cepts it is designed to illustrate.” (p. 564). For him, a man- agement game needs to “exhibit both theoretical validity (coherence with existing body of knowledge) and behav- ioral validity (coherence with the behavior in the real world)” (p. 565). Similarly, Keys & Wolfe (1990), Schwenk (1982) and Nees (1983) strongly believe that laboratory research is most effective when used in combination with field re- search. To illustrate this point, Camerer (1985: 06) argues that “seeing models as intermediate steps in ongoing model -building makes it clear that the realism of today‟s model is relatively unimportant, and blatantly unrealistic models may be better „building blocks‟ than realistic inductive frameworks.” As such, we believe that management games can be used as an experimental research tool and might be an ade- quate testing ground for empirical results generated by other kinds of research instruments. For many authors, management games are a valid instrument to contrast theo- ries, if the experiment and the games are carefully de- signed. However, the virtuous cycle of laboratory and field research proposed by Schwenk (1982) needs to exist in order to reinforce the findings and guarantee an increase in the body of knowledge. As Jemison (1981) observed, the “managerial environment … is inherently more complex than the degrees of freedom available to the researcher” (p. 640), and that is a strong reason in favor of using both methods in combination. To conclude, we provide the following recommenda- tion, “When no simple experiment with all-but-one variable held constant will provide the answers we seek, it will be profitable to simulate the organization” (Bass, 1964, p. 547). REFERENCES Abelson, R.P. (1968). Simulation of Social Behavior. In Lindzey, G., & E. Aronson, Handbook of Social Psy- chology, Reading, PA: Addison-Wesley. Aronson, E. & Carlsmith, J.M. (1968). Experimentation in Social Psychology. In Lindsey, G., & Aronson, E. (Eds.), Handbook of Social Psychology, Reading, PA: Addison-Wesley. Babb, E.M., Leslie, M.A. & Van Slyke, M.D. (1966). The Potential of Business-Gaming Methods in Research. Journal of Business, 39, 465-472. Bass, B.M. (1964). Business Gaming for Organizational Research. Management Science, 10, 545-556. Biggs, W. (1990). Introduction to Computerized Business Management Simulations. In Gentry, F.W. (Ed.) Guide to Business Gaming and Experiential Learning, AB- SEL, Nichols/GP Publishing, East Brunswick/Kogan Page, London. Camerer, C. (1985). Redirecting Research in Business Pol- icy and Strategy. Strategic Management Journal, 6, 1- 15. Cangelosi, V.E. & Dill, W.R. (1965). Organizational Learning: Observation Toward a Theory. Administra- tive Science Quarterly, 10, 175-203. Page 90 - Developments in Business Simulation and Experiential Learning, Volume 38, 2011 Cohen, K.J. & Rhenman, E. (1961). The Role of Manage- ment Games in Education and Research. Management Science, 7, 131-166. Dickinson, J.R., Gentry, J.W. & Burns, A.C. (2004). A Seminal Inventory of Basic Research Using Business Simulation Games. Development in Business Simula- tion and Experiential Learning, 31, 345-351. Festinger, L. (1953). Laboratory Experiments. In Festinger, L. & Katz, D. (Eds.) Research Methods in Behavioral Sciences, Holt, Rinehart & Winston, Ney York, 1953, 136-172. Gentry, J.W., Tice, T.F., Robertson, C.J. & Gentry, M.J. (1984). Simulation Gaming as a Means of Researching Substantive Issues: Another Look. Developments in Business Simulation & Experiential Exercises, 11, 1-5. Gold, S.C. & Pray, T.F. (1990). Modeling Demand in Computerized Business Simulation. In Gentry, F.W. (Ed.) Guide to Business Gaming and Experiential Learning, ABSEL, Nichols/GP Publishing, East Brunswick/Kogan Page, London. Hambrick, D.C. (2007). Upper Echelons Theory: An Up- date. Academy of Management Review, 32, 334-343. Harrison, J., Lin, Z., Carroll, G. & Carley, K. (2007). Simu- lation Modelling in Organizational and Management Research. Academy of Management Review, 32, 1229- 1245. Horn, R.E. (1977). The Guide to Simulation and Games. Cranford, NJ: Didactic Systems, Inc. Jemison, D.B. (1981). The Contribution of Administrative Behavior to Strategic Management. Academy of Man- agement Review, 6, 633-642. Keys, J.B. (1987). Total Enterprise Business Games. Simu- lation & Games, 18, 225-241. Keys, J.B. & Biggs, W.D. (1990). A Review of Business Games. In Gentry, F.W. (Ed.) Guide to Business Gam- ing and Experiential Learning, ABSEL, Nichols/GP Publishing, East Brunswick/Kogan Page, London. Keys, B. & Wolfe, J. (1990). The Role of Management Games and Simulations in Education and Research. Journal of Management, 16, 307-336. Lant, T. & Montgomery, D.B. (1987). Learning from Stra- tegic Success and Failure. Journal of Business Re- search, 15, 503-517. Lant, T.K. (1989). Simulation Games: A Research Method for Studying Organizational Behavior. Unpublished Manuscript, New York University. Lant, T.K. & Montgomery, D.B. (1989). A Note on MARKSTRAT as a Strategy Research Environment. Unpublished Manuscript, New York University. Larréché, J.C. (1987). On Simulations in Business Educa- tion and Research. Journal of Business Research, 15, 559-571. Mathieu, J.L. & Schulze, W. (2006). The Influence of Team Knowledge and Formal Plans on Episodic Team Process-Performance Relations. Academy of Manage- ment Journal, 49, 605-619. McGrath, J.E. (1982). Dilemmatics: The Study of Research Choices and Dilemmas. In J.E. McGrath, J. Martin & R.A. Kulka (Eds.), Judgment calls in research: 68-102. Bervely Hills, CA: Sage Publications. Mintzberg, H. (1977). Policy as a Field of Management Theory. Academy of Management Review, 2, 88-103. Nees, D.B. (1983). Simulation: A Complementary Method for Research on Strategic Decision-Making Processes. Strategic Management Journal, 4, 175-185. Saunders, C.B. & Thompson, J.C. (1980). A Survey of the Current State of Business Policy Research. Strategic Management Journal, 1, 119-130. Schwenk, C. (1982). Why Sacrifice Rigour for Relevance? A Proposal for Combining Laboratory and Field Re- search in Strategic Management. Strategic Manage- ment Journal, 3, 213-225. Schlenker, B.R. & Bonoma, T.V. (1978). Fun and Games – The Validity of Games for Study of Conflict. Journal of Conflict Resolution, 22, 7-38. Teach, R.D. (1990). Designing Business Simulations. In Gentry, F.W. (Ed.) Guide to Business Gaming and Experiential Learning, ABSEL, Nichols/GP Publish- ing, East Brunswick/Kogan Page, London. Teisch, H.P. (1964). The Validity of a Business Game. Un- published Master‟s Thesis, University of Oregon, Eugene. Table of Contents Volume 38, 2011 Simulated Tabletop Exercise for Risk Management - Anti Bio-terrorism Scenario Simulated Tabletop Exercise From Business Games to Simulations - Simuworlds & Microworlds Demand Equation Redux: The Design and Functionality of the Gold/Pray Model in Computerized Business Simulations Managing Client-Based Learning: Insights from Successful Teaching Project Courses in Marketing Tracking Forecast Error Type, Frequency and Magnitude with the Forecast Error Package Responding to Facilitate Collaboration Simulating Sudden Change and the Value of Timely Information Managing Human Resources Simulation A Study on Collectivism and Group Decision-Making: An International Comparison of Japan, China, and Russia Using a Gaming Simulation The Use of Management Games in the Management Research Agenda Gaming On-Line: A Simulation Application Positioning and Performance in Simulated Networks Supply Chain Management: A Simulation Application Simulation as a Teaching Method in Strategic Management Distance Studies Entrepreneurship: A Game of Risk and Reward Phase II: The Start-Up Return to the Paradise Islands: From Confrontation to Cooperation Effect on Market Performance of Displaying Supply and Demand Curves in a Business Simulation Appreciating Complexity: The Chief of Staff of the Army Game Managing Organizations: Experiential MBA Course Teaches Alternatives to the Machine Model A Simulation Model for Analyzing the Night-Time Emergency Health Care System in Japan The Continuing Evoluation of Assessing Project Management as an Academic Learning Outcome (ALO) Should College Instructors Change Their Teaching Styles to Meet the Millenial Student? The Mouse Game and its Effects on Team Interdependence Learning from the Gulf Oil Spill to Prepare for a Brighter Future: A New Game Engaging Stake Holders in Triple Bottom Line Accounting & Strategic Planning Video Killed the Biblio Star: The Impact of Digital Media on Student Learning Outcomes Exploring Motivation: Using Emoticons to Map Student Motivation in a Business Game Exercise An Alternative to PC and Internet Based Simulations: The Internet Integrated Mode MiddleState University -- A Crisis in Education Complexity Avoidance, Narcissism and Experiential Learning Examining the Cognitive, Affective, and Psychomotor Dimensions in Management Skill Development Through Experiential Learning: Developing a Framework A Situational Leadership Exercise Based on the Biology of a Starfish JOGAI CEFET -- The Industrial Administration Undergraduate Game Would You Take a Marketing Man to a Quick Service Restaurant? Modeling Corporate Social Responsibility In A Food Service Menu-Management Simulation Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met A Team Based Information Literacy Exercise ABSEL Marketing Communications Plan An Interdisciplinary Study of the Impact of Playing a Marketing Simulation Game on Student Knowledge of Management Accounting/Finance Principles Analyzing Construction Planning of Interiro Finish Work of Apartment Building by Simulation Doing Murder One Again The Simple Business Game and Simulation Transfering the Knowledge of Middle Management to Novices Infectious Disease Simulation Model for Estimation of Spreading Understanding the Relative Influence of Several Factors in ERP Simulation Performance: An Exploration of Ecological Validity Tragedy of the Commons: An Exercise Using Clickers to Illustrate and Teach a Key Concept in Negotiations The Meaning of Firm Demand in Business Simulations If the Games Work, Why Aren't More Faculty Willing to Play? Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players