A COGNITIVE INVESTIGATION OF THE INTERNAL VALIDITY OF A MANAGEMENT STRATEGY SIMULATION GAME Developments in Business Simulation and Experiential Learning, Volume 31, 2004 A COGNITIVE INVESTIGATION OF THE INTERNAL VALIDITY OF A MANAGEMENT STRATEGY SIMULATION GAME T. Rick Whiteley The University of Western Ontario twhitele@uwo.ca Ray Leduc The University of Western Ontario rleduc@uwo.ca Bill Dawson The University of Western Ontario bdawson@uwo.ca ABSTRACT The nature of the game parameters are set by the game creator and, in a number of games, can be manipulated by the game administrator. While the nature of the parameters generally reflect those in the real world, the advantage of a simulation game where the parameters can be manipulated is that the game participants have to learn the particular nature of these parameters (i.e., gain cognitive insight) in order to make effective decisions (i.e., behavioural response). If such learning does take place, then game participation can be considered to be an internally valid experience. This study investigates the internal validity of a management simulation game from a cognitive perspective. The results indicate that game participants were unable to determine (i.e., cognitive insight) the nature of the response functions for the demand-generating variables of the game. The internal validity of such a pedagogical tool, therefore, is open to question. If this cognitive insight is found to have occurred, then the next step in the research process would be to determine if GPs make decisions which are consistent with the perceived versus the actual importance of the game parameters. The present study focuses on the internal validity issue of simulation games from a cognitive perspective. It will be left to future research to investigate the cognitive-behavioural consistency relationship. INTRODUCTION Using a simulation game in a business course is a very time consuming activity for both students and the course instructor. However, participating in a business simulation game provides the game players (GPs) with the opportunity to engage in a dynamic case study and provides the course instructor with the opportunity to help students learn how to apply the theoretical and conceptual knowledge acquired during their academic careers. Unlike a static case analysis, which only allows the learner to suggest solutions to the case problem, a simulation game allows the participant to actually see if the proposed and implemented solution (i.e., the decision) works or not. If a simulation game is to be an effective pedagogical tool, it is necessary that the desired learning objectives are achieved. For this reason, the internal validity of such a learning experience needs to be established. LITERATURE REVIEW Simulation Literature Despite the long history (Horn & Cleaves, 1980) and widespread use of business simulation games in academia and business (Faria, 1987, 1998; Vaidyanathan & Rochford, 1998), no definitive answer can be given pertaining to the internal validity of this pedagogical tool, despite the fact that a number of research studies have empirically investigated the internal validity of simulation games (Dickinson & Faria, 1997) or have at least shown concern for this issue (Engeholm & Bigel, 1996; Wolfe, 1976). Concern for this issue is of even more significance since the demand functions in simulation games are being made more complex (Gold & Pray, 1999). There has also been effort to formulate a coherent framework to guide validation research in order to appropriately access learning outcomes (Feinstein & Cannon, 2002). If participation in a simulation game is to achieve the status of an internally valid learning experience, then it is necessary that the GPs determine the importance of the game variables that lead to effective decisions and the desired outcomes. In fact, the purpose of a simulation game is to place students in a dynamic and competitive environment where they must apply their knowledge and analytical skills to make effective decisions (Chapman & Sorge, 1999). Game participation is also expected to result in the integration of the knowledge learned from the game into the cognitive structure of the participants (Randel, Morris, Wetzel, & Whitehall, 1992). It is generally assumed that the GPs that score higher on performance measures, such as EPS, ROI, and/or sales, are the 290 mailto:twhitele@uwo.ca mailto:rleduc@uwo.ca mailto:bdawson@uwo.ca Developments in Business Simulation and Experiential Learning, Volume 31, 2004 ones who reveal the higher level of decision-making skill (Whiteley, Dickinson, & Faria, 1991). It is also assumed that those who perform as such have made decisions which are more consistent with the encountered simulation environment and have adapted the best to this environment (Whiteley, Dickinson, & Faria, 1991). A student team in a business simulation is perceived as “a simulated business organization with a coherent set of strategies, a well defined set of internal processes, and measurable outcomes” (Walters, Coalter, & Rasheed, 1997, p. 171). Before making a game decision, the presumption is that the GPs had determined the nature of the underlying response function for each demand-generating decision area and that the decision made is consistent with the structure of the underlying response function (Faria, Dickinson, & Whiteley, 1992; Whiteley, Dickinson, & Faria, 1991). Because of this expectation of cognitive-behavioural consistency in the decision-making process, it is important to determine if the cognitive structures of the GPs’ decision-making processes are consistent with the nature of the parameters of the game (Whiteley, 1993b, 1993c, 1994; Whiteley, Dickinson, & Faria, 1992). Past research focussing on the behavioural consistency between the decisions made and the structure of the game parameters failed to find the expected consistency (Dickinson, Faria, & Whiteley, 1988, 1989; Dickinson, Whiteley, & Faria, 1990; Faria, Dickinson, & Whiteley, 1992; Faria, Whiteley, Dickinson, 1990; Whiteley, Dickinson, & Faria, 1991; Whiteley, Faria, & Dickinson, 1990). This finding leads to the conclusion that the GPs failed to determine the correct structure of the response functions for the game parameters. A number of cognitively-focussed studies arrived at the same conclusion (Whiteley, 1993a, 1993b, 1993c, 1994; Whiteley, Dickinson, & Faria, 1992). All of the studies identified made use of a marketing simulation game. Attitude Formation (Cognitive Insight) and Cognitive Involvement State At the start of the game, the participants may or may not have formed a preconceived notion (i.e., attitude) about the general nature of the response functions for each of the demand- generating decision variables relevant to the competitive environment. A response function shows the relationship between inputs (e.g., price) and outputs (e.g., sales) (Arens, 2002; Lilien & Rangaswamy, 2003). The implementation (i.e., a game decision) of an input variable (i.e., the independent variable) reflects a behavioural response which is expected to influence the output variable (i.e., the dependent variable). When the GPs approach the simulation environment with a preconceived notion about the nature of the response functions and use this information in the decision-making process, then a state of cognitive involvement exists. Specifically, cognitive involvement is characterized as the presence of a heightened level of thinking and information processing with respect to a goal object (Arnould, Price, & Zinkhan, 2002). The goal object in this case is the implementation of an effective decision. The High Cognitive Involvement State shown in Figure 1 indicates that such a preconceived notion directs the behaviour of the participant. If the behaviour successfully achieves the desired goals, then a repeat of the behaviour is expected; however, if the desired goals are not achieved, then a change in attitude and subsequent behaviour is expected. This high involvement learning process should continue until satisfactory outcomes are achieved. Responses of this nature are consistent with reinforcement theory (i.e., instrumental learning) (see Hawkins, Best, & Coney, 2004; Solomon, Zaichkowsky, & Polegato, 2002). Figure 1 Level of Cognitive Involvement and Attitude/Behaviour Relationship High cognitive involvement state Low cognitive involvement state Attitude  Behaviour Behaviour  Attitude  Behaviour If the GPs have no preconceived notion about the nature of the response functions in general, or specific to the game, then no attitude directs the behaviour of the individual. The Low Involvement Cognitive State depicted in Figure 1 indicates that no attitude precedes the behaviour of the participant. However, once the outcomes of such behaviour can be assessed, an understanding of the response function (i.e., an attitude) may evolve. Once formed, this attitude is expected to direct subsequent behaviour, thereby resulting in the evolution from a low cognitive involvement state to a high cognitive involvement state. The advantage of having GPs participate in one or more trial periods prior to the start of the official game is that they can try different decision scenarios in a risk-free environment in order to gain an understanding of the nature of the response functions. However, once the real competition begins, such trial behaviour is not without risks. The GPs must then make tradeoffs between continuing in the same manner versus making adjustments in order to improve performance. Analyzing game data, including information about the behaviour and results of the competition, reflects a high cognitive involvement state, a state which ultimately is expected to direct behaviour. Thus, no matter what level of understanding or the nature of the attitude the GPs have at the start of the game, the expectation is that effective behaviour will be maintained or that effective adaptive behaviour will occur, as long as the GPs understand, or have learned, the nature of the underlying response functions. As the game progresses, random or sporadic decision making would entail a high degree of risk and, as such, should less likely occur. An assessment of an end-of-game measure of the GPs understanding of the importance of the demand-generating variables would provide the means by which to determine if effective decision-making behaviour was likely to occur. Such a finding would also provide support for the internal validity of this experiential learning tool. HYPOTHESIS 291 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 As GPs progress through the competitive environment, they are expected to adapt to the encountered environment by making decisions which will enhance their game performance. With respect to the demand-generating variables, the expectation is that the nature of the decisions (behaviour) should gravitate in a manner which is consistent with the importance of the variables, as reflected by the importance of the variables set by the game administrator. In the case of the high cognitive involvement state, it is expected that such behaviour is based on an understanding (cognitive) of the importance of each variable in terms of stimulating demand. Under the low cognitive involvement state, it is expected that by making certain decisions (behaviour), which are either rewarded or not rewarded in terms of performance results, the game participants will learn from the outcomes and adapt their subsequent behaviour accordingly. Based on these expectations, a specific hypothesis can be formulated. If the GPs have gained an understanding of the environment with which they have interacted, by the end of the game, the perceived importance of a demand-generating variable should be consistent with the actual importance of the variable as set by the game administrator. The hypothesis under investigation is as follows: H1: The perceived importance of a demand-generating variable will vary in a manner which is consistent with the actual importance of the variable, as set by the game administrator. [Cognitive Hypothesis] Specifically, the variables in the game which have been assigned a higher importance weight by the game administrator should be perceived as more important by the GPs and the variables which have been assigned a lower importance weight by the game administrator should be perceived as less important by the GPs. METHODOLOGY Subjects The students in eight sections of an advanced administrative studies course served as subjects for the study. As part of the course, the students were required to compete in a business simulation. The 7th edition of the Business Strategy Game (BSG) (Thompson & Stappenbeck, 2001) was the selected game for the course. Game performance counted for 25% of the course grade. An unobtrusive investigation was conducted, since none of the students were aware that the game environments which they faced had been manipulated by the course instructors or that data obtained from the simulation game were going to be subsequently analyzed for purposes beyond student performance evaluation. A total of 410 students played the simulation game in groups, with section sizes ranging from 45 to 56. The 108 companies in the game submitted all of the required decision material. Completed attitude (variable-importance) surveys were obtained from 312 students. The percentage-completion rates of the variable-importance surveys by industry ranged from 56.5% to 92.9%, averaging 76.1% [Industry 1 (78.2%), Industry 2 (75.9%), Industry 3 (62.2%), Industry 4 (74.5%), Industry 5 (90.7%), Industry 6 (92.9%), Industry 7 (56.5%), and Industry 8 (71.7%)]. The fact that no direct grade was assigned to the submission of the survey and variable student class attendance are the most likely reasons for the differences in the survey submission rates across eight industries (i.e., class sections). Procedure The students in the eight sections of the course, taught by two instructors, were assigned to groups (companies) of 3 or 4 members (self-selection and direct assignment used). Each class represented a separate industry (1 to 8). Within each class, each group was assigned a company letter (A to P, or to whatever was needed). Only the companies within each class competed with each other, resulting in the running of eight separate competitive games. In each game, the students participated in one trial decision period and 10 decision periods which were subject to assessment. The decisions represented the behavioural measures for the study. The industry and company structure and participant information for the game are presented in Table 1. Table 1 Industry/Company Structure and Participant Information Industry Number of companies Company letters Number of company participants Number of completed variable-importance surveys 1 16 A to P 55 43 2 14 A to N 54 41 3 14 A to N 45 28 4 15 A to O 47 35 5 15 A to O 54 49 6 16 A to P 56 52 7 13 A to M 46 26 8 15 A to O 53 38 After the last decision was submitted, the GPs were given the variable-importance survey to complete during class. The data 292 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 obtained from this survey served as the cognitive measures for the study. Design Structure. Of the eight industries in the study, two were randomly assigned to the Control Condition (CC), three were randomly assigned to Experimental Condition 1 (EC1), and three were randomly assigned to Experimental Condition 2 (EC2). Random assignment was determined via a random number table, using the single-digit industry number (i.e., 1 to 8) as the assignment criterion. The first two industry numbers which appeared in the random number table identified the industries for the control condition (Industries 3 and 5). The next three, non- assigned, industry numbers which appeared in the random number table identified the industries for EC1 (Industries 1, 6, and 7). The next three, non-assigned, industry numbers which appeared in the random number table identified the industries for EC2 (Industries 2, 4, and 8). The Business Strategy Game (BSG) allows the game administrator to manipulate the importance of nine demand- generating variables: (1) Price, (2) Quality, (3) Service, (4) Image, (5) Model Availability, (6) Advertising Level, (7) Number of Megastores, (8) Number of Retail Outlets, and (9) Value of Customer Rebates. The default demand-weight importance index is 1.00. The authors of the BSG suggest not altering the indices beyond the 0.75 (less importance) to 1.25 (more importance) range. For EC1, three variables were randomly selected to be assigned a weight of 0.75 (Variables 3, 4, and 7) and three were randomly selected to be assigned a weight of 1.25 (Variables 2, 6, and 8). The remaining three variables (1, 5, and 9) were assigned a weight of 1.00. For EC2, variables 3, 4, and 7 were assigned a weight of 1.25 (i.e., the extreme opposite to ECC1); variables 2, 6, and 8 were assigned a weight of 0.75, and variables 1, 5, and 9 were assigned a weight of 1.00 (i.e., the extreme opposite to ECC1). Variables 1, 5, and 9 were assigned a weight of 1.00 (i.e., the same as in ECC1). All variables were assigned a weight of 1.00 for the Control Condition. A summary of the experimental design is presented in Table 2. Table 2 Experimental Design Condition Control condition (CC) Experimental condition 1 (EC1) Experimental condition 2 (EC2) Industry 3, 5 1, 6, 7 2, 4, 8 Variables Variable demand weight 1, 5, 9 1.00 1.00 1.00 3. 4. 7 1.00 0.75 1.25 2, 6, 8 1.00 1.25 0.75 Variable-importance survey. At the end of the game, the participants were required to complete a survey indicating their perception (i.e., a cognitive measure) of the importance of each of the nine demand-generating variables (i.e., to reflect their interpretation of the environment that they faced). Specifically, using a scale ranging from 1 (Very Unimportant) to 10 (Very Important), the respondents were asked to indicate how important each of the following variables was with respect to stimulating demand: (1) Low Price, (2) High Quality Rating (3) High Service Rating, (4) High Image Rating, (5) High Model Availability, (6) High Advertising Level, (7) High Number of Megastores, (8) High Number of Retail Outlets, and (9) High Value of Customer Rebates. Statistical analysis. The structure of the experimental design [3 levels of the independent variable (condition) and 9 dependent variables (demand-generating variable)] indicates that MANOVA is the most appropriate statistical analysis approach for the analysis of the cognitive measures. A significant MANOVA result would require a variable-by-variable ANOVA investigation. A significant ANOVA result would require a follow-up analysis to investigate each pairwise contrast. In the latter case, the Tukey HSD approach was used to control the level of the family-wise error (FWI) to the level of the value set for alpha. RESULTS The initial analysis of the raw data using MANOVA indicated that the hypothesis of homogeneous covariance (dispersion) matrices was not tenable. Attempts at trying to meet this underlying assumption of the statistical approach through various data transformations were unsuccessful. The primary problem appeared to be with the Price (1) and Quality (2) variables. The unequal sample sizes across groups appeared to exasperate this problem [n(EC1) = 118, n(EC2) = 114, n(CC) = 76]. By running variables 3 to 9 in a separate MANOVA using the raw data, the homogeneity of covariance assumption was met (i.e., the test of the null hypothesis that the observed covariance matrices are equal across groups) [Box test of equality of covariance matrices: Box’s M = 71.067; F(56, 197401.2) = 1.226, p > 0.05]. A separate MANOVA using just the Price (1) and Quality (2) variables met the homogeneity of covariance assumption after different transformations were applied to the raw data. In the case of the Price variable, the data were normalized by dividing the raw value by the mean value obtained for the set of nine demand-generating variables. In the case of the Quality variable, the cube-value of the raw data was used. Using these values for the two identified variables with the MANOVA 293 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 approach revealed that the homogeneity of covariance assumption was met [Box test of equality of covariance matrices: Box’s M = 2.739; F(6, 1017163) = 0.452, p > .05]. Results for Variables 3 to 9 The MANOVA results for Variables 3 to 9 indicate that there is a significant difference across the set of seven variables across the three experimental conditions [Wilks’ Lambda: F(14, 602) = 2.051, p < .05 (similar results were obtained for Pillai’s Trace, Hotelling’s Trace, and Roy’s Largest Root)]. The specific differences can be determined by examining the mean values and the correlation matrix based on the data (see Tables 3 to 5). Table 3 Summary Values for Demand-Generating Variables 3 to 9 Experimental conditions Experimental condition 1 (n = 119) Experimental condition 2 (n = 115) Control condition (n = 76) Variable M SD M SD M SD High service rating (3) 5.09 2.56 6.01 2.75 5.64 2.65 High image rating (4) 7.16 1.80 7.33 2.16 7.11 1.87 High model availability (5) 6.75 2.09 7.07 1.91 6.28 2.30 High advertising level (6) 7.03 2.22 6.44 2.30 6.51 1.92 High number of megastores (7) 4.47 2.48 4.83 2.13 4.67 2.40 High number of retail outlets (8) 5.64 2.13 5.50 2.06 5.39 2.28 High number of customer rebates (9) 5.55 2.25 4.97 2.07 5.45 2.16 Notes. Mean values based on raw data. Scale ranges from 1 (Very Unimportant) to 10 (Very Important). Levine’s test of equality of error variances reveal significant results only for High Image Rating (4) and High Number of Megastores (7) (see Table 4). The univariate F-test results for variables 3 to 9 indicate that only the results for the Service (3) and Model (5) variables are significant (see Table 5). While a number of the correlations across the set of demand-generating variables presented in Table 6 are significant, the fact that the magnitude of the corresponding r2 values are so small (maximum r2 = 0.14) indicates that, for the most part, each of the decision variables essentially represents an independent decision area. Table 4 Levene’s Test for Equality of Error Variances for Demand-Generating Variables 3 to 9 Variable Fa High service rating (3) 1.45 High image rating (4) 3.28* High model availability (5) 2.96 High advertising level (6) 1.98 High number of megastores (7) 3.34* High number of retail outlets (8) 1.20 High value of customer rebates (9) 0.63 *p < .05. df = 2, 307 in each case. 294 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Table 5 Univariate F-Values for Demand-Generating Variables 3 to 9 Variable Fa High service rating (3) 4.25* High image rating (4) 0.37 High model availability (5) 3.32* High advertising level (6) 2.45 High number of megastores (7) 0.71 High number of retail outlets (8) 0.37 High value of customer rebates (9) 2.33 *p < .05. df = 2, 307 in each case. Table 6 Correlation Matrix of Demand-Generating Variables Dependent variablesa 1 2 3 4 5 6 7 8 Price (1) Quality (2) - .08 Service (3) - .22** .33** Image (4) - .05 .22** .15* Model (5) - .15* - .21** - .20** .13* Advertising (6) - .10 .13* - .09 .37** .13* Megastores (7) - .17** - .38** .04 .15** .08 .32** Retail outlets (8) .08 - .04 - .20** - .07 .11 .13* .16** Rebates (9) - .05 .11 - .03 - .09 .04 .04 - .03 .27** Notes. N = 308 (based on listwise deletion). Pearson product-moment correlation. *p < .05 (2-tailed). **p < .01 (2-tailed). aDetailed description of dependent variables provided in Design (Structure) section and other tables. The follow-up analysis for the significant univariate-F results for the Service (3) and Model (7) variables are presented in Table 7. Table 7 Multiple Comparisons (Tukey WSD Technique) of Service (3) and Models (5) Demand-Generating Variables across Experimental Conditions Variable Pairwise comparison Means compared Mean difference High service rating (3) EC1 - EC2 5.09 - 6.01 - .92* EC1 - CC 5.09 - 5.95 - .85* EC2 - CC 6.01 - 5.95 .06 High model availability (5) EC1 - EC2 6.75 - 7.07 - .32 EC1 - CC 6.75 - 6.28 .47 EC2 - CC 7.07 - 6.28 .79* *p < .05. Correlation Analysis The correlation results presented in Table 6 indicate that the set of nine demand-generating decision variables are relatively independent. Such a state of affairs would indicate that it is appropriate to bypass the MANOVA approach and go directly to the ANOVA approach (Biskin, 1983). Furthermore, the fact the investigation of the experimental treatments involves conceptually unrelated variables would also indicate that use of the multivariate approach is unnecessary (Biskin, 1980, 1983). Nonetheless, the fact that the MANOVA approach can reduce the within-subjects error, while still controlling for Type I error, warrants use of the multivariate approach. Results for Variables 1 and 2 The MANOVA results for variables 1 to 2 indicate that there is a significant difference across the set of these two variables across the three experimental conditions [Wilks’ Lambda: F(4, 608) = 6.21, p < .05 (similar results were obtained for Pillai’s Trace, Hotelling’s Trace, and Roy’s Largest Root)]. The specific differences can be determined by examining the mean values and the correlation matrix based on the data (see Tables 8 to 11). 295 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 Table 8 Summary Values for Demand-Generating Variables 1 and 2 Experimental conditions Experimental condition 1 (n = 119) Experimental condition 2 (n = 115) Control condition (n = 76) Respective normalized or cube values Variable M SD M SD M SD Low price level (1) 1.20 0.41 1.03 0.39 1.10 0.42 High quality rating (2) 421.83 281.94 320.54 297.07 353.17 290.36 Raw data values Low price level (1) 7.30 2.09 6.16 2.37 6.63 2.66 High quality rating (2) 7.02 1.97 5.96 2.50 6.43 2.27 Notes. The mean values for variable 1 are based on the normalized value relative to the respondent’s average mean (raw data) value for the set of nine variables. The mean values for variable 2 are based on the cube value of the original raw data value. The scale for the raw data ranges from 1 (Very Unimportant) to 10 (Very Important). The raw data values for each of the variables are only included for reference purposes. Levine’s test of equality of error variances reveals no significant results for either the Low Price Level (1) and High Quality Rating (2) variables (see Table 9). The univariate F-test results for the Price (1) and Quality (2) variables indicate that both are significant (see Table 10). The correlation values between all of the variables in the study are presented in Table 6 (based on raw data values). As indicated in Table 6, the correlation between variables 1 and 2 is -.08, a nonsignificant result. The follow-up analyses for the significant univariate-F results for the Price (1) and Quality (2) variables are presented in Table 11. Only two of the six comparisons are significant. Table 9 Levene’s Test of Equality of Error Variances for Demand-Generating Variables 1 and 2 Variable Fa Low price level (1) 1.58 High quality rating (2) 0.67 p > .05. adf = 2, 305 in each case. Table 10 Univariate F-Values for Demand-Generating Variables 1 and 2 Variable Fa Low price level (1) 5.14** High quality rating (2) 3.71* *p < .05. **p < .01. adf = 2, 305 in each case. Table 11 Multiple Comparisons (Tukey WSD Technique) of Price (1) and Quality (2) Demand-Generating Variables across Experimental Conditions Variable Pairwise comparison Means compared Mean difference Low price level (1) EC1 – EC2 1.20 – 1.03 0.17* EC1 – CC 1.20 – 1.10 0.10 EC2 – CC 1.03 – 1.10 -.07 High quality rating (2) EC1 – EC2 421.83 – 320.17 101.29* EC1 – CC 421.83 – 353.17 68.66 EC2 – CC 320.54 – 353.17 -32.63 *p < .05. 296 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 DISCUSSION For the High Service Rating (3) variable, the expectation was that the variable would be perceived to be most important by the EC2 group and least important by the EC1 group, with the importance evaluation by the CC group falling in between that of each of the other two groups. The results provide limited support for this hypothesis. The importance rating for the EC2 group was higher than that for the EC1 group, but it was not greater than that indicated by the CC group. However, the importance rating for the EC1 group was significantly lower than that indicated by the CC group. Thus, two of the three mean comparisons are consistent with expectations. For the High Model Availability (5) variable, the expectation was that there would be no significant difference in the variable importance rating across the three experimental conditions, since an importance index-weight of 1.00 was applied to all conditions. However, the results indicate that the EC2 group considered this variable to be more important than did the CC group. The failure to find any other between-group differences for this variable is consistent with expectations. The fact that no between-group differences were found for the High Value of Customer Rebates (9) variable is consistent with expectations. However, the failure to find any of the expected between-group differences for variables 4, 6, 7, and 8 [i.e., High Image Rating (4), High Advertising Level (6), High Number of Megastores (7), and High Number of Retail Outlets (8)] is inconsistent with expectations. For the Low Price Level (1) variable, the expectation was that there would be no significant differences in the variable importance rating across the three experimental conditions, since an importance index-weight of 1.00 was applied to all conditions. The mean comparisons between the EC1 and CC groups and between the EC2 and CC groups are consistent with this expectation. The results for the mean comparison between the EC1 and EC2 groups, however, are inconsistent with expectations: The importance rating assigned by the EC1 group was significantly higher than that assigned by the EC2 group. For the High Quality Rating (2) variable, the expectation was that the variable would be perceived to be most important by the EC1 group and least important by the EC2 group, with the importance evaluation by the CC group falling in between that of each of the other two groups. The results provide limited support for this hypothesis. The importance rating for the EC1 group was higher than that for the EC2 group, but there were no other, between-group, significant differences. Thus, only one of the three mean comparisons is consistent with expectations. CONCLUSION Similar to the findings of past studies, but using a different sample and a different simulation game, the results of the present empirical study once again failed to provide sufficient evidence that GPs develop the cognitive insight to understand the nature of the response functions underlying the game parameters. In the “real world,” marketers try to understand the nature of the underlying response functions for each of the decision areas of the marketing mix which have an impact on the level of demand, given the defined target market. In a simulated environment, GPs must do the same. Whether it is the focus of the GPs on the competition, on a particular game performance criterion (e.g., profit), on group interaction and dynamics, or on some other area of concern which is preventing them from determining the nature of the response functions is not known. Perhaps what is required before embarking on any simulation activity is to ensure that all participants have a complete understanding of response functions; it cannot be assumed that they have learned or remember such information. It might even be better to design simulation games where the participants only compete against the computer and not other teams, thereby removing the influence and dynamics of across-group interaction (i.e., group competition). A similar approach is used for 1-on-1 computerized chess games, where the competitor is the computer. It may be that too many decision variables and too many teams become too overwhelming for the GPs, causing them to lose focus during the game. At the extreme, the GPs could even be given a complete description of the nature of the game parameter response functions. If this approach were taken, then the extent to which the GPs apply this knowledge via their decisions (i.e., cognitive-behaviour consistency) could be investigated. If simulation games are to serve as a pedagogical tool designed to develop and improve decision making skills, it is essential that the internal validity of such an educational exercise is established. Finding a way to help GPs develop an understanding of the nature of the underlying response functions of the game parameters is one step in the right direction. Future research needs to address this issue. Once this is accomplished, then research can focus on whether GPs who develop such an understanding make decisions which are consistent with the nature of the response functions. REFERENCES Arens, W. F. (2002). Advertising. New York, NY: McGraw- Hill Irwin. Arnould, E., Price, L., & Zinkhan, G. (2002). Consumers. New York: McGraw-Hill Irwin. Biskin, B. H. (1980). Multivariate analysis in experimental counseling research. The Counseling Psychologist, 8(4), 69-72. Biskin, B. H. (1983). Multivariate analysis in experimental leisure research. Journal of Leisure Research, 15(4), 344- 358. Chapman, K. J., & Sorge, C. L. (1999). Can a simulation help achieve objectives? An exploratory study investigating differences among instructional tools. Journal of Education for Business, 74(4), 225-230. Dickinson, J. R., & Faria, A. J. (1997). A random strategy criterion for validity of simulation game participation. Simulation & Gaming, 28(3), 263-275. Dickinson, J. R., Faria, A. J., & Whiteley, T. R. (1988). The responsiveness of players’ strategies to the simulation 297 Developments in Business Simulation and Experiential Learning, Volume 31, 2004 environment. Proceedings of the 1988 Annual Meeting of the Decision Sciences Institute, 19, 765-767. Dickinson, J. R., Faria, A. J., & Whiteley, T. R. (1989). Do players respond to the simulation environment? Proceedings of the 1989 Conference of the Administrative Sciences Association of Canada, 10(Part 3), 102-110. Dickinson, J. R., Whiteley, T. R., & Faria, A. J. (1990). An empirical investigation of the internal validity of a marketing simulation game. Developments in Business Simulation & Experiential Exercises, 17, 47-52. Engeholm, G. M., & Bigel, H. S. (1996). The evaluation of a management simulation as a method of learning. Business Education Forum, 50 (April), 21-24. Faria, A. J. (1987). A survey of the use of business games in academia and business. Simulation & Games, 18(3), 207- 224. Faria, A. J. (1998). Business simulation games: current usage levels–an update. Simulation & Gaming, 29(3), 295-308. Faria, A. J., Dickinson, J. R., & Whiteley, T. R. (1992). A measure of the internal validity of a marketing simulation game. Simulation/Games for Learning, 22(4), 351-366. Faria, A. J., Whiteley, T. R., & Dickinson, J. R. (1990). A measure of the internal validity of a marketing simulation game. Proceedings of the Twenty-First Annual Conference of the Southwest Region of the Decision Sciences Institute, 21, 131-133. Feinstein, A. H., & Cannon, H. M. (2002). Constructs of simulation evaluation. Simulation & Gaming, 33(4), 425- 440. Gold, S. C., & Pray, T. F. (1999). Changing customer preferences and product characteristics in the design of demand functions. Simulation & Gaming, 30(3), 264-282. Hawkins, D. I., Best, R. J. & Coney, K. A. (2004). Consumer Behavior (9th ed.). New York: McGraw-Hill Irwin. Horn, R. E., & Cleaves, A. (1980). The guide to simulation/games for education and training. Newbury Park, CA: Sage. Lilien, G. L., & Rangaswamy, A. (2003). Marketing engineering (2nd ed.). Upper Saddle River, NJ: Prentice- Hall. Randel, J. M., Morris B. A., Wetzel, C. D., & Whitehall, B. V. (1992). The effectiveness of games for educational purposes: a review of recent research. Simulation & Gaming, 23(3), 261-276. Solomon, M. R., Zaichkowsky, J. L., & Polegato, R. (2002). Consumer Behaviour (2nd Cdn. ed.) Toronto: Prentice Hall. Thompson, A. A., Jr., & Stappenbeck, G. J. (2001). Business Strategy Game (7th ed.). New York: McGraw-Hill Irwin. Vaidyanathan, R., & Rochford, L. (1998). An exploratory investigation of computer simulations, student preferences, and performance. Journal of Education for Business, 73(3), 144-149. Walters, B. A., Coalter, T. M., & Rasheed, A. M. A. (1997). Simulation games in business policy courses: Is there value for students? Journal of Education for Business, 72(3), 170-174. Whiteley, T. R. (1993a). An empirical investigation of cognitive and performance consistency in a marketing simulation game environment. Developments in Business Simulation & Experiential Exercises, 20, 144. Whiteley, T. R. (1993b). Cognitive and performance consistency in a marketing simulation game competition: an empirical analysis. Proceedings of the 11th Annual Conference of the Association of Management, 11(2), 76-81. Whiteley, T. R. (1993c). Cognitive and performance consistency in a marketing simulation game environment: an empirical investigation. Proceedings of the 10th International Conference on Technology and Education, 10(1), 229-231. Whiteley, T. R. (1994). Cognitive, behavioral, and performance consistency in a computer-based marketing simulation game. Proceedings of the 11th International Conference on Technology and Education, 11(1), 479-481. Whiteley, T. R., Dickinson. J. R., & Faria, A. J. (1991). An empirical investigation of the nature and the stability of the adaptive behavior of business-game simulation teams. The Twelfth Educational Computing Organization of Ontario Conference and the Eighth International Conference on Technology and Education, 12, 91-93. Whiteley, T. R., Dickinson, J. R., & Faria, A. J. (1992). A cognitive approach to the measurement of simulation-game participant adaptability to game parameters. The Ninth International Conference on Technology and Education, 9(2), 684-686. Whiteley, T. R., Faria, A. J., & Dickinson, J. R. (1990). The responsiveness of simulation-game participants to game parameters. Proceedings of the Seventh International Conference on Technology and Education, 7( 1), 331-333. Wolfe, J. (1976). The effects and effectiveness of simulation in business policy teaching applications. The Academy of Management Review, 1(2), 47-56. 298 Table of Contents Volume 31, 2004 Controlling the Complexity and Orenting Target Groups by a Modular, Server-Based Business Game System Learning Network Demonstration: Delivering Business Education in a Distance Learning Environment Economic Evolution, Human Capital Investment, and Adult Distributed Electronic Learning: A Literature Review Designing a Globalization Simulation to Teach Corporate Social Responsibility Developing and Teaching an Online / In-Class Hybrid: A Demonstration A Model for Evaluating Online Instruction An Evaluation of a Distributed Learning Course: A Students'-Eye Perspective Blended Learning Strategy Improved Business Writing Skills How to Receive and Process Attachemnts while Greatly Reducing the Risk of Viruses and Trojans Introducing Online Components to a Class: How to Increase teh Likelihood of Success Teaching Strategic Communications Online: Using Learning Outcomes to Develop a Case-Based Course Implementing Distance Approaches to Education: A Panel Discussion for ABSEL: Las Vegas, 2004 MANDI: Learning Management Through Field Sales Experience An International Capital budgeting Experiential Exercise A Primer To Combating Terrorism: Playing It Safe While On Overseas Assignment (An Experiential Exercise) Integrating The Business Curriculum With A Comprehensive Case Study: A Prototype The Case Brief: A Model For Case Analysis, Writing And Discussion Technology Infused Pedagogy And Delivery – A Sure Bet? A Proposal For Panel Discussion Absel Conference 2004 Research Strategy And The Bkl: Getting The Most From The Absel Archives Simple But Effective: Rediscovering The Class Discussion Needle And Thread: An Activity For Examining Various Management Behaviors A Customized Excel Data Analysis System For Use In Undergraduate Marketing Research Team Leader Selection - Does It Matter? The Power Of Perspective: Reframing Your Framing Skills For Innovative Instruction In Leadership And Influence Exercise: How Should Merit Raises Be Allocated? An Online Situation For Problem-Based Learning In A Junior-Level Management Course The Eden Alternative As A Roadway For Change: A Service Learning Quality Improvement Project Avoiding Catastrophe: The Role Of Individual Accountability In Team Effectiveness Omega Systems: A Change Management Exercise The Risks And Rewards Of Providing Students A Structured Cheating Opportunity Experimentation With Assessment Techniques: A Proposal For Panel Discussion Using A 2 - Page Case To Introduce Concepts Of Business Strategy Interactive Session The Integration Of Appreciative Inquiry And Experiential Learning For Peak Performance Appreciative Inquiry Case Story: New York City Leadership Challenge Individual Achievement Versus Team Performance: An Empirical Study With Business Games Some Strategists Don't Learn Or Can't Learn Computer Simulation: A Design Architectonic On The Value Of Bugs In Simulation Environments Online Sales Forecasting With The Multiple Regression Analysis Data Matrices Package Simulation Exercises And Problem Based Learning: Is There A Fit? A Study Of Business Game Stock Price Algorithms Assessing Individual Performance In A Total Enterprise Simulation Information Use In A Business Game Determining The Value Of A Firm Unsorting Algorithms For An Ordered List And Its Application To Business Simulations Teaching Public Finance Management Through Simulation Antecedents Of Game Performance Student Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Implementation And Impacts Of The Balanced Scorecard: An Experiment With Business Games Impact: Shocking The Legacy Mindset Implementation Of The Eepad Framework Of Business Processes In An Accounting Information Systems Course Are Business Games Really Delivering What Students Are Led To Believe?? Reporting Lessons Learned: What Gets Reported; Who Gains Value Teacher Expectations Of Classroom Teaching Practices In Developing And Presenting Course Information In Hong Kong Student Reactions To The Use Of A Computer-Based Simulation As An Integrating Mechanism For A Mba Curriculum A Cognitive Investigation Of The Internal Validity Of A Management Strategy Simulation Game The Casino Challenge: Making Simulation Delivery A Safe Bet! Accounting For Company Reputation: Variations On The Gold Standard Foreign Currency Hedging: A Simulation The Influence Of Variables Easily Controlled By The Instructor/Administrator On Simulation Outcomes: In Particular, The Variable, Reflection. Absel Awareness Among Business School Faculty Validating Business Simulations: Does High Market Share Lead To High Profitability? Simulation Debriefing Procedures Coaching And Business Simulations: A Formula For Success? A Seminal Inventory Of Basic Research Using Business Simulation Games The Influence Of Scorecard Evaluation On Decisions And Outcomes