STUDENTS’ PERCEPTIONS ON THE INDIVIDUAL MANAGERIAL PERFORMANCE Developments in Business Simulation and Experiential Learning, Volume 34, 2007 161 STUDENTS’ PERCEPTIONS ON THE INDIVIDUAL MANAGERIAL PERFORMANCE Ricardo Bernard Universidade Federal de Santa Catarina – Brazil bernard@cse.ufsc.br ABSTRACT Profit performance indicators, such as net income, ROI and ROA, are usually used to assess students in a business simulation experience. Such indicators are the results of decisions made by the team. Therefore, it is not possible to measure the student’s individual effort to achieve the simulated company performance. In order to solve this problem, Bernard (2004) has proposed a methodology to assign individual performance of managerial functions to students. This paper shows the students’ perceptions of simulations utilizing the proposed methodology. More specifically, it shows how the students reacted to the introduction of this methodology and their perceptions about the importance of individual managerial performance in the grading system. Some related benefits and limitations of the proposed methodology are also highlighted. KEYWORDS: business simulation; assessment, individual performance. INTRODUCTION Business simulation literature has demonstrated that there is no relationship between simulation performance and learning (Anderson and Lawton, 1992a; Anderson and Lawton, 1997; Thorngate and Carroll, 1987; Teach, 1990; Wasbush and Gosen, 2001). Thus, as mentioned by Wasbush and Gosen, 2001), if instructors want to grade students solely on learning, they should not use profit-based performance as a criterion. However, Anderson and Lawton (1992b) documented that such criterion is commonly used by instructors to grade students. Washbush and Gosen (2001) explain such apparent paradox by stressing that in real-life organizations, managers and employees are continually evaluated on performance and rarely on learning, even in the university, professors often grade their students based on performance, not on learning. The matter becomes more problematic if one considers that profit-based performance is obtained from the company performance as a whole, and that the decision making process to achieve such performance is usually conducted by teams. A simplified grading system assigns the same grade to every individual inside each team based on the simulated company performance. In doing so, it is presumed that all individuals have contributed in an equal manner to the achieved results. It is also assumed that there is no relationship between performance and learning. Otherwise, the assumption would be that every individual inside the team had the same learning in a given business simulation experience; something even more difficult to be assessed. Bernard (2004) devised a methodology to deal with this issue; that is, each individual is formally assigned to a managerial function, such as CEO, marketing, finance or operations inside the team, and each function has a set of individual performance indicators. In doing so, it is possible to assess individual performance of each managerial function. The author has expected to bring two main contributions to the business simulation field. First, it would be aggregating a missing factor to the grading system; that is, an individual managerial performance indicator. Other studies have already discussed multidimensional criteria, as reviewed by Biggs (1978), but in every study the emphasis was on the team, not on the individual. Second, it could also bring more realism to the business simulations because, as in real-life, trade-offs between functions are expected to be more accentuated during the decision making process. This paper aims to test the Bernard’s (2004) methodology, capturing the students’ perceptions about the application, both in terms of its impact in the decision making process (e.g., realism, perceived learning and satisfaction), and in the grading system of the course (e.g., the importance of the introduction of an individual managerial performance indicator). Moreover, based on simulations using the proposed methodology and the students’ perceptions, a set of advantages and limitations are presented to help future applications. TABLE 1 – Methodology to assess individual performance in a business simulation (Bernard, 2004) Step Activity 1 Assign students to functions and teams 2 Define individual performance indicators to each function 3 Weight indicators 4 Score individual performance 5 Show results of the individual performance 6 Adjust team or function assignments Developments in Business Simulation and Experiential Learning, Volume 34, 2007 162 METHODOLOGY TO ASSESS INDIVIDUAL MANAGERIAL PERFORMANCE The methodology used to evaluate individual managerial performance in a business simulation was devised by Bernard (2004). It presumes that two basic conditions exist: the decision making process is performed by teams, not individually, and that the business simulation is expected to provide performance indicators for each function; that is, a total enterprise simulation is necessary. The steps of the methodology are presented in Table 1. In a first step each student is assigned to one team and to one managerial function such as CEO, finance, marketing, production, or personnel. Usually, these assignments can be random, self-defined, constrained self- selected, or defined by the instructor (Bacon et al., 2001). Bernard (2004) suggests a self-selected assignment to managerial functions followed by a random assignment to the team. According to the author, when students select their own functions, confidence in the decision making process is expected to be higher. Moreover, random assigning to teams is expected to create more heterogeneous groups. Associated with individual evaluations, this heterogeneity could proportionate a more realist experience since conflicts are expected to emerge more frequently in the decision making process. The next step is defining performance indicators to each function. Considering practical aspects, the author suggests that all indicators should be extracted from the reports issued by the business simulator. Moreover, no more than three or four indicators should be selected to simplify the methodology. Table 2 shows a list of suggested indicators to each managerial function. The CEO position, when existent, will continue to be assessed by the company performance indicators. In the third step weights are attributed to each indicator. In a simpler strategy all indicators receive the same weight within a function. Another strategy is differentiating weights among indicators, but the total sum of weights must be 1.0 (i.e. 100%) to each function. In the second strategy the weights can be altered by the instructor, tailoring them to meet specific needs. The fourth step is dedicated to score individual performance. Bernard (2004) suggests 3 options. The first option is scoring each indicator using a discrete scale ranging from 1.0 to 10.0. The best performance in a given indicator receives a score of 10.0, while worst performance receives a score of 1.0. Proportional scores are assigned to the remaining performances. The second option is also based on a discrete scale, but ranging from 1 to the number of simulated companies (X). The worst performance is scored as 1, while the best performance receives a score related to X. Other performances are scored between 2 and X-1. In the two previous scales a constant gap exists between scores, therefore, students can estimate how many positions they can gain or lose in the next scoring, regardless of the gaps between performances. However, as the gap between the best and the worst scores remains constant, distortions can arise; that is, lower performances will receive the same score, no matter how distant they are TABLE 2 – Performance indicators associated to functions (Bernard, 2004) Function Performance Indicator Assessment Market share (%) HB Sales growth (%) HB Sales ($) HB Marketing Demand to sales ratio (%) NZB Unit product cost ($) LB Productivity (number) HB Production programming NZB Production Employee motivation (scaling) HB Cash flow balance ($) LB Abnormal interest paid ($) LB Current liquidity ratio (%) HB Finance Debt to asset ratio (%) LB Employee turnover (%) LB Employee productivity (number) HB Motivation (scaling) HB Personnel Employee balance (necessary / existent) NZB Share value ($) HB Return on equity (%) HB Net profit margin (%) HB CEO Cumulative dividends ($) HB NOTE: HB = Higher Better; LB = Lower Better; NZB = Near to Zero Better (negative and positive values are possible) Developments in Business Simulation and Experiential Learning, Volume 34, 2007 163 from the top performance. The third option prevents such distortions from occurring, giving the top performance a score of 1.0, while the remaining performance scores represent a proportion of the top mark. Biggs (1978) describes the third option as a ‘relation approach’, while the two other options are called ‘ranking approach’ (i.e., a lower limit is set). But, regardless of the option to score individual performance, the individual scores will be composed by the sum of scores achieved in each indicator. Bernard (2004) advises to adjust the scoring of absent students in a given decision making process, by assigning zero score to all indicators of the related student. The author justifies this adjustment by expecting more commitment from individuals; otherwise, they will receive zero grades in each absent round. The next step is disclosing the achieved scores by round (e.g., by quarter) and cumulatively. This procedure is important because, according to the author, integrating performance assessment using more than one indicator and over a series of rounds reduces the luck factor and the ‘good-day, bad-day’ syndrome. Table 3 and Table 4 present examples of scoring individual performance in the quarter, and cumulatively, respectively. The final step is the reassignment process. Bernard (2004) suggests this step to avoid the creation of ‘bad’ teams resulted from the random assignments or, considering educational purposes, to permit students to perform more than one managerial function. If the reassignment is to be applied, the author advises to maintain the scores received by the student in the previous rounds, no matter to which new team, or function, he/she will be assigned to. RESEARCH METHODOLOGY The subjects for the study were 74 undergraduate students enrolled in required business simulation capstone courses at Universidade Federal de Santa Catarina – Brazil, during the first semester of 2006. Students came from 3 independent groups. A single instructor has conducted all the simulations. In one group a manufacturing simulation – SIND (2006) was applied. Students from this group were in their second experience with business simulation, but it was their first experience with individual managerial performance. The remaining groups took part of a retailing simulation – SIMCO (2006) and they were formed by students in their first experience with business simulations. Both simulators are top management games with more than 30 decision inputs per round. They are considered complex simulators according to Keys and Wolfe’s (1990) definition. The team members were selected using a self-selected assignment to functions and random assignment to the teams. Each team was composed by four members performing the following functions: CEO, marketing, finance, and personnel (in retailing simulations) or production (in manufacturing simulation). The courses were conducted during 10 simulated quarters, including a practice round. The decision making process was accomplished in the classroom. The results were available in a website and accessed by the students using an individual login. The initial two quarters were simulated using a traditional simulation performance evaluation; that is, return on equity and share value (an aggregated indicator composed by financial, economic, and market parameters). TABLE 3 – Scoring individual performance in a given quarter Function Indicator CP (Company Performance) BP (Best Performance) Score : (CP/BP) X 2.5 Market share (%) 22 28 1.96 Sales growth (%) 1 3 0.83 Sales ($) 2,987,451 3,489,988 2.14 Marketing Demand to sales ratio (%) 13 13 2.50 Total Score Company 2 7.43 NOTE: Scores in quarter 2, using 5 companies, same weight (2.5) to each indicator and option C scoring. TABLE 4 – Scoring individual performance cumulatively Company Marketing Manager Quarter 1 (score) Quarter 2 (score) Accumulated Score Ranking 1 Member A 6.89 8.78 15.67 4th 2 Member B 9.01 7.43 16.44 2nd 3 Member C 8.15 8.21 16.36 3rd 4 Member D 8.79 9.71 18.50 1st 5 Member E 7.33 6.14 13.47 5th NOTE: Scores from the quarter cells are extracted from Table 3, in row ‘Total Score’. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 164 The individual performance was measured in the last eight quarters using the indicators presented in Table 2. All indicators were equally weighted. Scores were attributed by assigning to the top performance a score of 10 and to the remaining performances, scores which represent a proportion of the top. Absent students in a given round were assigned zero scores in all indicators, regardless of the achieved function performance in the quarter. The process of scoring performance was computerized and integrated to the business simulator. This procedure has saved time and reduced processing errors. After round 6, all teams were asked to dismiss one student-manager and to hire another from within the group. The given instructions were that the changes were mandatory, the accumulated scores of the replaced student- manager would be retained, and functions could be modified for the student-manager that would be hired. It was also mentioned that the CEO had the final decision concerning the dismissal and hiring, except for the situation where the student-manager had previously negotiated his/her hiring with another company. The business simulation courses were graded using managerial performance (50%) and academic performance (50%). The indicators of managerial performance were based on company performance (15% to share value and 15% to ROE); and individual performance (20% to indicators related to managerial functions). Grading system was adjusted considering that all the students received enough managerial performance grades to be succeed in the course; that is, the worst performances in share value, ROE, and function indicators were always attributed the grade required to be approved in the course. In doing so, it would be assured that a weak business performance would not be responsible to the student’s failure in the course. This strategy is consistent with the rationale that simulated business performance is not related to learning. Therefore, if a student failure occurs, it will be associated with academic performance criteria. The indicators used to evaluate academic performance were oral debriefing (5%), written debriefing (15%), participation in decision making sessions (15%) and access to the website containing reports and graphics of the simulation (15%). A blind questionnaire was administrated after the business simulation courses had finished to gathering students’ perceptions about the individual managerial performance methodology. No previous information was provided to students associating the simulation with the present research. The questionnaire was semi-structured, that is, it contained both, closed and open-ended questions. Ninety students have participated in the simulations. Seventy four of them have answered the questionnaire, providing an 82 % response rate. RESULTS AND DISCUSSIONS Before discussing the individual managerial performance methodology it is important to verify if the students believed that the indicators chosen were appropriate to measure performance in each managerial area. Thus, two questions were formulated. Firstly, it was asked how important each indicator was (Table 2). To do so, each student only assessed those indicators that he/she was being evaluated. CEO, marketing, personnel, and production indicators had no ‘low’ importance attributed to them. Finance indicators were the only to have ‘low’ importance attributed to, but this attribution only corresponded to 1% of the total. ‘Medium’ importance and ‘High’ importance represented 56 % and 43% of all answers respectively. Moreover, when asked to suggest new indicators, only the marketing function had 2 indicators suggested. Secondly, it was asked in which round the student understood the indicators through which they were being evaluated. Forty- seven percent of the students had understood the indicators at the beginning, while forty-four percent understood them by the middle of the simulation. As a result of these two questions, it can be considered that all used indicators were appropriate to measure the functions. A question was asked to evaluate the procedure used to form the teams. It was observed that the students (48%) liked the format used to select the teams, that is, the functions were self-selected while the teams were random. This format is important because, in real-life, managers usually do not have the chance to select their colleagues. They have to adapt to the team. By applying such format, FIGURE 1 – Students’ preferences to select teams and functions 48% 9% 38% 5% Assignments were ideal Random assignments to funct ion and team Student chooses funct ion and team Student chooses team and instructor the funct ion Developments in Business Simulation and Experiential Learning, Volume 34, 2007 165 instructors are making the simulation more realistic, allowing the emergence of more conflicts in the decision making process. But, students (38%) also stressed preferences to select not only the function, but also their team mates. Such preference can be related to the fear of creating a ‘bad team’ in the random assignment (Bacon et al., 2001). However, the opportunity to change members in the middle of the simulation attenuates the consequences of a bad team. Other options cited to select teams were random assignment to functions and teams (9%), and the students select the team while the instructor assigns the functions inside the team (5 %). No other options were suggested by the students. Figure 1 shows a graphic representation of assignment preferences to function and to team from a student viewpoint. Students were questioned if conflicts had occurred during the decision making process due to the individual managerial performance methodology. Eleven percent of the students stressed that conflicts had occurred, while 79 % of the students said that they had not occurred and 10 % said that they did not know the answer. The same question was asked in an indirect way; that is, it was asked if any decision was made by the student that had generated a conflict between the function that he/she was performing and his/her company; namely, if a trade-off had existed between his/her self-interest and the team’s (company) interest. In this second question the lack of conflict in the decision making process dropped to 31 %. In other words, conflicts had occurred according to 69 % of the students. However, only 5 % of them stressed that their decisions were always based on self-interest. Sixty-one percent of the students had prioritized the company when the trade-offs appeared. Figure 2 shows a graphic representation of the students’ perceptions on the occurrence of conflicts during the decision making process. Conflicts between the managers’ interests and the interests of the company during the decision making are well documented in the management literature. This study shows that the business simulation can also simulate conflict of interests when students-managers are making decisions. However, the great majority of the students’ priorities were given to the company. An explanation can be the missing reward system that could privilege the student-managers to behave in a self-interest manner. The only reward given was the grade to the individual performance, but it was only 20 % of the grading system. The introduction of real money based on individual managerial performances could prioritize self-interest, as some authors have done to create more realism to their simulations (Chervany and Dickson, 1974; Lucas and Nielsen, 1980; Mock, 1973). However, this procedure can be questioned in the educational environment. Therefore, a simpler solution could be increasing the weight of the individual managerial performance in the student grading system. A question was formulated to evaluate the students’ perceptions on how individual managerial performance can be considered as a good indicator of learning. Learning was not defined or measured. In fact, students were only asked to link the grade they have received, based on their individual managerial performance, to their perceived learning. Answers showed that 51% of respondents considered that the grade get by this indicator were below the grade they expected to receive in terms of the perceived learning. Such result reinforces arguments that there is no relationship between simulation performance and learning cited in the literature (Anderson and Lawton, 1992a; Anderson and Lawton, 1997; Thorngate and Carroll, 1987; Teach, 1990; Wasbush and Gosen, 2001). Forty-three percent of the students considered that the individual managerial performance grade was correspondent to their learning; and 6%, that is, 4 students, reported that such indicator gave them a higher grade than they expected to receive based on their learning. Figure 3 shows a graphic representation of the team members’ expectations on grades they should have received in terms of individual managerial performance. Grades higher than perceived learning were a surprise. It is known that distortions can occur when a common grade FIGURE 2 – Existence of conflicts during the decision making process 31% 5%61% 3% No Yes. My function w as prioritized Yes. The company w as prioritized Yes. My function or the company w as prioritized in different moments Developments in Business Simulation and Experiential Learning, Volume 34, 2007 166 is to be given as a result of a team task. This is the case when the instructor uses company economic performance to grade a team member in a business simulation. But, individual performance based on function indicator was not expected to cause any distortion, especially considering that absenteeism would result in zero score in the round regardless of the performance of the related function. At least two possibilities can justify the answers given by the four students, which stated that they had higher grades in the individual managerial performance than their achieved learning. First, good students can have performed well in the simulation, but no new learning was provided to them. A second alternative is that bad students did not play the role that they were expected to play and other team members assumed the function to avoid being harmed in the company performance indicators. Some indications were given by one respondent. He/she claimed to have taken over the entire decision making process because the other members were not very interested in the exercise. The introduction of the individual managerial performance methodology has also allowed changing students between simulated companies. This procedure could also be performed without such methodology, but unfair distributions could occur. For example, presuming that instructors would grade students based on company financial performance (Anderson and Lawton, 1992), students from outperforming companies would not want to change companies since in doing so, their grades would decrease. However, in real-life, managers usually receive financial incentives to transfer to low performing companies. The introduction of the individual managerial performance is an incentive, assuring that a partial grade will be based on individual performance, no matter how weak the performance of the company is. Therefore, to some indicators, the change to a low performing company can enhance the chance to improve individual performance (e.g., sales growth). During the change process, a question was raised on the decision criteria used to dismiss and to hire student- managers. There were twenty-three changes, one in each team. Twelve changes occurred by the initiative of the students who had left their companies. In the remaining changes, the CEOs dismissed the student-managers. The reasons for the dismissals were the lack of participation in the decision making process (5), random basis (4), conflicts between student-managers (1), and low individual managerial performance (1). This result was a surprise not because of the high rate of outgoing student-managers initiative, but especially because of the random criterion employed by the CEOs to dismiss their managers. Even though all student-managers had actively participated in the decision making process (avoiding to use the ‘lack of participation’ reason for dismissal), the CEOs still could have used low individual managerial performance as a criterion to dismiss the students-managers. However, they probably preferred to use the random process to avoid having a supposed unfriendly act towards the dismissed colleague. Two questions were asked to study the impacts of student changes in the companies: one question was asked to the students that had changed companies and the other to the remaining students. When asked which impacts the change had had on the company, 43 % of the remaining students stressed that it was helpful while an equal percentage reported that it had not made any difference. Fourteen percent considered the change harmful. Some students’ comments justify their answers. Harmful impacts were attributed to the attempt of the incoming student to implement the strategies adopted in his/her previous company (2 comments) and the priority given to the function instead of thinking of the company (3 comments). Helpful impacts were associated with the personal characteristics of the incoming student (6 comments) and the student’s experience accumulated in the previous company (3 comments). Figure 4 shows a graphic representation of the students’ perception on the impacts of the replacements occurred inside the companies. The answers given by the students who had participated in two simulated companies were more expressive. Overall, these students did enjoy the experience. Some words mentioned were ‘improved learning’, ‘new knowledge’, ‘more freedom to make decisions’, and ‘new perceptions of the reality’. When analyzing the answers and comments given, it can be suggested that the change of students during a business simulation improves the experience. In the present study, only 23 out of 90 students have moved to FIGURE 3 – Students’ expectations on grades received based on individual performance 6% 43% 51% Gr ade > Lear ning Gr ade = Lear ning Gr ade < Lear ning Developments in Business Simulation and Experiential Learning, Volume 34, 2007 167 another simulated company. It was spontaneously suggested by students, both those who had changed or not, that this should be experienced by more students in the simulation, maybe by all of them. The introduction of the individual managerial performance as an indicator improves the grading system in a business simulation exercise. However, do students believe that it is a valid grading indicator? A five-point Likert scale (1 = less important; 5 = more important) was used to measure this indicator compared to other indicators used to grade students. Individual managerial performance achieved the highest evaluation with a score of 4.1, the same score get by the participation in the decision making process. Both indicators are related to the individual. Bernard (2006) has also achieved similar results when using individual managerial performance in 8 business simulation courses between 2003 and 2005. One hundred and eighty two students scored 6 indicators. Participation in the decision making process had the highest score (4.4), slightly higher than the individual managerial performance (4.1). Although in Bernard’s (2006) study the individual performance was dealt marginally (the focus was on the structure of a business simulation course as a whole), both studies indicate the importance of adopting individual managerial performance as a grade indicator when simulations use such methodology. Moreover, the highest score obtained by individual managerial performance indicates that students believe this methodology must also take part of a business simulation. Returning to the present study, share value, a team indicator, was scored 3.7 as an important indicator. Following that, the individual access to simulation results on the Internet was scored 3.2. Other indicators were: return on equity (3.1), written debriefing (3.0), and oral debriefing (3.0). Figure 5 shows a graphic representation of the importance of each grade indicator used in the business simulation course from a student viewpoint. A final question was only asked to those students that had already participated in a previous business simulation one year before. In this previous experience, the instructor was the same and a similar business simulation methodology was applied, except for the simulator (retailing simulation at first and manufacturing simulation following that) and the absence of individual managerial performance in the first experience. Thus, these students were able to compare simulations with, and without, individual managerial performance. When asked which one they had enjoyed the most, in terms of the presence of individual managerial performance, or not, the result was 28% for the simulation without individual managerial performance and 72% for the simulation with individual managerial performance. The main reason for students choosing the business simulation without individual performance was the FIGURE 4 – Impact of the students’ changes on the company 43% 14% 43% Helped Har med No di f f er ence FIGURE 5 – Importance of grade indicators from a student’s perspective 0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 Dec isi on m ak ing W rit ten de br ief ing ROE Int er ne t a cc es s Oral de br ief ing Fu nc tio n Sh ar e v alu e Developments in Business Simulation and Experiential Learning, Volume 34, 2007 168 generated conflicts. Students claimed that the procedure was harmful because in some occasions students acted individualistically. However, from an instructor’s viewpoint such behavior was expected to occur; that is, creating conflicts and trade-offs in the decision making process. The main reasons for choosing individual managerial performance were ‘a fair assessment’, based on the effective individual participation and a new opportunity to experience new environments, especially to those students who had changed of companies. These results show that, even though the second simulation meant more work to be done, the students still preferred it. However, the reasons for this choice were not directly asked for. Therefore, the comments reinforce the importance to study more on this issue as an aim to improve future business simulation applications. LIMITATIONS The limitations of this study are based on its external validity; that is, its lack of generability. Results were provided using a relatively small number of students (74). Moreover, a single instructor was used to administer the simulations and only one methodology of individual managerial performance was applied. Therefore, replications of this study are necessary to validate the present results. In a broader perspective, more studies are necessary to gather evidence of internal and external validity of this methodology, both in terms of representational and educational validity, as suggested by Feinstein and Cannon (2002). CONCLUSIONS The study showed four main benefits that the introduction of the individual managerial performance can proportionate. Firstly, it brought more realism to the business simulation. Trade-offs between the managers’ self- interest and the interest of the company which are well known in real-life were also present in the business simulation. Secondly, the students had the opportunity to learn more with the change of members between the teams. The students that moved to another company reported great satisfaction as they experienced two different ‘realities’ during the simulation. The remaining students also reported some satisfaction. Thirdly, a new indicator was inserted in the student grading system. The individual managerial performance indicator is important not only because it was validated by the students, but also because it was a missing criterion in the grading system; that is, an indicator based on the results of the individual effort in the decision making process which was not present before. Finally, it enhanced motivation and interest during the simulation as was observed by the students who experienced simulations with, and without, individual managerial performance. Limitations are also present in the introduction of the individual managerial performance. The first limitation is instructor related. Its use means more work to the instructor. The work is even more expressive if the process to attribute scores to each indicator in the managerial function is not automated. Another limitation was identified based on the data obtained from the students who had participated in the study. The introduction of individual managerial performance had an important impact on the behavior of many students. However, a small number of students did not seem to be affected by this experience. One indication was the reported higher grade received in the individual managerial indicator when compared with the perceived learning. Such indication is reinforced by comments made by a student performing a CEO role. According to the student, his/her teammates were not interested in the process and he/she basically made all the decisions alone. As a result, the individual grades attributed to his teammates were actually not originated from their efforts. This problem also occurs with traditional simulations, but it is more accentuated in individual managerial performance because the main appeal of this methodology is highlighting the effort of the individual inside the team during the decision making process. An alternative to mitigate the problem would be to consider part of the grade based on attributions given among students inside the team. Future research in the field can take many directions, as follows: • Evaluate the relationship between individual managerial performance and learning achieved in the specific function being managed. • Verify if the introduction of individual managerial performance improves simulated company performance. • Study the impact of student changes among simulated companies (with or without function changes) on team motivation and cohesion. REFERENCES Anderson, P. H. & Lawton, L. (1992a) “The relationship between financial performance and other measures of learning on a simulation exercise.” Simulations & Gaming, 23, 326-340. Anderson, P. H. & Lawton, L. (1992b) “A survey of methods used for evaluating student performance on business simulations.” Simulations & Gaming, 23, 326- 340. Anderson, P. H. & Lawton, L. (1997) “Designing instruments for assigning the effectiveness of simulations.” Developments in Business Simulations and Experiential Learning, 24, 300-301. Bacon, D. R., Stewart K. A., & Anderson E. S. (2001) “Methods of assigning players to teams: A review and novel approach.” Simulations & Gaming, 32, 1, 6-17. Bernard, R. R. S. (2004) “Assessing Individual Performance in a Total Enterprise Simulation.” Developments in Business Simulations and Experiential Learning, 31, 197-203. Developments in Business Simulation and Experiential Learning, Volume 34, 2007 169 Bernard, R. R. S. (2006) “Estrutura de utilização dos jogos de empresas nos cursos de graduação em administração e Ciências Contábeis do país e avaliações preliminares de uma disciplina baseada neste método.” Encontro Nacional dos Cursos de Graduação em Administração, CD-ROM. Biggs, W. D. (1978) “A comparison of raking and relational grading procedures in a general management simulation.” Simulations & Gaming, 9, 2, 185-200. Chervany, N. L. & Dickson, G. W. (1974) “An experimental evaluation of information overload in a production environment.” Management Science, 20, 10, 1335- 1344. Feinstein, A. H & Cannon, H. M. (2002) “Constructs of simulation evaluation.” Simulations & Gaming. 33, 4, 425-440. Keys, B. & Wolfe, J. (1990) “The role of management games and in simulation in education and research.” Journal of Management, 16, 2, 307-336. Lucas, H. C. & Nielsen, N. R. (1980) “The impact of the mode of information presentation on learning and performance.” Management Science, 26, 10, 982-993. Mock, T. J. (1973) “The value of budget information.” Accounting Review, 48, 3, 520-534. SIND – Manufacturing Simulation. Bernard Sistemas (2006). Florianópolis, Brazil: www.bernard.com.br SIMCO – Retailing Simulation. Bernard Sistemas (2006). Florianópolis, Brazil: www.bernard.com.br Teach, R. (1990) “Profits: the false prophet in business gaming.” Simulations & Gaming, 21, 12-25. Thorngate, W. & Carroll, B. (1987) “Why the best person rarely wins.” Simulations & Gaming, 18, 299-320. Washbush, J. & Gosen J. (2001) “An exploration of game- derived learning in total enterprise simulations.” Simulations & Gaming, 32, 3, 281-296. http://www.bernard.com.br/ http://www.bernard.com.br/ Table of Contents Volume 34, 2007 Experiential Teaching May Lead To Experiential Learning The Role Of Learning Versus Performance Orientations When Reacting To Negative Outcomes In Simulation Games: Further Insights An Analysis Of The Interaction Of Firm Demand And Industry Demand In Business Simulations Consistency Of Participant Simulation Performance Across Simulation Games Of Growing Complexity Assessing And Incentivizing Learner Contribution And Performance Excellence: Creating Win-Win Evaluation Options The Technological Impact Analysis: A Research-Based Exercise To Heighten Learners' Technological Sensitivity Beginnings: How We Start the Semester and Individual Classes : A Roundtable Discussion Simulation Performance and Its Effectiveness As A PBL Problem: A Follow-Up Study The Use of Multimedia Learning Tools to Facilitate Online Learning of Business Statistics Forecasting Accuracy And Learning: A Key To Measuring Business Game Performance Panel Discussion: Alternative Ways of Using the Internet for Business Simulations to Input Decisions, Process, and Present Financial and Economic Data Output Absel Research -- One Additional Perspective On Where We Are And Where We Have Come From Assessing participant learning in a business simulation From Case Presentation To Case Facilitation: How Assessment Changed The Capstone Course The Use Of Computer-Assisted, Interactive Role-Play Simulation In Hong Kong Corporate Positioning: A Business Game Perspective Demonstration Of A Computer-Assisted Global Business Simulation Applying .NET Remoting To A Business Simulation Assessing Emotional Intelligence: The EQ Matrix Exercise Outcomes And Observations Of An Extended Accounting Board Game The Use Of Learning Styles Questionnaire In Hong Kong Forming Teams For Classroom Projects Online Budgeting And Marketing Control With The Proforma Analysis Package The Effect Of Experiential Learning Experiences On Management Skills Acquisition Students' Perceptions on the Individual Managerial Performance Orderness: A New Definition Of Alignment An Exploratory Study Of The Efficacy Of Servsafe® Online Team Behavior And Team Success Results From A Board Game Simulation Telecommuting Internships - Do They Work? The WEE GAME: A Pre-Game Effects On Learning When Students Have Information About Games And Their Outcomes When Playing Them Assessment And Simulations: Measuring The Academic Learning Compacts Within Dangerous Business: An Interactive Ethics Case Activity The Paradise Islands Assessing And Applying fiAppropriatefl Conflict Management Styles Distance Learning In Communication: Blended Or On-Line? Developing An On-Line Advanced Communication Course World Café: Simulating Seminar Dialogues in a Large Class The Programming Game: An Exploratory Collaboration Between Business Simulation And Instructional Design The Canary Principle: An Alternative Model for Providing Real-Time Coaching in an On-Line Discussion Environment Assessing Character Development Using A Direct-Approach Business Ethics Exercise Persistence Of Decisions By Simulation Game Participants The Invalidity of Profit = f(Product Quality) PIMS Validation of Marketing Games A Study of The Applicability of The Perceptions of Organizational Politics Scale (POPS) For Use in The University Classroom The ROI Of Connected Projects: A Payroll Example Initiation of Research on Gaming Simulation in Japan Inappropriate Use of Citations and Corrupting the Body of Knowledge: Accepting Urban Legends as Truth An Exploration Of The Perceived Value Of Highly Socio-Inductive Learning How "Whole" Is Whole Person Learning? An Examination Of Spirituality In Experiential Learning Modeling Outsourcing and Strategy Alignment into a Business Game Successful Integration Of Webct Into A Small Business School The Thinking Steps Model In Game Theory: A Qualitative Approach In Following The Rules The Application Of Means-End Theory To Understanding The Value Of Simulation-Based Learning Armchair Travel: An active learning approach to increasing global awareness and participant self-efficacy Goldratt's Thinking Process: Is there a place for it in the Total Enterprise Simulation Assessing And Developing Student Skills Using a Group Exercise Computer Business Simulation Design: Novelty & Complexity Issues