Simulation as a Teaching Method in Strategic Management Distance Studies Page 115 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 ABSTRACT The purpose of this article is to present the experience gained from inclusion of a simulation game in the distance learning model of the “Strategic Management” course given by the Spanish National Distance Education Univer- sity (UNED). We also aim to determine the learning meth- odology that, from the students’ point of view, is considered to be the best. For this purpose, the article presents the results of a survey in which the students are asked to judge whether different teaching methods develop the skills that, according to Rodriguez Carrasco (1975), a person prepar- ing for a management post should have. INTRODUCTION Because of the training needs that people have throughout their lives, teaching should be adapted to the new social circumstances and also to the demands of stu- dents and businesses. In this respect, it should be noted that the student population is an increasingly heterogeneous entity, meaning that the education system should find the new keys to efficiently satisfy student needs for achieving their objectives. In this setting, experimental learning tools will un- doubtedly be fundamental to achieve the desired results, and it is on this premise that our proposal is based; it is intended to contribute to the satisfactory application of the new techniques to complete and complement student train- ing in the challenging context of the new European Higher Education Area. In this regard, the universities play a key role because they contribute to the production, transmission and use of new knowledge in a way that the European Commission considers unique: “research and exploitation of its results, thanks to industrial cooperation and spin-off; education and training, in particular training of researchers; and regional and local development” (Commission of the European Communities, 2003, p. 2). Experience-based instruction methods or experiential methods have the potential to overcome many of the limita- tions of the traditional paradigm. They contain different, more complex approaches to learning processes and the results; they enable interactivity and promote mutual learn- ing and collaboration; they support both cognitive and af- fective aspects of learning; and, even more importantly, they promote active learning. These methods include Business Management Simula- tors, also known as Business Simulation Games, which try to reproduce the reality of business by using the same vari- ables, relations and events found in the business world, thus providing the opportunity to develop Business Administra- tion and Management skills that would be hard to obtain outside the labor market. But in education, just as in any other activity, a tool by itself is not sufficient to achieve the proposed objectives, especially if capabilities and skills are to be developed. The tool must be accompanied by an adequate teaching method- ology that focuses its use on the proposed objectives. It is here where we believe it is of utmost importance to examine the teaching methods most suited to these tools. Logically, it is not possible to speak of a single methodol- ogy because there are significant differences between the technical characteristics of the business management simu- lators available on the market and the models for using them. Moreover, many other factors that may affect the scope of these methods, e.g., student profile, training objec- tives, time, available technological means and infrastruc- tures, economic resources, etc., must be taken into consid- eration. SIMULATION AS A TEACHING METHOD IN STRATEGIC MANAGEMENT DISTANCE STUDIES María Romero Cuadrado Universidad Nacional de Educación a Distancia (UNED) mromero@cee.uned.es Marta Solórzano García Universidad Nacional de Educación a Distancia (UNED) msolorzano@cee.uned.es Milagros Gutiérrez Fernández Universidad Nacional de Educación a Distancia (UNED) milagros.gutierrez@cee.uned.es mailto:mromero@cee.uned.es mailto:msolorzano@cee.uned.es mailto:milagros.gutierrez@cee.uned.es Page 116 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 In this article, we present the proposed teaching model: the inclusion of a business simulation game in the distance education model of the “Strategic Management” course given by the Spanish National Distance Education Univer- sity (UNED). This model was developed as part of the re- search project called “Simulation as a teaching method in Business Administration and Management studies”, under- taken in the framework of “Research Networks for Teach- ing Innovation: Development of Pilot Projects for Adapta- tion of Teaching to the European Area” (BICI, 2009) that was promoted and funded by the Spanish National Distance Education University (UNED). Together with this, we aim to determine the learning method that students consider to be the best, which means their preferences of distance learning; as well as to present the experience gained from inclusion of a simulation game in the distance learning model of the “Strategic Manage- ment” course given by the authors. To do so, we propose a multi-criteria methodology based on the MACBETH method, which classifies the alternatives on the basis of qualitative judgments about differences of attractiveness between criteria. TEACHING METHODS Teaching does is not only the mere transmission of information, but rather is a process that requires under- standing and assimilation by the recipient of the transmitted message. Therefore, it is essential that the channel selected by the teacher (the way to transmit the information) be the proper one to achieve the teaching objectives and the learn- ing principles, which to a great extent depend on the effec- tiveness of the selected method or methods. Therefore, learning enables people to become involved and put all their senses to use, creating spaces for reflection about how they do things. This is the reason why the meth- ods proposed below are especially important; if they are properly used, they lead to highly significant, lasting levels of learning, as the recipient develops a series of crucial skills to make the acquisition of knowledge effective. Campo et al. (2009) propose the use of mixed teaching systems (complementing different methods) to foster the development of skills among students and thus obtain the greatest possible pedagogical performance. To explain the different methods that are usually ap- plied to business teaching, and more specifically to Strate- gic Management instruction, we are going to classify these methods by the type of teaching provided to the student. In this way, we will be able to identify the methodol- ogy or methodologies that help to achieve the expected learning results based on the development of skills which we believe are essential for students to expand their knowl- edge of Business Administration and Management. TRADITIONAL TEACHING According to Gross and Adrián (2004), as teaching involves a transmission of knowledge from professors to students, all the different aspects of the traditional teaching system develop certain skills, including the ability to con- centrate on achieving the main objectives pursued, the use of analytical and conceptual thinking, the search for infor- mation and application techniques, decision making and the ability to think. We emphasize the following typologies: Master Class A master class is the classic teaching method. Accord- ing to Wates and Ritchie (2008), it is one of the most cus- tomary approaches because it is a fast, adequate method for learning certain tools, but it is also the most widely criti- cized method because it is very impersonal and provides little capacity for analysis (which is necessary to develop certain skills). The above is important in most disciplines but it can be considered as absolutely crucial when the subject is Strat- egy Management, since it is useless to convey numerous pieces of knowledge if the student cannot validate them when confronted with the real world. Therefore, the current procedure could be improved by correcting some of the most serious drawbacks, to enable a more active participa- tion by students and using learning methods based on ex- perience (experiential learning). Priest et al. (1997) believe it is necessary to comple- ment this methodology with this experiential learning ap- proach which, as we will see hereinafter, is more of a phi- losophy of education than a tool and which is based on the premise that people learn better when they enter into direct contact with their own experiences. It is learning “by do- ing” that reflects on the very act of “doing”. In other words, it is not limited to a mere exposition of concepts (such as a master class), but rather, through exercises, simulations or dynamics, it encourages people to assimilate the principles and put them into practice in order to develop a greater number of personal and professional skills. Practical Classes From a pedagogical perspective, practical classes are necessary to reinforce the theoretical explanations and fos- ter the capacity for analysis and synthesis. In addition, they are used as a way to motivate students because the students are able to perceive the utility of the various subject matters in actual practice. They also serve to check the degree of assimilation of the subject and to detect possible training deficiencies among the students, and as well they are a mechanism for practical-professional learning because the theoretical knowledge is applied to reality. It is important to note that practical classes come in multiple forms (Wolfe & Deloach, 2009) and that, because of the importance they have gained in recent years, they have become a part of experience-based learning. In addition to the above methods, traditional teaching also includes some procedures that can be considered as a complement to these, including the following: Page 117 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Seminars Seminars freely discuss the opinions of all the partici- pants, on the basis of previously researched material, in order to arrive at certain conclusions. In addition to broad- ening the students’ knowledge, they help to develop a criti- cal spirit (Hansen, 2005). In this way, the students learn to develop the initiative to undertake entrepreneurial ventures or propose different strategic options for the business. Therefore, this is a participative teaching method which, although of a more classic style than business simu- lation games and the cases we will analyze below, favors the direct participation of the students in preparing, ex- plaining and discussing the subject in public, in posing new problems and in thinking through discussions. According to Vélez (2007), it is a teaching technique based on teamwork and oral exchange of information that is used to work on and delve more deeply into a predeter- mined subject by collective debate and analysis. The stu- dents will therefore learn to organize their work, at both an individual and a team level, by planning, programming and making decisions. Moreover, seminars allow for a closer professor- student relationship that is beneficial for both parties. This is due to the feedback created between professor and stu- dent, making it possible to more satisfactorily evaluate the student and improve his/her knowledge and, at the same time, allowing the professor to detect the concerns of his/ her students and seek possible solutions to the doubts posed by them. On the basis of the above, we stress that seminars are not a replacement for ordinary classes, but rather they are a complement to them, to study in depth the content of one or more specific aspects of the course. Tutorials This teaching method is considered as a complement to practical and theoretical classes, as they are sessions which students voluntarily attend outside normal school hours. In these sessions, thanks to direct contact with the professor, the students are able to clear up their doubts and overcome the difficulties they face throughout the study and learning of a subject. According to Markulis (2008), this is a very valuable resource that improves the student’s study skills and en- hances his/her integral training, knowledge, attitudes and values. Lectures These are also considered as a complementary teach- ing method to normal classes. In order for this to be a truly enriching teaching method for students, it is advisable that lectures be fol- lowed by a colloquy where students can pose their ques- tions and comments, thus developing skills such as creativ- ity and imagination to find new solutions or seek alterna- tive solutions for the strategies they propose for business management, communication skills, etc. (Yu & Chan, 2005). ALTERNATIVES TO THE TRADITIONAL MODEL Today there are numerous alternatives to the tradi- tional model which generally try to improve the quality of university education. However, in view of the particulari- ties of the Spanish university, which often suffer from over crowding, some of these alternatives are difficult to use. We emphasize the ones presented below, as we consider they are the ones that best promote the development of cer- tain skills required for student learning, e.g., teamwork, use of logical, rational arguments to persuade others, informa- tion sharing to achieve goals, knowing how to plan, being participative, knowing how to apply knowledge to practical tasks, motivation, developing creativity, and analyzing and synthesizing data (Gross & Adrián, 2004). Teaching in Small Groups In this option, there is greater contact with the profes- sor because of the small number of students. Consequently, it is more likely there will be a professor-student exchange of ideas, thus creating more motivation for the students, which plays a fundamental role in learning (Iwai, 2009). For this alternative methodology to succeed, some practical, organizational and interpersonal conditions must be established to enable the group to reach its maximum potential: know the curriculum the students study, know the group of students, structure the learning in the three classic stages of beginning, development and evaluation, have a clear idea of satisfactory learning results and the techniques and methodologies required to achieve them, and use of certain resources. As seen from the above, the professor passes from one technique, e.g. master class, in which he/she merely trans- mits information, to another in which his/her fundamental role is to inform, motivate, coordinate and tutor the stu- dents so they will develop other kinds of skills. Programmed Teaching Programmed teaching is a personal, individualized instruction method in which the student follows a series of reports, questions and answers that lead him/her to the ac- quisition of the knowledge targeted by the teaching. This will enable the student to handle different decision-making tools, prepare reports and presentations, and communicate and argue the results obtained with his/her management. The main advantage of this method is that the individ- ual progresses at a pace suited to his/her qualifications and previous training. However, the thing that most character- izes programmed teaching is its cost, as many of the pro- grams have to be specifically developed for each situation and there are not many people who master both techniques at the same time – the subject matter to be explained and its programmed design (Rodríguez Carrasco, 1975). Page 118 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Personalized Teaching System This technique, originally designed by Keller in 1968, features a personalized teaching method adapted to the stu- dent’s own rate of comprehension. In this method, the work is done by the students at home and, at the same time, they can ask their tutors ques- tions (in small groups) or discuss the subjects with their companions. This will develop numerous learning skills that will broaden their knowledge of the subject. It should be noted that, in order to successfully imple- ment this technique, it must be complemented with the use of a study guideline, technological support resources, tuto- rials, group classes or seminars and continuous evaluation (Cantón, 2007). EXPERIECE-BASED LEARNING Many people would agree with the following quotation attributed to Confucius: “I hear and I forget. I see and I remember. I do and I understand” (ABSEL, 1990). This phrase expresses the effectiveness of experience-based learning or experiential learning which, as commented above, is characterized precisely by the belief that “you learn by doing”. Consequently, the advocates of this theory consider that students should actively participate in their own learning process in order to boost their ability to learn to learn, i.e., understanding one’s own way of learning and the processes required for it (Rivera, 1996). Kolb’s cycle of experiential learning (Kolb, 1984) per- fects the work of other authors in this field, e.g., Ackoff (1959), Dewey (1938) & Piaget (1978). It focuses on the idea that experiential learning exists as a particular way of learning distinguished by the central role that experience plays in the learning process. As we will show later, this model assumes that, in order to learn something, the infor- mation that is received must be worked on and processed. According to this idea, we can start with a direct, specific experience, or else with an abstract experience such as the one we have when we read about something or when we are told about it. The experiences we have, either concrete or abstract, are then transformed into knowledge when we elaborate on them in one of two ways: either by reflecting on and thinking about them, or else by actively experiment- ing with the information received. For this reason, whatever the methods or techniques to be used, they should help the student become involved in his/her learning experience; the student is the one who should observe, test, analyze and participate in the various activities of the process to integrate the new knowledge. Bearing this in mind, experiential learning influences the student in two ways; it improves his/her cognitive struc- ture and modifies attitudes, values, perceptions and behav- ioral patterns. These two elements of the individual are always present and interconnected. Therefore, student learning is not a development isolated from cognitive de- velopment, but rather a change in the whole cognitive- affective-social system. It is thus through active, significant and experiential participation that students will build new and meaningful knowledge that will influence their education and result in responsibility for and commitment to their own learning. As Ausubel (1976: 259) says, only when learning is mean- ingful does the deliberate intention to learn arise. The pedagogical methods most directly related to ex- periential learning are the case method and simulation models. Both allow the student to live through a business situation and to learn through the experience obtained from it. The combination of both methods will help to boost the student’s learning in a more effective way, by applying knowledge to practical tasks (Aidar, 2006) and enabling the student to develop skills such as those related to decision making and problem solving, teamwork, negotiation and the use of critical thinking (Blasco, 2000; Gross & Adrián, 2004; Escobar & Lobo, 2006). The Case Method This method can be considered as a pedagogical tech- nique somewhere between traditional teaching methods and practical training (García Echevarría, 1974). Its pedagogical purpose is to drill the student in the decision making technique, after having analyzed, dis- cussed and reasoned a situation based on his/her knowledge and previous experiences. Thus, the student will be able to properly employ techniques and methods of business analy- sis throughout the decision making process by confronting real situations, compiling and evaluating the facts, defining the problems that may arise, identifying possible solutions, choosing the best alternative and preparing an action plan suited to each situation. In order for the method to be effective and manifest all its pedagogical force, a series of conditions must be met: the document to be analyzed should be a real, authentic case because, if not, there is a risk of students losing confi- dence; the participants should have a theoretical knowledge consistent with the difficulty of the problem; and the stu- dents should be willing to take part in discussions, which must be conducted on the most participatory basis possible. In this respect, the director must take care not to impose, and not even suggest, a solution a priori. In addition, it is very important that the students be repetitively exposed to a considerable range of situations so they will familiarize themselves with all of them; this will improve their general training and help to get around one of the main drawbacks of this method, which is precisely its specialist nature based on the study of specific business cases. The experts claim that the use of less than twenty cases makes the method incomplete and inefficient and that the optimum level is achieved as of fifty cases (Christensen, 1991). Business Games It can be safely said that business simulation games combine the advantages of other methods, especially those Page 119 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 referring to student participation and decision making. This is why they are considered as one of the most comprehen- sive methodologies that best develops the pedagogical skills of the users (Domingo, 2004). Business games are “... a form of simulation where decisions must be made in successive stages; the players enter their decisions into a model, which simulates interac- tions between the simulated business environment and the participants’ decisions. The participants, once they have examined the results, make another series of decisions and the cycle is thus repeated” (Rodríguez Carrasco, 1975). In this way, students will learn to manage a business from the perspective of strategic management, thus improv- ing their capacity for negotiation as they will have to trans- act operations with other businesses and be able to devise and implement competitive strategies. Therefore, business games can be considered as a more perfected version of the case method and also as a type of practical class, as explained above. The main difference between this technique and the case method is that, while in the latter the solutions are compared in a group discussion and the students cannot know the results of their decisions, in a business game the simulation provides objective, continuous feedback on the consequences of those decisions, which results in a closer approximation to the reality of business and a greater moti- vation for the students. Nevertheless, we should be cautious and realize that, at least at present, there is no one game, nor a set of them, that is capable of providing a complete business education be- cause it is not possible to capture all the real day to day problems that businessmen face. In addition to the methods mentioned above, we should include projects and company internships in the experiential teaching techniques, as they are based on the premise of these techniques: “learn by doing”. Projects In this method, the students prepare projects, whether individually or in groups, on a subject related to the course they are studying. Projects are intended to stimulate the training of stu- dents and prepare them to independently analyze an issue in which they may have an interest (Potosky, 2004). It is an activity in which students are able to develop their creativ- ity and it is carried out under the direction and guidance of the professor, who tries to ensure that all the students par- ticipate and also develop certain skills: the ability to find solutions, bibliographic analysis and critique, form and express their opinions, teamwork, communication and speaking in public, etc. To develop the latter skill, however, a necessary condition is the prior exposition and subse- quent defense of the project before the professor and the rest of the group. Internships in Companies and Institutions The purpose of this method, which complements theo- retical and practical teachings, is to contribute to the inte- gral training of students by having them perceive the insti- tutional, entrepreneurial and labor reality of their social environment in the area of business administration (Sanders, 2001). In this way, the students will develop a critical spirit and a capacity for decision making and team- work, as well as learn a work methodology suited to the professional world where they will have to operate when they graduate, comparing and applying the knowledge ac- quired in the university. BUSINESS GAMES AS A TEACHING METHODOLOGY LEARNING OBJECTIVES USING BUSINESS MAN- AGEMENT SIMULATORS At present, the undergraduate level is configured by theoretical and practical studies that will enable students to acquire a series of skills demanded by the market. Bearing in mind that our study objective is to analyze new learning methods for Business Administration undergraduate stu- dents, we have analyzed the generic skills considered in the Business Administration and Management degree course, and we see that the ones that can be developed using man- agement simulators are as follows: Generic Skills Instrumental: Information analysis and synthesis; Work organization and planning; Diagnosis of situations; Problem solving; Decision making; Resource management; Use of Information Tech- nologies Personal: Teamwork; Multi-disciplinarity and multi- culturalism; Critique and self-criticism; Work ethics; Work under time pressures. Systemic: Autonomous learning; Adaptation to new situations; Creativity; Leadership; Initiative and entrepreneurial spirit; Motivation to do a job well done; Quality; Focus on results; Capacity for ne- gotiation. Specific Skills: Comprehension of the fundamentals of business man- agement. Comprehension of the existing relationships between different business activities and areas. Use of business analysis techniques and methods for decision making. Formulation and implementation of competitive strate- gies. Forecasting and planning. Analysis of the effect of financial statements on finan- cial operations. Comprehension of the basic financial instruments and their relationships. Use of management marketing resources. Putting knowledge into practice. Page 120 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 Analysis of the cause-effect relationships of the deci- sions made. It is important to note that the development of any of these skills will depend on the usefulness and the function- alities of the tool. These skills, expressed in terms of learning objectives in a business management simulation seminar/course, would be as follows: Students should be able to analyze economic-financial information and synthesize it in a way that facili- tates its analysis. They should be able to organize their work, both indi- vidually and as a team, by planning and program- ming the tasks to be performed. They will be able to diagnose the situation of a busi- ness and the environment in which it operates, considering the strategic options that may result from the diagnosis. They should confront and deal with problems and pro- pose, evaluate and implement solutions in order to understand their consequences. In other words, they will make decisions about concrete situations and see their impact. They will be able to manage all kinds of resources, all on an interrelated, interdependent basis, and use them as efficiently as possible. They will handle different tools for decision making and preparation of reports and presentations. Since the students will basically have to work as a team, they should be able to organize the functions to be performed by each of the members, resolve any conflicts that may arise and achieve the tar- gets they set. They should develop their capacity for critique and self -criticism. Since there is a time limitation on the activities to be carried out, they should be able to work under the pressure of deadlines, which is the situation they will encounter in a business environment. They should be able to make an ethical assessment of their decisions and their teamwork. They will develop a capacity for autonomous learning based on previous knowledge in business subjects and on the experience gained in business manage- ment using a simulator. They will adapt to all facets of new, rapidly changing, complex situations. They will be creative in the strategies proposed for management of their business. They will improve their leadership skills and their abil- ity to influence others and to manage or move their team in a certain direction. They will develop their initiative to take entrepreneu- rial actions and also to propose strategic options for the business. They will learn to perform tasks with the best possible quality and to do their jobs well. They will be able to focus on results and learn to man- age a business from the perspective of strategic management. They will improve their capacity for negotiation in transactions with other businesses. They will be able to understand the fundamentals of business management, the different areas of a business and the relationships between them. They will know how to use the appropriate business analysis techniques and methods during the deci- sion making process. Figure 1 MACBETH Method Scheme Source: Bana e Costa, De Corte and Vansnick (2005). Page 121 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 They will be able to devise and implement competitive strategies. They will be able to make forecasts of the evolution of both environmental and business variables and plan the actions to be taken. They will comprehend and be able to analyze the ef- fect of financial operations on corporate financial statements. They will learn to use marketing resources in manage- ment as effectively as possible. They will be able to transfer to others the knowledge acquired in the seminar/course in the simplest way, as they will have applied that knowledge to a practical situation. They will be able to analyze the cause-effect relation- ships of the decisions made. By covering all the objectives that are set, the expected learning results will be achieved. These results are as fol- lows: Students should know how to use the Business Man- agement knowledge acquired and be able to use analysis techniques and make business decisions. They should be capable of integrating the knowledge acquired on the various business areas and assess the implications that the decisions in one area have on the rest. They should know how to analyze the complexity of markets based on limited information, make judg- ments about the potential of the resources they manage and focus on business decisions. They should know how to communicate and defend, orally and in writing for a specialized public, the results obtained from their management, using the appropriate tools for this purpose. They should develop learning skills that allow them to broaden their knowledge of Business Administra- tion and Management. APPLICATION OF BUSINESS SIMULATORS TO A REAL CASE. PILOT TEST WITH STUDENTS AND TEACHERS OF THE “STRATEGIC MANAGE- MENT” COURSE OF THE BUSINESS ADMINI- STRATION AND MANAGEMENT DEGREE PRO- GRAM. Since one of our objectives is to test business manage- ment simulators as a suitable teaching method for under- graduate courses, we have conducted a pilot test with a group of students from the Spanish National Distance Edu- cation University. We should mention that this was a spe- cial case because the game would be played in a distance education setting by students who had the advantage of already having some business experience. A pilot test was carried out in the month of March with students from the “Strategic Management” course, a fourth- year course of the Business Administration and Manage- ment undergraduate degree program. Since the ideal number of participants for a simulation would be from 25 to 30, 24 students were selected for the test out of a total of 35 applications from students in the course. The methodology used was as follows: The game selected for the seminar was the INTOP simulation game. A message was posted on the student forum of the course webct, and on the course Teaching Staff and Gen- Figure 2 Tree of criteria selected for choosing a suitable method Source: Prepared by authors based on the M-MACBETH software Selection of a suitable method  Objectives  Information  Planning  Generalist and specialist  Team  Motivation  Participation  Feedback Page 122 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 eral Queries web, which explained the project and re- quested student volunteers to take part in it. The request explained that 24 students would participate and that the selection would be made in keeping with the order of appli- cation. Each of the participants was assigned to a work team with 4 or 5 people who assumed the responsibility for man- aging one of the computer-simulated market businesses. The first sessions focused on familiarizing the teams with the “rules of the game” and stressing the most useful con- cepts and techniques for analyzing the information to be provided to each team at the end of each period, and after processing the decisions that had been made. The simulation was developed with a distance teaching model. In an early mail, the coordinator sent to the mem- bers of each team the composition of each group, a game manual, some templates for decision making and the rules to be followed, which included the timeline for decision making. After that, and using the e-mail of the webct plat- form, the head of each team would send to the project coor- dinator his/her team’s decision for each period. In accor- dance with the organization of the game, the coordinator had to send the result of the decision within two days. On the established date, the team, in accordance with the re- sults received and its business strategy, would send the next decision, and so on. The students were evaluated according to their partici- pation and the final report which was submitted by each simulation team. Having analyzed the teaching methods considered as most suitable for teaching Business Administration and having presented the teaching project, we can now proceed with an analysis of the ideal teaching methodology from the students’ point of view. SELECTION OF THE MOST ADEQUATE METHOD Both the generic and specific skills that can be devel- oped using business management simulators have been presented above; however, when carrying out an empirical analysis, it is a complicated matter to verify if each of the above mentioned skills has actually been developed. There- fore, to check whether business games help the students become effective executives, it was decided to follow Rod- riguez Carrasco (1975), who considers that a person prepar- ing for a management post should have the following quali- ties: Ability to set targets and operationally define them: the business game will be more comprehensive if it has features that help the student establish and achieve the business targets. Ability to abstract, organize and use information from a complex, diffuse environment: executives live in a complex world that provides them with a lot of information, which is why one of their main goals will be to select the most relevant data for the or- ganization and seek solutions to important prob- lems. The business game should simulate the real world and help the students develop these skills. Ability to predict and plan: from the very beginning, the student should learn to understand the conse- quences of his/her decisions or predict their re- sults, in order to compare the predicted results to those actually obtained after the simulation. Ability to combine the roles of generalist and special- ist: a good executive should combine his/her train- ing as a specialist, which will condition his/her conduct in the most important decisions, with that of a generalist concerned with the general policy of the business. Therefore, business simulators should enable the students to develop specific and generic skills to successfully manage the organiza- tion. Ability to work effectively with other people: execu- tives should be able to maintain cooperative rela- tionships with their superiors, with other execu- tives and with their subordinates. Therefore, it is essential that the business game foster the capacity for teamwork in a student. Moreover, in addition to developing the above men- tioned capabilities, the student must be motivated and actively participate in his/her teaching, and for this purpose constant feedback from the teacher is essential. Obviously, none of the teaching methods described above is comprehensive enough to succeed in sufficiently developing the indicated skills and transforming the student YES Figure 3 Equivalence between MACBETH scale and the one used in the survey Source: Prepared by authors NO Page 123 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 into a true manager. We stress that the ideal thing is to combine methods1; therefore, in the case under study, we have chosen to combine business simulators with the tradi- tional distance education provided by the UNED. DESCRIPTION OF THE METHODOLOGY USED The methodology used in this work is based on a multi -criteria analysis to obtain a classification of the previously analyzed teaching methods (which act as classifiable alter- natives) in terms of the pedagogical skills developed by them (which act as criteria or variables to be considered). The selection of a multi-criteria methodology is based on the existence of multiple criteria (in this case, skills) which should be evaluated and classified from a uniform perspective. As a result, the decision maker’s objective will be to identify an alternative – in this case, a teaching method – which satisfies all the selected criteria as far as possible. However, it should be noted that there will be no absolute “best solution” that is valid for all decision mak- ers, since the results of the multi-criteria problem will de- pend on the individual preferences of each evaluator or group of evaluators. These preferences may be quantita- tively or qualitatively expressed, and there are different methods for processing the information. In the case under study, qualitative information is available (expressed in affirmative and negative terms) on the opinion of a group of students regarding the extent to which certain teaching methods develop a series of peda- gogical skills. This information was compiled through a survey taken of a total of 24 students from the Strategic Management course given in the UNED by the Business Organization Department2. To process the data, we have used the MACBETH method introduced by Bana e Costa and Vansnick in 1994 and developed in the autumn of 2002 via the M- MACBETH software by the same authors, together with De Corte3. This selection is motivated by the fact that this technique requires exclusively qualitative judgments about the differences of attractiveness between elements, to gen- erate scores for the options of each criterion and to weight the criteria (Bana e Costa, De Corte, Vansnick, 2009). The MACBETH method is based on the theory of multi-attribute utility developed by Keeney and Raiffa in 1976, in which each criterion indicates a partial description of the utility of the alternatives from a certain point of view. These partial utilities, aggregated by a value function, will serve as a reference for the calculation of the overall utility, which will be used as the basis for solving the prob- lem. The application of this technique includes a series of phases that begin with a first stage of structuring, which establishes the criteria and options that should be evaluated, along with their respective performances. In a second phase, the attractiveness scores or preferences of each op- tion are evaluated with respect to the different criteria on a semantic scale of 7 categories: null, very weak, weak, mod- erate, strong, very strong and extreme, thus enabling the evaluator to express indecision (Sánchez & Bana e Costa, Figure 4 Performance Table Source: Prepared by authors based on the M-MACBETH software 1 Aidar (2006) proposes a combination of the case method and the business game. 2 The referenced survey can be consulted in the appendix and is characterized by its simplicity, as the primary goal of its developers was to avoid having students show any indecision in their answers. 3 An introduction to and demonstration of this software can be found at http://www.m-macbeth.com. http://www.m-macbeth.com Page 124 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 2009). It is from this scale that the acronym MACBETH is derived: “Measuring Attractiveness by a Category Based Evaluation Technique”. In this second stage, weights should also be generated for the criteria. In the third and final stage, the consistency of the evaluator’s judgments is verified as the judgments are entered into the software, which suggests different possibilities in the cases of incon- sistency. Once the matrix of qualitative judgments is con- sistent, the software evolves towards a quantitative evalua- tion model that provides an overall score scale which re- flects the attractiveness of the alternative in question with respect to the whole set of criteria and which should be confirmed by the evaluator (Gironés, Madrid & Valls, 2008). Finally, the sensitivity and robustness of the model results are analyzed, considering a key factor in the multi- criteria method: data uncertainty. The following figure shows the scheme that should be followed on implementing the MACBETH approach: APPLICATION OF THE MACBETH METHOD TO THE STUDY IN QUESTION Having explained the reasons for selecting the MAC- BETH method and its main features, it is possible to begin applying this technique to the case under study, following the phases indicated above. The options will logically be the various training meth- ods that have been chosen to undertake the study: master class, practical classes, seminars, tutorials, lectures, teach- ing in small groups, programmed teaching, personalized system, projects, company internships, case method and business games. Eight criteria have been assigned to the problem and they coincide with the specific skills they are intended to develop in the student on teaching the “Strategic Manage- ment” course. Figure 2 shows the tree provided by the soft- ware that contains these skills or criteria. As explained above, to acquire the qualitative data, a survey has been conducted of each of the students who forms part of the teaching innovation group in which this study has been carried out. In order to facilitate the stu- dents’ responses and prevent the possibility of expressing indecision, only affirmative or negative answers (YES or NO) were permitted in the survey. If compared to the ter- minology of the MACBETH method, these answers would refer to the extremes of the semantic scale explained above, i.e., YES would be equivalent to “extreme” and NO to “null ”. As there were 24 surveys and the application permits only one general answer, it was decided to process the data of all the students and calculate the mode of each answer, which resulted in the table of performances shown in figure 4 where the opinions of the students regarding the skills developed by each of the teaching method are expressed. As explained above, the criteria should be weighted. However, considering that the objective of this research is to analyze which method is most appropriate from the stu- dents’ point of view, it was decided not to enter judgments into the software, meaning that all the skills to be devel- oped were considered as equally important (the relationship between the variables is “positive”) and, consequently, accepting the scale proposed by M-MACBETH. Taking the above into consideration, the results can be analyzed. For instance, figures 6 and 7 show that, from the students’ point of view, the methods considered to be best (with the highest score) are the ones that form part of the so -called “experiential learning”, and specifically company internships, case method and business games. Nevertheless, to be able to safely say that these meth- Figure 5 Criteria Weighting Source: Prepared by authors based on the M-MACBETH software Page 125 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 ods would be the ones chosen by students in all cases, a robustness analysis should be carried out in order to elimi- nate the data uncertainty and reliably show the conclusive results. The preceding figure contains a global comparison by pairs of options and shows the relationship between them. Consequently, different symbols may appear: if “a prevails over b” a triangle will appear in line a and column b; if “a is globally more attractive than b considering the available information” a cross will show the relationship between the options; and if they are equivalent an “equal” sign will be displayed. In the case under study, it can be seen that there is no difference between the experiential teaching methods and that they are preferred over the rest of the options; there- fore, after the robustness analysis, it can be concluded that the students prefer experiential learning, in any of its forms, over any other didactic methodology. CONCLUSIONS This experience with distance teaching of the “Strategic Management” course is the result of a desire to change the teaching of skills that a person preparing for a management post should have, and also the need to adapt the Business Administration teachings to the European Higher Education Area. Using a business simulator, the students have had the chance to learn business management concepts and skills and put their knowledge of the subject into practice. This article proposes a method for distance teaching of management skills that uses a dual methodology: tradi- tional teaching and experiential learning. When judging the benefits of the different teaching methods for the distance teaching of the “Strategic Man- agement” course, the students give the highest scores to the experiential learning approaches. REFERENCES Ackoff, R. L. (1959): Game, decisions and organization. General Systems, vol.4, pp.145-150. Aidar, A. (2006): Cases and business games: the perfect match! Developments in Business Simluation and Ex- Figures 6 and 7 Overall Thermometer and Table of Scores Source: Prepared by authors based on the M-MACBETH software Page 126 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 periential Learning, vol. 33, pp- 204- 212. Ausubel, D. P. (1976): Psicología educativa. Un punto de vista cognoscitivo. Trillas. México. Bana e Costa, C.A. & Vansnick, J.C. (1994) MACBETH, an interactive path towards the construction of cardinal value functions, International Transactions in Opera- tional Research, vol. 1, pp. 489-500. Bana e Costa, C.A. & Vansnick, J.C. (1997a) Applications of the MACBETH Approach in the Framework of an Additive Aggregation Model, Journal of Multi-criteria Decision Analysis, vol. 6, pp. 107-114. Bana e Costa, C.A. & Vansnick, J.C. (1997b) A theoretical framework for Measuring Attractiveness by a Cate- gorical Based Evaluation Technique (MACBETH), in Clímaco J. (ed.), Multicriteria Analysis, Springer, Ber- lín, pp. 15-24. Bana e Costa, C.A. & Vansnick, J.C. (1999) The MAC- BETH Approach: basic ideas, software, and an appli- cation, in Meskens, N. and Roubens, M., Advances in Decision Analysis, Kluwer Academic Publishers, Dordrecht, PP. 131-157. Bana e Costa, C.A., De Corte, J.M. & Vansnick, J.C. (2005), M-MACBETH: Guía del usuario, Retrived from: www.m-macbeth.com Blasco, F. (2000): Teoría y aplicación de los métodos de aprendizaje en las Ciencias Empresariales: análisis comparativo entre el Método del Caso y el Juego de Simulación de Empresa. EU Empresariales. Seminario de Marketing. Universidad Complutense de Madrid. Campo, J., Cano, J., Herrera, D. & Nova, J. (2009): Aplica- ción de un sistema mixto de enseñanza. Revista Com- plutense de Educación, vol. 20, nº1. pp: 135- 150. Cantón, I. (2007): Enseñanza en pequeños grupos en edu- cación superior. Tutorías, seminarios y otros agrupa- mientos. Revista Interuniversitaria de Formación de Profesorado, vol. 21, nº 2- 3, pp- 201- 203. Universi- dad de Zaragoza. España. Chan, J., Jennings, D. & Sun, L. (2005): Strategic manage- ment: an evaluation of the use of three learning meth- ods in China. Developments in Business Simulation and Experiential Learning, vol. 32, pp- 96- 104. Chiavenato, S. (1997): Qualitative methods in organisa- tional research. SAGE. London. Christensen, C. R. (1991): Education for Judgment: The Artristy of Discussion Leadership. Harvard Business School. Boston. Dewey, M. (1938): Mental models. Towards a cognitive science of language, inference and consciousness. Har- vard University Press. Cambridge. Domingo, M. A. (2004): Los juegos de empresa sobre ges- tión de operaciones en la formación universitaria. Do- cumento de trabajo no publicado. Universidad de Se- villa. Elgood, C. (1981): Handbook of Management Games, Go- wer Publishing Company, Hampshire. Escobar, B. & Lobo, A. (2006): Empleo de juegos de simu- lación empresarial como herramienta para la innova- ción docente: experiencia en control de gestión de la diplomatura de turismo. Revista de enseñanza univer- sitaria, extraordinario, pp. 34- 47. Farooqi, N. M. (2009): Increasing student learning in an investment management course through the innovative use of experiential learning pedagogy- an interactive learning session. Developments in Business Simulation and Experiential Learning, vol. 36, pp. 56- 62. Galagovsky, L. R. (2004): Del aprendizaje significativo al Figure 8 Robustness Analysis Source: Prepared by authors based on the M-MACBETH software http://www.m-macbeth.com Page 127 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 aprendizaje sustentable. El modelo teórico. Enseñanza de las Ciencias, vol. 22, pp. 229- 240. García & Meneses (2000), “Decisiones Multicriterio: un enfoque alternativo al método MACBETH”, in Anales de Economía Aplicada, XIV Reunión ASEPELT- España, Oviedo. García, S. (1974): Economía de la empresa y política económica de la empresa. Díaz de Santos. Madrid. Gironés, X., Madrid, D. & Valls, A. (2008), Comparación de Dos Métodos de Toma de Decisiones Multicriterio: MACBETH y PROMETHEE, Reportes de Investiga- ción: DEIM-RR-08-004, Universitat Rovira i Virgili. Gros, B. & Adrián, M. (2004): Estudio sobre el uso de los foros virtuales para favorecer las actividades colabo- rativas en la enseñanza superior. Retrieved from: http://campus.usal.es/teoriaeducacion/rev_numero_05/ n5_art_gros_adrian.htm Hansen, K. (2005): Application of traditional and online journaling as pedagogy and means for assessing learn- ing in an entrepreneurial seminar. Developments in Business Simluation and Experiential Learning, vol. 32, pp- 172- 179. Iwai, C. (2009): Group Decision Experiments using busi- ness game- problem solving with conflict. Develop- ments in Business Simulation and Experiential Learn- ing, vol. 36, pp- 150- 163. Joubert, J. (2002): Pensamientos. Edhasa. Barcelona. Keeney, R.L. & Raiffa, H. (1996), Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley, New York. Kolb, D.A. (1984): Experiential learning: Experience as the source of learning and development., Englewood Cliffs, Prentice Hall, New Jersey. Markulis, P. M. & Strang, D. R. (2008): Assessment in the modern large high- tech classroom. Developments in Business Simluation and Experiential Learning, vol. 35, pp. 191- 196. Oxford English Dictionary (2005): Oxford English Diction- ary Online. Oxford, England: Oxford University Press. Parras Ramírez, I. (2008): Enseñar es aprender dos veces. Mc GrawHill. Barcelona. Piaget, J. (1978): The role of management games and simu- lations in education and research. Journal of Manage- ment, vol. 16(2), pp. 307- 336. Platt, A. & Alexakis, G. (2005): Employing progressive practices and principles to facilitate seminar room leadership among learners: shared power and collec- tive accountability. Developments in Business Simula- tion and Experiential Learning, vol. 32, pp. 43- 50. Potosky, D. (2004): Developing and teaching an online/in- class hybrid: a demonstration. Developments in Busi- ness Simulation & Experiential Learning, vol. 31, pp. 28- 31. Priest, M., Simon, L. & Gass, M. (1997): Effective Leader- ship in Adventure Programming. Prentice May. Ox- ford. Ramírez Fritz, P. (2005): La teoría humanista del aprendi- zaje. Atenea. Buenos Aires. Rivera, G. (1996): Estrategias para el aprendizaje. Kronos. Buenos Aires. Rodríguez Carrasco, J. M. (1975): Juegos de empresa. ESIC. Madrid. Salas, M. (2009): Del proceso de enseñanza aprendizaje tradicional al proceso de enseñanza aprendizaje para la formación de competencias, en los estudiantes de la enseñanza básica, media y superior, Cuadernos de Educación y Desarrollo, retrived from http// www.eumed.net/rev/ced/, Universidad de Málaga. España. Sánchez, R. & Bana e Costa, C.A. (2009) The MACBETH approach for multicriteria evaluation of rural develop- ment projects in face of Cross-Cutting Issues, Working Paper LSE OR 09.107, The London School of Eco- nomics and Political Science. Sanders, L. R. (2001): Improving assessment in university classrooms. College Teaching, 49 (2), 62. Scalzo, C. Y Turner, L. (2007): The effect of experimental learning experiences on management skills acquisition. Developments in Business Simulation and Experiential Learning, vol. 34, pp. 155- 160. Soleil, N. (2000): Toward a pedagogy of reflective learn- ing: lived experience in research and practice. Journal of College Reading and Learning, 31 (1), 73. Vélez, C. (2007): El seminario investigativo. Prentice May. Buenos Aires. Wates, K. & Ritchie, C. M. (2008): A case for experiential learning: using Central Europe as a classroom. Devel- opments in Business Simulation and Experiential Learning, vol. 35, pp. 180- 182. Yu, B. & Chan, P. (2005): Exploring the preference in learning approach among the Hong Kong university students: case study, problem- based or traditional text- book question. Developments in Business Simulation and Experiential Learning, vol. 32, pp. 333- 339. http://campus.usal.es/teoriaeducacion/rev_numero_05/n5_art_gros_adrian.htm http://campus.usal.es/teoriaeducacion/rev_numero_05/n5_art_gros_adrian.htm Page 128 - Developments in Business Simulation and Experiential Learning, volume 38, 2011 APPENDIX: SURVEY 1. What skills do you think are developed with each of the methods explained above? Answer YES if you think the skill is developed with the teaching method, and NO otherwise. Objectives Information Planning Generalist & Specialist Team Work Motivation Participation Feedback Master Class Practical Classes Seminars Tutorials Lectures Teaching in small groups Programmed Teaching Personalized System Projects Company Intern- ships Case Method Business Games 2. Compare the pedagogical effectiveness of business games with other teaching methods. Consider 1 (lowest qualifi- cation) and 5 (highest qualification) and do not repeat the score assigned to each one. LECTURE OR MASTER CLASS SEMINAR OR GROUP DISCUSSION CASE METHOD BUSINESS PROJECT BUSINESS GAME Table of Contents Volume 38, 2011 Simulated Tabletop Exercise for Risk Management - Anti Bio-terrorism Scenario Simulated Tabletop Exercise From Business Games to Simulations - Simuworlds & Microworlds Demand Equation Redux: The Design and Functionality of the Gold/Pray Model in Computerized Business Simulations Managing Client-Based Learning: Insights from Successful Teaching Project Courses in Marketing Tracking Forecast Error Type, Frequency and Magnitude with the Forecast Error Package Responding to Facilitate Collaboration Simulating Sudden Change and the Value of Timely Information Managing Human Resources Simulation A Study on Collectivism and Group Decision-Making: An International Comparison of Japan, China, and Russia Using a Gaming Simulation The Use of Management Games in the Management Research Agenda Gaming On-Line: A Simulation Application Positioning and Performance in Simulated Networks Supply Chain Management: A Simulation Application Simulation as a Teaching Method in Strategic Management Distance Studies Entrepreneurship: A Game of Risk and Reward Phase II: The Start-Up Return to the Paradise Islands: From Confrontation to Cooperation Effect on Market Performance of Displaying Supply and Demand Curves in a Business Simulation Appreciating Complexity: The Chief of Staff of the Army Game Managing Organizations: Experiential MBA Course Teaches Alternatives to the Machine Model A Simulation Model for Analyzing the Night-Time Emergency Health Care System in Japan The Continuing Evoluation of Assessing Project Management as an Academic Learning Outcome (ALO) Should College Instructors Change Their Teaching Styles to Meet the Millenial Student? The Mouse Game and its Effects on Team Interdependence Learning from the Gulf Oil Spill to Prepare for a Brighter Future: A New Game Engaging Stake Holders in Triple Bottom Line Accounting & Strategic Planning Video Killed the Biblio Star: The Impact of Digital Media on Student Learning Outcomes Exploring Motivation: Using Emoticons to Map Student Motivation in a Business Game Exercise An Alternative to PC and Internet Based Simulations: The Internet Integrated Mode MiddleState University -- A Crisis in Education Complexity Avoidance, Narcissism and Experiential Learning Examining the Cognitive, Affective, and Psychomotor Dimensions in Management Skill Development Through Experiential Learning: Developing a Framework A Situational Leadership Exercise Based on the Biology of a Starfish JOGAI CEFET -- The Industrial Administration Undergraduate Game Would You Take a Marketing Man to a Quick Service Restaurant? Modeling Corporate Social Responsibility In A Food Service Menu-Management Simulation Use of a Simulation in a Large Class Environment for a Marketing Principles Class: A Qualitative Analysis of Whether Learning Objectives were Met A Team Based Information Literacy Exercise ABSEL Marketing Communications Plan An Interdisciplinary Study of the Impact of Playing a Marketing Simulation Game on Student Knowledge of Management Accounting/Finance Principles Analyzing Construction Planning of Interiro Finish Work of Apartment Building by Simulation Doing Murder One Again The Simple Business Game and Simulation Transfering the Knowledge of Middle Management to Novices Infectious Disease Simulation Model for Estimation of Spreading Understanding the Relative Influence of Several Factors in ERP Simulation Performance: An Exploration of Ecological Validity Tragedy of the Commons: An Exercise Using Clickers to Illustrate and Teach a Key Concept in Negotiations The Meaning of Firm Demand in Business Simulations If the Games Work, Why Aren't More Faculty Willing to Play? Those Who Do and Those That Don't: A Study of Engaged and Disengaged Business Game Players