From the Classroom to the Laboratory: A Framework Proposal to Design a Management Laboratory Page 119 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 ABSTRACT This work presents a framework to design a management laboratory. It was formulated through the analysis of 19 papers published in the literature using the Proknow-C method. Ten elements were extracted from these works to form the framework. These elements are as follows: Flexibility; Facilitators; Data collect; Team interaction; Feedback, Presentations; Theoretical basis; Realism; Processes and Artifacts. The dynamics of the framework were constituted of the interrelation of these elements. Its main goal is to define the requirements to design a management laboratory. The framework is recommended to educational institutions, and other organizations that intend to use games and simulations as teaching tools in a predominantly face-to-face manner. INTRODUCTION According to the National Institute of Educational Studies and Research Anísio Teixeira (INEP, 2018), the number of students enrolled in higher education management area courses in Brazil in 2016 was 1,212,231 of a total of 8,048,701 students enrolled in higher education institutions in the country. This number represented 15.06% of all students registered in undergraduate courses in higher education in Brazil that year. The number of graduates was even more representative, with 20.67% of the 1,169,449 students graduating in 2016. If the number of students in Accounting and Economics courses are added, with 355,425 and 49,642 enrollments respectively, the percentage of representation of students in higher education in 2016 jumps to 20.09% in Brazil (INEP, 2018). These data shows the representativeness of courses in the management area in Brazil, which justifies studies related to the best forms of teaching, searching better learning ways. A quick analysis in the curricula of the public universities face-to-face management courses in the state of Santa Catarina, (UFSC, 2018; UDESC, 2018) and in the higher education institutions associates from ACAFE (2018) (Santa Catarina Association of Educational Foundations) system in this same state shows that a large part of them use business games, as well as mandatory internships in their teaching program. Both teaching tools aim to bring the student closer to the reality of the job market, linking classroom learning with complex situations of reality. Only 4 out of a total of 16 face-to-face courses analysed do not have the discipline of business games in their curricula. This analysis corroborates the placement of several authors that explain the increasing use of games and simulations, as well as their importance as teaching, learning and researching tools (Lainema & Lainema, 2007; Pittaway & Cope, 2007; Sauaia & Zerrenner, 2009; Bell, Kanar & Kozlowski, 2008; Qudrat-Ullah, 2010; Pasin & Giroux, 2011; Duncan, Miller & Jiang, 2012; Siewiorek et al., 2012; Geithner & Menzel, 2016). According to the authors, one advantage of games and simulations is their ability to create an experience for the students, bringing them closer to reality in a safe environment, where they are allowed to make mistakes unlike in the real world. Even with the dissemination of practices in business games in Santa Catarina States higher education courses, the Federal University of Santa Catarina (UFSC) does not have an adequate environment that provides the physical, technological and technical support to facilitators and students in the use of games and simulations in its management course. Thus, in order to solve this problem, which is shared by other educational institutions, this work proposes the development of a theoretical framework with the goal to design an environment that supports experiential activities based on games and simulations, as well the support to the researches derived from them, based on the management laboratory concept proposed by Sauaia (2010). This work is structured as follows: initially, the theoretical basis will be presented based on the 10 elements identified in the literature that integrates the proposed framework. Secondly, the methodological procedures based on ProKnow-C, which were responsible for the literature review, will be exposed. This will be followed by the structure of the framework proposed which will be presented, as well as the interrelationship of its elements. Finally, the limitations and future work will be discussed. THEORETICAL BASIS The term management laboratory was coined by Sauaia (2010) as the association of organizational simulators, business games and applied research. According to the author, the managerial simulator is the tangible element, being represented by a set of From the Classroom to the Laboratory: A Framework Proposal to Design a Management Laboratory Pedro Primo Bristot Federal University of Santa Catarina pbristot@gmail.com Rolf Hermann Erdmann Federal University of Santa Catarina rolf.erdmann@ufsc.br Page 120 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 economic rules described in a business case with qualitative and quantitative data. The participants assimilate the rules to apply them in a game of companies. The business game is the intangible element, being a decision-making process in cycles, where groups compete for objective results. The applied research, theoretical or empirical, is carried out by the participants, having their primary data collected in the business game activity, and discussed based on the theories that one intends to study. According to the author, this proposal goes beyond the simple use of educational games based on Kolb's experiential learning cycle. For an optimized use of the management laboratory in a face-to-face course, an environment that provides technological support is desired, especially when it is necessary to use computerized simulators. With the objective of identifying the necessary elements to compose an environment that supports the concept of Sauaia management laboratory, a bibliographic survey was carried out in scientific papers. Ten elements were extracted from this survey, which base the framework proposed in this work. The first is the element (i) presentation. This element has been extracted from the literature due to the use of visual presentations verified in the selected works (Lainema & Lainema, 2007; Sauaia & Zerrenner, 2009; Siewiorek et al., 2012; Fitó-Bertran, Hernández-Lara & Serradell-López, 2014; Silva, Oliveira & Leal Junior, 2016; Botelho et al., 2016; Lee, Long & Visinescu, 2016; Huggins, 2017). The presentations were used by the teachers and facilitators with the objective to present the tools (games and simulations) used, as well as to deliver classes of the subjects approached. In the same way, students also used this support to present their progress in games and their group discussions. The (ii) interaction was one of frequent element observed in the analysed literature (Pittaway & Cope, 2007; Bell, Kanar & Kozlowski, 2008; Sauaia & Zerrenner, 2009; Huerta-Wong & Schoech, 2010; Eskrootchi & Oskrochi, 2010; Siewiorek et al., 2012; Konak, Clark & Nasereddin, 2014; Botelho et al., 2016; Geithner & Menzel, 2016; Lee, Long & Visinescu, 2016). Group work, negotiation, face-to-face contact and collective learning are characteristics that should be motivated and facilitated in a management laboratory environment based on the authors ideas. The application of simulations and games as tools for experimental research was verified in the selected papers, and their effectiveness was tested as well. For that, different data collection techniques were used to give substrate to the researches. Techniques such as application of knowledge assessments, participant observation reports and student perceptions reports were the most frequent data collection methods in the analysed studies (Lainema and Lainema, 2007; Qudrat-Ullah, 2010; Siewiorek et al., 2012; Konak, Clark & Nasereddin, 2014; Pittaway & Cope, López, 2014; Lee, Long & Visinescu, 2016; Huggins, 2017). It is important to find ways to facilitate the collection of this data from the laboratory. Thus, (iii) data collection is the third element identified in the literature. The objectives of the use of simulations and games also vary among the analysed works. They range from teaching, training, development and research, with profiles of different people, varied methods and disciplines, and technologies employed. (Lainema & Lainema, 2007; Pittaway & Cope, 2007; Kanar & Kozlowski, 2008; Sauaia & Zerrenner, 2009; Eskrootchi & Oskrochi, 2010; Bell,; Qudrat-Ullah, 2010; Pasin & Giroux, 2011; Siewiorek et al., 2012; Maiti & Tripathy, 2013; Konak, Clark & Nasereddin, 2014; Fitó-Bertran, Hernández-Lara & Serradell-López, 2014; Silva, Oliveira & Leal Junior, 2016; Botelho et al., 2016; Lee, Long & Visinescu, 2016; Geithner & Menzel, 2016). Because that, the analysis of the public that one wishes to attend and the technology of support of the games and simulations that will be employed must be taken into account in order to design a management laboratory. These points base the fourth element found in the literature, (iv) flexibility. The role of the (v) facilitator, the fifth identified element, who conducts the activities in the laboratory, explains the rules of the simulations and games, presents the classes related to the content and interacts with the student was a frequent question in the literature.( Pittaway & Cope, 2007; Botelho et al., 2008; Eskrootchi & Oskrochi, 2010; Long & Visinescu, 2016; Geithner & Menzel, 2016; Huggins, 2017). Thus, a management laboratory should support the facilitators and be prepared to promote their interaction with the participants. Feedback (vi) is another element extracted from the analysed papers. They must be fast, effective and constant. It is important to increase interactivity and immersion, and it is also one of the benefits of the use of games in education (Lainema & Lainema, 2007; Bell, Kanar & Kozlowski, 2008; Qudrat-Ullah, 2010; Eskrootchi & Oskrochi, 2010; Lee, Long &Visinescu, 2016; Botelho et al., 2016; Geithner & Menzel, 2016; Huggins, 2017). Since the attempt to portray reality accurately can make the game extremely complex, (vii) realism is an element that should be treated with parsimony in serious games. On the other hand, the lack of realism can make the simulation less exciting, leading to less engagement, and consequently retention of knowledge, losing the main benefit of experiential learning. (Lainema & Lainema, 2007; Bell, Kanar & Kozlowski, 2008; Sauaia & Zerrenner, 2009; Eskrootchi & Oskrochi, 2010; Qudrat-Ullah, 2010; Lee, Long & Visinescu, 2016). In this way, an environment that seeks to simulate the reality without excesses and that elevates the immersion of serious games has the potential to elevate the retention of knowledge and to potentialize the use of experiential learning in teaching. A laboratory that considers these characteristics must be different from a simple classroom with computers. It must transport the students to the place of execution of their future professional activities. The (viii) theoretical basis of learning used to construct a class or serious games drives certain activities to be performed, which demands an appropriate structure. The theoretical basis most found in the this research was the experiential learning based on the Kolb's cycle, which is in line with what is proposed in a management laboratory with the characteristics verified in this paper (Lainema & Lainema, 2007; Pittaway & Cope, 2007; Abdulwahed & Nagy, 2009; Eskrootchi & Oskrochi, 2010; Huerta-Wong & Schoech, 2010; Duncan, Miller & Jiang, 2012; Duncan, Miller & Jiang, 2012; Konak, Clark & Nasereddin, 2014; Botelho et al., 2016;; Geithner & Menzel, 2016; Silva, Oliveira & Leal Junior, 2016; Lee, Long & Visinescu, 2016). Clearly described processes make it easier for different instructors, teachers and researchers to use a laboratory. Besides solving problems related to the equipment usage, physical space and simulations, a space management system is desired. This management system could define the ones who are responsible for the laboratory, its schedule, among other issues, according to the characteristics of the institution that will operate it. (Abdulwahed & Nagy, 2009; Eskrootchi & Oskrochi, 2010; Huerta-Wong & Schoech, 2010; Qudrat-Ullah, 2010; Maiti & Tripathy, 2013; Lee, Long & Visinescu, 2016; Geithner & Menzel, 2016). In this way, the element (ix) process appears in the literature with the objective to ensure the proper functioning of the laboratory structure and its file:///C:/Users/Home/Documents/ABSEL/2019/Raw/Innovations/ABSEL2019_Innovations_8.docx#230#230 Page 121 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 management. The (x) artefacts, our tenth and last element, are physical and technological elements that can compose a management laboratory. They range from computers, internet access, software, database, among others components. (Lainema & Lainema, 2007; Bell, Kanar & Kozlowski, 2008; Abdulwahed & Nagy, 2009; Eskrootchi & Oskrochi, 2010; Qudrat-Ullah, 2010; Pasin & Giroux, 2011; Siewiorek et al., 2012; Maiti & Tripathy, 2013; Konak, Clark & Nasereddin, 2014; Fitó-Bertran, Hernández-Lara & Serradell- López, 2014; Silva, Oliveira & Leal Junior, 2016; Botelho et al., 2016; Lee, Long & Visinescu, 2016; Botelho et al., 2016; Geithner & Menzel, 2016). The artefacts are the result of the choice within the other elements, being the physical representation of the needs of the organization that are developing the laboratory, through the analysis of the previous elements. Framework is another important definition in the present work. According to the Cambridge Dictionary (2018), the term framework can be defined as a support structure in which something can be built around, or a system of rules, ideas, or beliefs that is used to plan or decide something. The term framework is used in several disciplines with different purposes and styles of presentation. It is a way of representation and approach in the administration, whose form depends on the subject in question (Shehabuddeen et al., 2000). According to these authors, there is no standard definition for the term. However, a framework should generally have the following characteristics: (i) Represent an issue for a specific purpose; (ii) Connect several elements and show their relationships; (iii) Allow a holistic view of a situation to be captured; (iv) Demonstrate a situation or provide a basis to solve a problem; (v) Provide a structured approach to deal with a particular problem. In this way, Shehabuddeen et al. (p.9, 2000) defines framework as: "The framework supports understanding and communication of structure and relationship within a system for a defined purpose." Miles and Huberman (1994), in turn, define a conceptual framework as capable of explaining, graphically or in narrative, the most important issues to be studied - key factors, constructs, or variables - and presuming relationships between them. They can be rudimentary or elaborate, guided by theory or common sense, descriptive or causal. Shehabuddeen et al. (2000) states that it is widely used in management disciplines to translate complex issues into a simpler analysable format and it is generally used to: (i) Communicate ideas and findings from academics to academics or academics to industry; (ii) Make comparisons between different situations or approaches; (iii) Define the domains and frontiers of a situation; (iv) Describe a context or discuss the validity of a finding; (v) Support the development of procedures, techniques, methods and tools. METHODOLOGICAL PROCEDURES The Proknow-C method was used to select the article portfolio, which gave rise to this work. Proknow-C is a structured process of bibliographic review, whose acronym stands for Knowledge Development Process - Construtivist, which describes and presents a process for building the knowledge needed to investigate and analyse a theme (Ensslin et al., 2014). With a constructivist approach, it has the capacity to generate the researchers knowledge to guide them in investigations about the proposed theme (Marafon et al., 2012). It was developed by researchers from the LabMCDA (Laboratory of Multicriteria Methodologies in Support of Decision from the Federal University of Santa Catarina), aiming at guiding the researchers to select the relevant papers to the research subjects. Its first scientific publications occurred at the end of 2009. The Proknow-C was being used as a bibliographical survey tool in several subjects, such as performance evaluation of environmental disclosure (Rosa, Ensslin & Ensslin, 2009), the evaluation of performance in the management of R & D (Marafon et al., 2012), performance evaluation in general (Rosa, Ensslin & Ensslin, 2013), and performance evaluation in public university administration (Valmorbida, Ensslin & Ensslin et al., 2015), performance evaluation of energy efficiency implementation processes (Ensslin et al., 2014), performance evaluation of the quality of banking service (Ensslin, Ensslin & Pinto, 2013). The present research can be represented as follows. The ProKnow-C is a multi-step structured process that has been applied to this research. Ensslin et al. (2014) divides the procedures into three stages: (i) Preliminary investigation; (ii) Selection of articles that will compose the research portfolio; and (iii) Bibliometric analysis of the portfolio. For this work, steps (i) and (ii) were performed. In the preliminary investigation stage, key words were defined by reading articles from Simulation and Gaming Journal (Sullivan, 1971; Umpleby, 1971; Barton, 1972; Anderson, 1982; Chin, 1989). The databases were selected from the available ones EXHIBIT 1 SEARCH REPRESENTATION USING THE PROCKNOW-C Page 122 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 from UFSC and the Capes Journal Portal. The research was carried out based on the following parameters: the articles should be at least 10 years old; the key words needed to appear in the title, abstract or keywords of the articles. Two search rounds were made. The first with the first group of words and the second with the second group. The survey of the papers in the databases took place between 6/10/2017 and 15/12/2017. With these parameters, a base of 4340 raw papers were obtained, which went to the selection stages. For the selection of the final portfolio, the duplicate works were initially withdrawn, and a selection was made by the title aligning with the non duplicated works. The second selection point was the exclusion of papers that were not in English, Portuguese or Spanish, and the works that were not classified as a paper, like thesis or other kinds of work that were collected from the data bases. The scientific recognition from the selected papers were verified through the citations on Google Scholar®. There were 10 citations cut-offs in the first survey of key words. The papers with higher than 0.5% of the total citations were selected for the second survey, with the article with the lowest number scoring 19 citations. From the papers with low representativity that were previously discarded, those that had been published in the year 2016 onwards, and those in which the authors had been selected in the scientific recognition selection were re-analysed. These papers were joined with the scientific recognition selection. From the remaining papers, the abstracts were read to check their alignment with the EXHIBIT 2 SURVEY OF RAW PAPERS EXHIBIT 3 PAPER PORTFOLIO SELECTION Page 123 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 theme, followed by the search for the work in its entirety. With the complete works in hand, they were fully read, reaching a final portfolio of 19 works. The EndNote X7® software support was used for this process. The 19 papers selected were read in full (Lainema & Lainema, 2007; Pittaway & Cope, 2007; Bell, Kanar & Kozlowski, 2008; Abdulwahed & Nagy, 2009; Sauaia & Zerrenner, 2009; Eskrootchi & Oskrochi, 2010; Qudrat-Ullah, 2010; Huerta-Wong & Schoech , 2010; Pasin & Giroux, 2011; Siewiorek , et al. 2012; Duncan, Miller & Jiang, 2012; Maiti & Tripathy, 2013; Konak, Clark, Nasereddin, 2014; Fitó-Bertran, Hernández-Lara & Serradell-López, 2014; Silva, Oliveira & Leal Junior, 2016; Botelho et al., 2016; Geithner & Menzel, 2016; Lee, Long & Visinescu, 2016; Huggins, 2017). The extraction of representative passages referring to the needs of the use of games, simulations as a management laboratory in the concept of Sauai (2010) was carried out as well as the structural and management needs of a physical environment for such. These passages were grouped by affinity, thus conceiving the 10 elements that theoretically base the framework for the design of a management laboratory. With the definition of the elements, they were organized in a similar structure to the one proposed by Duncan, Miller & Jiang (2012), being later interrelated, presenting the final dynamics of the framework proposed in this paper. FRAMEWORK STRUCTURE AND INTERACTION Inspired by the work of Duncan, Miller & Jiang, (2012), who performed the taxonomy of virtual worlds in six categories answering the questions: Who? What?; Why?;Where?; How?; and research areas; the 10 elements raised in the literature were grouped in order to answer these 5 questions, arriving at the following configuration according to exhibit 4. The Who? question refers to the audience, for whom the laboratory is intended. It includes the Flexibility element, since the laboratory can be used to teach higher level students through simulations and serious games, as a tool for research, games and simulation development, or even for training professionals. The specifications of the public also request the technology that is intended to use computer games, no-computer games or both. Due to that, the laboratory should have different configurations. The Facilitator element also helps to answer the question who? It is also a target audience and according to the literature analysed, it should participate directly in the activities developed in it. Teachers, researchers or instructors can play this role. The question What? should be answered with support activities to be developed in an environment for simulations and games. In this group, the following elements are presented: Data Collect; Team Interaction; Feedback and Presentation. The Data collect element comprises data collection for research and student learning assessment, and it can be broken down into simple questionnaires and exercises, to participant observation reports. If the laboratory is also targeting the development of the research, it should support activities of this nature. Regarding the element of Team Interaction, it must be promoted, either face-to-face or through communication technologies. The Feedback and Presentations elements are constant activities in games and simulations, as verified in the literature. According to the literature, a management laboratory, focused on activities of this nature, should provide support for fast and constant feedback, as well as a suitable structure for presentations by facilitators and learners, as it is part of the EXHIBIT 4 THE 10 FRAMEWORK ELEMENTS ORGANIZATION Page 124 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 learning process. The question Why? is related to why certain activities are being carried out (Duncan, Miller & Jiang, 2012). The element related to this question is the Theoretical Basis element. As verified in the literature, Kolb's experiential learning cycle is the most used conceptual basis in the construction of tools and classes in the analysed works. However, Duncan, Miller, Jiang, (2012) identify other theoretical basis that can also be developed in a management laboratory in their taxonomy. In Kolb's cycle case, which comprises four phases: concrete experience, reflective observation, abstract conceptualization and active experimentation (Kolb & Kolb, 2008), each of these phases needs to be activated, and different activities must be developed for this goal. If the laboratory is developed having Kolb's cycle as the theoretical basis, it should have to provide structure for the activation of its four phases. The question Where? refers to the place where the activities are developed, it is associated with the Realism element. This element means that the laboratory should reflect the reality of the environment it is willing to represent, unlike a common classroom. The physical environment of the laboratory should seek to be realistic in relation to the disciplines taught, making the experience of the simulation or game even more real. Finally, the question How? is answered with the systems and supports used in the laboratory. For that, the elements Processes and Artifacts, which comprise the physical, technological and procedural elements that must be applied to the laboratory materialize the definitions made in the other elements. For instance, the need for computers, specialized software, non-computerized game materials and others. In addition to the structure of elements, they interrelate, where choices made in a given element will condition the choices within another element. This interaction gives the dynamics of the proposed framework. Exhibit 5 shows this interaction. According to exhibit 5, the interaction of the elements is initially given when defining the elements flexibility and facilitators. It is defined according to the purposes and audience the laboratory intends to attend, as well as the professionals who must use it. The laboratory can have undergraduate students, postgraduate researchers in the management disciplines as audience for example, and the professors of theses disciplines as facilitators. Another point to be defined is the technology required, such as computerized or non computerized games and simulations, both, or other kind of activity to be developed. With the target audience and technologies defined, the theoretical basis should be chosen, which will be the background for the definitions in terms of feedback, team interaction, data collect and presentation. Feedback could be computerized, through physical panels, orally, on the blackboard or others. The team interaction can be either face-to-face, through information technology, or even both. Data collection will depend on the type of evaluation and what the research methods require from the lab. The presentation element could be materialized by Power Point, blackboard, flipcharts, using microphones and sound speakers for example. After the alignment of these elements, processes and artefacts are defined, guided by the search for realism. The artefacts must represent the choices defined in the previous elements physically and technologically. For example, if it is decided that the laboratory should support research with observational techniques, the implantation of recording cameras can be considered as laboratory artefacts, as well as computers, software for conferencing in interactions through information technology, and others. Realism is also part of the artefacts, since the layout, and the decoration of the environment can create a realistic atmosphere. The processes are defined for a better use of the environment, its maintenance and management. CONCLUSIONS, LIMITATIONS AND FUTURE WORKS Based on 10 elements obtained from literature which are interrelated, it is possible to define the requirements for design a management laboratory with the framework proposed by this work. This laboratory comprises the physical space for the development of the activities of the conceptual management laboratory of Sauai (2010). Educational institutions or any other organization which wish to use games and simulations as a teaching, training or EXHIBIT 5 INTERACTION BETWEEN FRAMEWORK'S ELEMENTS Page 125 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 research tool, can build a laboratory that supports these activities more effectively with this framework support. With a prepared and thoughtful environment, a better learning performance is expected with the use of these tools. Thus, avoiding the use of games, specially the computerized ones in not prepared classrooms hinders the use of the games and the participants interaction. Another contribution of a management laboratory as propose is to support the different activities that involve the development, testing and use of games and simulations. If it was used in universities, it would serve as an embryo for the dissemination of theses teaching and research tools in various disciplines. It would assist teachers who wish to improve their classes with activities based on experiential learning. It is expected to encourage the use of game-based teaching methods and simulations as well as an increase in researches related to the topic. However, this framework has some limitations. The first is that it is a theoretical framework based on literature, and it has not been criticized by experts in the field. A second limitation is the lack of an empirical test in the management laboratory design. Thus, validation with experts and their empirical use is proposed as future work. The result of these works would generate increments and improvements, validating the framework empirically. Another suggestion as future work, in terms of learning, is the comparison of the use of games and simulations in an environment built with the premises of the framework proposed here, and using them in a common classroom environment or computer labs, as it happens today in administration courses in Brazil. ACAFE. (2018). Instituições. Retrieved 3 October, 2018, from http://www.new.acafe.org.br/instituicoes/ Anderson, D.E. (1982). Computer Simulations in the Psychology Laboratory. Simulation & Games, 13 (1), 13-36. Barton, R.F. (1972). Designing Simulation Laboratories. Simulation & Games, 219-225. Bell, B.S, Kanar, A.M & Kozlowski, S.W.J. (2008). Current issues and future directions in simulation-based training in North America. The International Journal Of Human Resource Management, 19(8), 1416-1434. Botelho et al. (2016). Kolb's Experiential Learning Theory and Belhot's Learning Cycle Guiding the Use of Computer Simulation in Engineering Education: A Pedagogical Proposal to Shift Toward an Experiential Pedagogy. Computer Applications in Engineering Education, 24(1), 79-88. Cambridge dictionary. (2018). Framework. Retrieved 11 July, 2018, from https://dictionary.cambridge.org/pt/ dicionario/ingles/framework Chin, J. (1989). Sociology Laboratory. Simulation & Games, 501-505. Duncan, I, Miller, A & Jiang, S. (2012). A taxonomy of virtual worlds usage in education. British Journal Of Educational Technology, 43(6), 949-964. Ensslin, L, Ensslin, S.R & Pinto, H. de M. (2013). Processo de Investigação e Análise Bibliométrica: Avaliação da Qualidade dos Serviços Bancários. Rac - Revisa de Administração Contemporânea, 17(3), 325-349. Ensslin, S.R. et al. (2014). Processo estruturado de revisão da literatura e análise bibliométrica sobre avaliação de desempenho de prcessos de implementação de eficiência energética. Revista Brasileira de Energia, 20 (1), 21-50. Eskrootchi, R & Oskrochi, G.R. (2010). A Study of the Efficacy of Project-based Learning Integrated with Computer based Simulation - STELLA. Journal Of Educational Technology & Society, 13(1), 236-245. Fitó-Bertran, A, Hernández-Lara, A.B & Serradell-lópez, E. (2014). Comparing student competences in a face-to -face and online business game. Computers In Human Behavior, 30(1), 452-459. Geithner, S & Menze, D. (2016). Effectiveness of Learning Through Experience and Reflection in a Project Management Simulation. Simulation & Gaming, 47 (2), 228-256. Huerta-Wong, J.E & Schoech, R. (2010). Experiential learning and learning environments: the case of active listening skills. Journal Of Social Work Education, 46(1), 85- 101. Huggins, S. (2017). Practice-Based Learning in Higher Education. Library Trends, 66(1), 1-12. INEP. (2018). Sinopses Estatísticas da Educação Superior – Graduação 2016. Retrieved 3 August, 2018, from http://portal.inep.gov.br/sinopses-estatisticas-da- educacao-superior Kolb, A.Y. & Kolb, D.A. (2008). Experiential Learning Theory: A Dynamic, Holistic Approach to Management Learning, Education and Development. InArmstrong, S.A & Fukami, C.V (Eds), The Sage Handbook Of Management Learning, Education And Development (pp. 42-68): SAGE Publications Ltd. Lainema, T & Lainema, K. (2007). Advancing Acquisition of Business Know-How: Critical Learning Elements. Journal Of Research On Technology In Education, 40(2), 183-198. Lee, H, Long, J & Visinescu, L.L. (2016). The Relationship between a Business Simulator, Constructivist Practices, and Motivation toward Developing Business Intelligence Skills. Journal Of Information Technology Education: Research, 15(1), 593-609. Maiti, A & Tripathy, B. (2013). Remote laboratories: Design of experiments and their web implementation. Journal of Educational Technology & Society, 16(3), 220-233. Marafon, A.D.et al. (2012). Revisão sistêmica da literatura sobre avaliação de desempenho na gestão de p&d. Revista Gestão Industrial, 8(3), 1-43. Miles, M.B & Huberman, A.M. (1994). Qualitative Data Analysis. (2nd ed.): Sage Publications. Pasin, F & Giroux, H. (2011). The impact of a simulation game on operations management education. Computers & Education, 57(1), 1240-1254. Pittaway, L & Cope, J. (2007). Simulating Entrepreneurial Learning. Management Learning, 38(2), 211-233. Qudrat-ullah, H. (2010). Perceptions of the effectiveness of system dynamics-based interactive learning environments: An empirical study. Computers & Education, 55(3), 1277-1286. Rosa, F.S da, Ensslin, S.R & Ensslin, L. (2009). Evidenciação Ambiental: Processo Estruturado de Revisão de Literatura Sobre Avaliação de Desempenho da Evidenciação Ambiental. Sociedade, Contabilidade e Gestão, 4(2), 24-37. REFERENCES Page 126 - Developments in Business Simulation and Experiential Learning, Volume 46, 2019 Rosa, F.S da, Ensslin, S.R & Ensslin, L. (2013). Avaliação de desempenho: Processo de revisão sistêmica de literatura internacional. Revista Produção Online, 13 (2), 390-416. Sauaia, A.C.A & Zerrenner, S.A. (2009). Jogos de Empresas e Economia Experimental: um Estudo da Racionalidade Organizacional na Tomada de Decisão. Rac - Revisa de Administração Contemporânea, 13(2), 189-209. Sauaia, A.C.A. (2010). Laboratório de Gestão: Simulador Organizacional, Jogo de Empresas e Pesquisa Aplicada. (2nd ed.). Brazil: Manole. Shehabuddeen, N. et al. (2000). Representing and approaching complex management issues: part 1 – role and definition. England: Working Paper University Of Cambridge. Siewiorek et al. (2012). Learning leadership skills in a simulated business environment. Computers & Education, 58(1), 121-135. Siewiorek, A. et al. (2012). Learning leadership skills in a simulated business environment. Computers & Education, 58(1), 121-135. Silva, S.S. .D.A, Oliveira, M. .A & Leal junior, I.C. (2016). Modelo de previsão de vendas em jogos de empresas: potencializando a prática dos gestores. Revista Pensamento Contemporâneo em Administração, 10(2), 50-64. Sullivan, J.J. (1971). The Economics Laboratory at UCBS. Simulation & Games, 1(1), 81-89. UDESC. (2018). Plano de Ensino Administração Empresarial. Retrieved 3 October, 2018, from http:// www.esag.udesc.br/?id=1497 UFSC. (2018). Currículo do Curso de Administração. Retrieved 3 October, 2018, from http:// cagr.sistemas.ufsc.br/relatorios/curriculoCurso? curso=316 Umpleby, S. (1971). The teaching computer as a gaming laboratory. Simulation & Games, 5-51. Valmorbida, S.M.I, Ensslin, S.R & Ensslin, L. (2013). Avaliação de Desempenho na Administração de Universidade Pública: Análise Bibliométrica da Literatura Nacional e Internacional. Administração Pública e Gestão Social, 5(3), 116-125. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001