Long-Term Experience with A Tool for Large-Scale Dissemination of Entrepreneurial Culture Page 21 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 ABSTRACT This article presents the experience in the construction and application of a business game competition developed in the Business Incubator of COPPE/UFRJ, whose purpose is to stimulate the large-scale spread of entrepreneurial culture. Besides reaching a broad audience, the approach promoted the communication of business concepts, provided robust experiences, exposed fundamental business characteristics and encouraged the participation in the entrepreneurial scenery. The number and diversity of people involved, over sixteen years, in the design of the Incubator's business games, and the variety of techniques and results, give this tool a significant uniqueness in the entrepreneurship education field. INTRODUCTION Over sixteen years, the Business Incubator of COPPE/ UFRJ developed and operated a large-scale business game competition that provided the experience of managing a company, to present, promote and cultivate the entrepreneurial culture in Brazil and other countries. Since 2000, the Incubator has an area focused on experiential education, responsible for the development and application of various games for students, teachers, business people, entrepreneurs and the general public interested in entrepreneurship. Moreover, because of its integration with the community of Federal University of Rio de Janeiro, this project has the participation of researchers in various fields, such as design, software engineering, and production engineering. The long term application of this business game in a competition format came true through a partnership with the following institutions: SEBRAE, from 2000 to 2012, with the participation of more than one million students from more than two thousand high education institutions in Brazil and in eight countries in Latin America; ANPROTEC, from 2013 to 2014, which provided it to business incubators managers throughout Brazil, and Oi Internet, in 2015, to the general public. During this period, the business game received different names, according to the supporting partner: Desafio SEBRAE (SEBRAE Challenge), Avance! (Advance!), and Oi Entrepreneurs. Currently, it has a new version, in a proprietary format, named Formula E². This name intends to denote the Laboratory's attributes, Entrepreneurship and Education. Formula E² found in the Business Incubator a design and development environment quite advantageous. That was due to the Business Incubator parallel academic activities of teaching and researching entrepreneurship, as well as its extensive practical experience in the selection, among hundreds, and guidance of ninety companies, created in its facilities and that now offer qualified employment to over a thousand people. In this context, the exploration of Formula E² history as a communication and awareness tool to build a culture focused on entrepreneurship and innovation is not only relevant but also advantageous for future works in game development for educational purposes. The paper is organized as follow. In the next section, we make a brief description of the Formula E2 game competition environment and its structure. In the section that follows, we expose the educational goals of the game. In the fourth section, the concern about the simulation model quality and the efforts for its improvement over the years are presented. The fifth section shows the aspects that contributed to the success and longevity of the game. And finally, we present our conclusions and future work suggestions. FORMULA E2 According to Iuppa and Borst (2010), serious games are designed to achieve the following objectives: transferring and strengthening knowledge and skills; persuading, through specific techniques and content, to cause social or personal behavior change. Formula E² is a serious game, which immerses participants Long-Term Experience with A Business Game Competition for Large-Scale Dissemination of Entrepreneurial Culture Luís Fernando Oliveira Universidade Federal do Rio de Janeiro, Brazil lfvsmo@cos.ufrj.br Geraldo Xexéo Universidade Federal do Rio de Janeiro, Brazil xexeo@cos.ufrj.br Eduardo Mangeli Universidade Federal do Rio de Janeiro, Brazil mangeli@cos.ufrj.br Claudio D’Ipolitto InoveLab Innovation Atelier, Brazil claudiodipolitto@inovelab.net Page 22 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 in a virtual environment of business competition, managing their own business. The game exercises the tangible and intangible dimensions of the production and sale of industrial products, as it incorporates the administration of a factory and retail stores. The participants start managing a company that manufactures a unique product, which changes for each competition, with the possibility of different models, whose production may be allowed during the game. For that, players need raw materials, workers, money, advertising, etc. The choices connected with these requirements compose a set of decisions for the company's management in different perspectives: strategy, human resources, production, logistics, research & development, and marketing. In the aforementioned competitions, participants group themselves in teams, often interdisciplinary, to enrich their experiences with the discussion that permeates every decision. The competitions were divided into phases, in order to promote the educational character (there was no elimination in the first two phases) and the character of tournament, in two or three phases, in which there was contention for prizes offered each year. The teams rely on the help of various information sources, some generic, related to business theory or the game itself, other directly reporting the situation of their company or the market inside the game. Some of these texts present a dialogical character, encouraging reflection and joint construction of knowledge. The list of information sources is: FIGURE 1 SOME SCREENS OF FORMULA E2 Page 23 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 FORMULA E2 GOALS Formula E2 aims to develop players’ competencies in three dimensions: cognitive, attitudinal and operational. The game is an experiential learning platform that encourages "learning by doing" that, according to Hoover (1974), is something that exists when a participant processes knowledge, skills, and/or attitudes, articulating cognitive, affective and behavioral dimensions in a learning situation characterized by a high level of active involvement. Thus, the creation of knowledge, fostered by the dynamics of the tool, happens through the transformation of experience in a four-step cycle, as described by Kolb (1984): solid experience; observation and experimentation; training concepts; tests in new situations. The members of each entrepreneurial team, responsible for managing a virtual company, live the concrete experience of taking, for example, a decision that involves production, pricing, and distribution. When they receive and analyze the results at the conclusion of the round, these virtual managers should observe reflectively the consequences of their decisions in respect of the decisions of the competing teams and socioeconomic conditions simulated by the game environment. They realize then abstractions of the concepts underlying the observed conditions and restart the cycle of active and concrete trials, which feedback the process of experiential learning. Thus, it tries to achieve the ultimate aim of the project, since it is meant that managing a project helps contributes to developing the participants with the knowledge of management and entrepreneurial attitudes, which will be useful for their future activity as an employer or as an employee, but above all, as an entrepreneur citizen. The base of this position is the educational references of SEBRAE (Wickert, 2006), which states that: "the management of companies may incorporate the new way of thinking, valuing intellectual capital and shared knowledge, understood as a sum of powers of individuals and corporation. Intellectual capital involves skills of the following fields: Cognitive: explicit knowledge (set of concepts, facts, and theories seized and incorporated by mobilizing cognitive schema) and tacit (personal construction through experiences and interpretations). Attitudinal: attitudes, values, creativity, intuition, emotion, feelings, self- esteem and interrelationships. Operational: experiences, productivity. Contemporary society requires entrepreneurial people, autonomous, with multiple skills, who can work as a team, have the ability to learn and to adapt to new and complex situations, to face, unceasingly, new challenges and to promote transformation. Due to this fact, education occupies a strategic and important position not only in the economic and social field but the preparation process of people. The educator's role is to provide a learning environment, using all the resources that technology and science provide for quick access to relevant information, to select them and to work with them to facilitate the construction of personal knowledge and development of entrepreneurial skills". (pp. 21-22) FIGURE 2. LIST OF INFORMATION SOURCES Page 24 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 A CONSISTENT MODEL One of the valuable achievements of this project is the resulting business model, which, during its lifecycle, has been constantly tailored to target the evolving requirements. We can understand a model as a compact representation of reality with an appropriate purpose. For example, for an ID card, just the picture of the citizen is enough. One could imagine the use of a 3D depiction of the person in question, but the representation through the photo meets the purpose of identifying, and its reduction requires a lower energy expense. The success achieved by the Formula E2 is largely the result of the uninterrupted use of the PDCA (Plan-Do-Check- Act), continuous improvement method (Moen, 2009), together with the care of keeping the developer team updated with the state of the art regarding technology, education and construction techniques of serious games, typical of a university environment. Also, being subject to a varied audience of over one million people was essential to the improvement of this tool over the years. That means that the current model of the tool, designed in 2000, has gone through several adjustments to achieve, through the balance between reduction and deployment, the level of efficiency according to proposed objectives. The ‘SEBRAE Challenge’ project reached, according to independent audits, high levels of satisfaction among participants, encouraging the entrepreneurial spirit in these young people and offering them a rich experience on the challenges of managing a company. Designed specifically for an academic audience, ‘SEBRAE Challenge’ showed an increase in participation over the years, shown in the chart below. It is worth saying that 2000 was a testing year, with the private participation of UFRJ students. During the following years, until 2006 inclusive, there were restrictions on the FIGURE 3. THREE DIMENSIONS OF FORMULA E2 COMPETENCIES Page 25 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 number of participants, since players received the software in CDs. With the evolution of Internet usage, it was possible to remove this restriction, increasing participation numbers. Linked to the excellence of the game, the disclosure and the power of capillary penetration in the audience by SEBRAE contributed significantly in the use of this tool, which obtained high acceptance levels and, consequently, the dissemination of entrepreneurial culture. From 2009 to 2011, SEBRAE commissioned an independent audit to assess the effectiveness of Formula E2’s methodology as a learning tool and its quality as an experience. The audit used focus groups and user surveys established that over 80% of the users judge both factors as good or excellent, as shown in Figure 5. Another interesting indicator that reinforces the game's value is the amount of academic papers that have arisen linked to it, showing the relevance of the methodology used and discussing the need for greater integration of games in education. Below, we present some examples: According to Alves, Haydu and Souza (2010): "in this particular context, we observe the application of the games and their use in the classroom. One of the best and most studied examples to be cited is the technic of business games in administration, which has offered a valuable alternative for the consolidation of the teaching-learning process of business management, favoring experimentation and influencing the motivation and interest in learning. We can cite as a good example the game SEBRAE Challenge, a virtual game that simulates the business everyday in a company". (p.164) To Caixeta (2008): "The growing use of game companies as an alternative teaching method has led students to put the theory experienced in a classroom into practice. That helps in the learning process, but in a different way, stimulating the construction of knowledge through discovery, what leads these students to think, to reflect, to create alternatives, to interpret and make the decisions that they judge to be the most proper. The practice of SEBRAE Challenge business game is also a skill development method, such as leadership, teamwork, ability to make decisions, creativity, and the much- needed entrepreneurial spirit. It complements in a satisfactory manner the theory studied in class, expanding the education, both academically and professionally, improving student's performance, enabling him to act more safely and responsibly during practical application". (p.9) Costa and Pastana (2015) present: "a case study from the research on 'Game Impact of SEBRAE Challenge Companies in Young College Students from Macapá State' through which were carried out all analyses that gave us the base to test the hypothesis that young people who participate in training with business game have greater chances of success in the business market or work than those who do not participate in this type of training."(p.1) According to Castro et al. (2014): "the present study, with a detailed character, was operated using data collection with students from Piauí (a Brazilian federal state), who participated in the business game SEBRAE Challenge. In this aspect, the FIGURE 4 NUMBER OF PARTICIPANTS OF SEBRAE CHALLENGE PER YEAR IN BRAZIL Page 26 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 objective of this study was to verify if the business game SEBRAE Challenge has significant influence so that the Piauí college student may get an entrepreneurial vision. Considering that entrepreneurial application of games and simulations in schools and educational institutions that will educate future market professionals helps significantly to their education, it could be realized, from the results of the research, that the virtual game SEBRAE Challenge fulfills its role, not only completing the gap not filled by the standard university education system, but also motivating the entrepreneurial culture among the students through practice incursions that market simulations offer." (p.104) COMPILATION OF FAVORABLE FACTORS The longevity and extent achieved by the Formula E2 project can be seen as a proof of its success. We believe that this success has three cornerstones: a robust and appropriate technological tool, a solution that supports a heterogeneous audience, and good advertisement. As the issue of advertisement is subject to other studies, we focus the following exposition on the other elements that contributed to the success of the game. Thus, the following topics show details of greater relevance to the success of the project, not ranked in order of importance since all are essential elements for the tool. TOPIC A - CONTENT The quality of the content, the variety of themes, the proposed levels of deepening, the form of the game, the FIGURE 5 RESULTS OF INDEPENDENT AUDIT EVALUATIONS BY PARTICIPANTS OF SEBRAE CHALLENGE FROM 2009 TO 2011. FIGURE 6 “GO BEYOND” EXAMPLES. Page 27 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 combination of the matters related to each area, the continuous improvement of material, the dialogue with the participant, the structuring of reports with their hermeneutical guidelines, the presentation of case studies and the care in the assistance process to reflect on the consequences of the performance of each participant were useful effort lessons to the entrepreneurial culture. Here's a whole condensate program of development of game narrative and dialogue between the participant and the concepts. As examples of content worked in Formula E2, we can mention: business models; the concrete process of formation of business strategies; organizational structure and management model; production/operations management; operations strategy; lean production; marketing management; consumption value attributes; financial management; cash flow; balance sheet; team management; innovation management; characteristics of entrepreneurs; entrepreneurial mindset; strategy; internationalization; exportation; merger or acquisition; franchises; barriers to internationalization; intellectual property; patents. In addition to the content presented, the game also provides a "go beyond", i.e. references for participants who seek more knowledge about a particular subject. TOPIC B - BALANCING The balancing of the elements in the game has been improved over the years. The game environment is part of the immersion in the process of incorporation of the participant in the virtual world, without taking the focus (context and history). Fun part is required in the learning process in a game, but should not be excessive (Playfulness). Characters assists in the "humanization" of the learning process. Outputs are required for evaluation of learning (results and clear feedback). In the game, the knowledge is sought by the participant, and not pushed by the teacher (Intense cognitive activity). The sense of overcoming is one of the success factors for this learning process (challenge). Is important to have highly qualified content, distributed throughout the game to facilitate assimilation, as described earlier (relevant content). The teamwork and between teams makes learning more intense and compelling (interactivity). The sense of Justice is an active element of the engagement of the participants (rules and clearly defined structure). Is a game, but, at the same time, the player faces like your own business (a fictional world, but believable). In Figure 7, the dark color represents the percentage of use of each element in the Formula E2. This balancing was made possible by the returns of the participants, through natural ways, by research and by focus groups conducted with participants and teachers in several cities in Brazil. It is interesting to know that a significant effort focused on the artistic and technical production of the project was, gradually, turning into energy used in modeling of feedback and the development and the structuring of the content. TOPIC C - DIVERSITY OF EVALUATION CRITERIA The game has ten evaluation criteria: a) market share of revenue; b) market share of quantity sold; c) profits distribution; d) profit/loss generated; e) profitability; f) sales management; g) capacity management; h) inventory management; i) cost management; j) marketing management. The perception of those objectives in different areas develops a systemic vision at the participant, with well-defined goals, measurable, achievable, realistic and proportionate to the scale of the game. That presents an overview of the effects caused by the decisions of the companies in the virtual market in which they find themselves. The shortcomings became patent, instigating the research for the understanding of the game model, founded on the concepts we want to transmit. TOPIC D - UPDATED MODELING AND WELL DOCUMENTED A great difficulty, during these 16 years, was to keep the game model documented in the way to allow evolution and adaptation to different situations. In this case, all the rules of the project were recorded accordingly to their nature, keeping the model formulas in spreadsheets. In this way, the changes in the project remained consistent, and the evolution occurred in a FIGURE 7 BALANCING OF GAME ELEMENTS. Page 28 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 linear way, according to the understanding of the balance. Another crucial point is that the philosophy of business game development implies in to model, assess and support relevant skills through assessment mechanisms embedded in environments of games aimed at the active learning. The evaluation methodology is an extension of the "Evidence- Centered Design" (Shute, Masduki, Donmez 2010), and has as its primary purpose to collect information allowing to make inferences about the state of students' skills (what they know, what they can do and to what extent) to support educational decisions that promote learning. The reference model integrates three models: (a) define the requirements for the student's skills; (b) establish what constitutes a valid evidence of these demands and (c) determine the nature and the form of tasks that will describe such evidence. The Model of Competences asks "what kind of collection of skills, abilities, and other attributes we must evaluate?". The Model of Evidences asks "which behaviors or performances reveal different levels of target skills?". The Model of Task asks "which tasks should clarify those behaviors that comprise the evidence?". This model of "Evidence-Centered Design" is consistent with the functional requirements of a serious game, namely: focus on problem-solving and decision-making, emphasis on learning, simulation of "realistic" fictional situations (verisimilitude), "imperfect" communication (allows ambiguity and subjective interpretations). The competencies, evidence, and tasks must meet a composite of expectations, as shown in Table 1 (adapted model of Faria and Wellington, 2004). Such expectations should reflect the specific requirements of the product in focus, considering their use in the context of knowledge-based competition and in generating learning, such as SEBRAE Challenge. Given this, the database, obtained over the years, with the decisions of the participants was of paramount importance for pattern recognition, which adjusted the mathematical model of the game, always with the purpose to promote a best experiential learning. When transported to logical models of serious games, the perception that the photograph is better than a sculpture for an ID card is not always trivial. It is necessary to be always detached from the past, intending to improving, according to the needs of each audience and their circumstances. Finally, on this topic, it is worth mentioning the role of uncertainty in the game. It is a fact that in any business situation there is always uncertainty. The higher the risk, the greater the return, says the adage. In a business game, uncertainty should be crafted with great care. The Formula E2 works with groups up to 10 teams to facilitate the understanding of the model and the cause-effect components. The decisions of the other companies in that market are already a substantial factor of uncertainty for a team. Meanwhile, in Formula E2, the degree of uncertainty in the market, as provided for the mathematical model, is configurable and adaptable during the development of a game. TOPIC E - WORK THE LEARNING CURVE The game has 250 possible decisions, arranged in such a way as to allow, in each phase, the participant to have a better experience, and gradually assimilate the concepts while Detailed round score in the ten evaluation criteria Text with feedback of the situation of the company in all evaluation criteria Chart comparing the companies in all evaluation criteria FIGURE 8 EXAMPLE OF EVALUATION SCREEN. Page 29 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 experiencing very different games, according to the numerous possible combinations of individual decisions by the game manager. Such a manager need to have a profound knowledge of the type and the characteristics of participants to provide adequate experience to the context. The problem of having a mixed audience is resolved with a phase of level adjustment. In all competitions, the phases were present. In the first phase, the setting of the game is restricted to the basic requirement, favoring this leveling. In the later stages, there are additional tasks, with more refined settings, propelling the participant to develop himself increasingly. In this "laboratory", each time you get involved, you experience a unique situation, adding increasingly strengthening in possibilities of knowledge. TOPIC F - USER EXPERIENCE In the course of the years of development, some specific points were perceived as crucial to producing a good user experience. In general, improve the user experience can be seen as a challenge tailored to each platform. However, the following exposition presents a broad view of heuristics to a good user experience, that the game implemented over time. Whenever possible, the game shall present graphical returns of decisions, even if accompanied by their respective contents. The platform usage should be as intuitive as possible, giving to the participant more time for the central objective. Surprise elements are welcome to break the routine of decisions. It is a must to use different communication channels to achieve the correct attention of the players. The attendance to the participants, with educational character is well done when the answers to the participants are regarded as part of the educational process, assuming the role of the teacher in a test day. Is very important to present the progress clearly and objectively. The characters must not occupy a prominent place, but stay as a complementary of the company's environment; The usage of the universal knowledge valid in other educational platforms has the objective of seek to learn from the solutions of the other platforms geared toward your audience. CONCLUSION AND FUTURE WORK Formula E2 performed as a tool mature enough in their development process, optimized over the years for the large- scale dissemination of the entrepreneurial culture. The triangle "Objective-Content-Form" described in this paper provides a solid foundation for the construction of new serious games. In the process of creating serious games, it is important to know, apply and balance the elements necessary to achieve the objective. It is a process that requires continuous monitoring of the tool and the state of the art of techniques and contents. To keep the model able to improvement is imperative to maintain proper documentation of all parts of the platform. A large-scale tool requires a logical model, with possibilities of several different settings, besides the ability to provide to the participant the appropriate content for each stage of its development in the knowledge of the entrepreneurial culture. Future works could present a comparison between this tool exposed and other educational games on purpose. Another way to add value to this work would be in creating playful tools to develop knowledge management from experience described here. ACKNOWLEDGEMENTS We would like to thanks SEBRAE and Incubadora de Empresas COPPE/UFRJ that kindly provided the data from Desafio Sebrae Business Game. Expectations of students Teacher expectations Expectations common to both 1. Support experiential learning 2. Show consequences of decisions 3. Require greater involvement 4. Be a realistic exercise 5. Expose students to business competition 1. Measure comprehension, understanding 2. Add variety to the course 3. To be easy to administer (in class) 4. Allow involve more than one instructor 5. Involve a lot of work (of students) 6. To facilitate the assignment of note, the evaluation 7. Require the creation of business objectives 1. Integrating different functional areas of the company 2. Allow the application of the theory 3. Requires teamwork 4. To be a dynamic, interactive exercise 5. Be fun 6. Awakening interest and motivate students TABLE 1 EXPECTATIONS RELATED TO THE BUSINESS GAMES (ADAPTED FROM FARIA AND WELLINGTON, 2004) Page 30 - Developments in Business Simulation and Experiential Learning, Volume 44, 2017 ABT, Clarck. Serious Games. University Press of America, Lanham, MD. 1997. ALVES, Juliano B.; HAYDU, Nicholas B.; SOUZA, Rodrigo M. de. O uso de simuladores para treinamento em áreas de alta periculosidade: Case Simulador de Guindastes Petrobras. In: SBGAMES, IX, 2010, Florianópolis: Anais, 2010. v. 1, p. 161 169. BORST, Terry; IUPPA, Nick. End-to-end game development creating independent serious games and simulations from start to finish. Burlington, Ma: Focal Press, 2009. p. 381. CAIXETA, R. P. "Desafio SEBRAE: Desenvolvendo competências empreendedoras". 2008. Trabalho de Conclusão de Curso (Graduação em Administração) Centro Universitário de Patos de Minas. CASTRO, Maurício Mendes Boavista de et al. Determinantes para a formação da cultura empreendedora: a experiência do projeto Desafio Sebrae. Revista Pensamento Contemporâneo em Administração, Niterói, v. 8, n. 1, p.104-121, mar. 2014. COSTA, Robson Antonio Tavares; PASTANA, Sheila Trícia Guedes. Jogos de empresas simulados como ferramenta de formação de empreendedores. Revista de Administração Geral, Macapá, v. 1, n. 1, p.1-22, nov. 2015. FARIA, A. J., WELLINGTON, W. J. A Survey of Simulation Game Users, Former-Users, and Never-Users, Simulation & Gaming, vol. 35, n. 2, pp. 178-207. 2004. HOOVER, J. D. 1974. Experiential Learning: Conceptualization and Definition, Simulations, Games and Experiential Learning Techniques, vol. 1, Proceedings of the First National ABSEL Conference, Oklahoma, April 26 -27. KOLB, David. Experiential learning: Experience as the source of learning and development, Prentice-Hall, Englewood Cliffs, N.J. 1984. SHUTE, V.J., MASDUKI, I., DONMEZ, O. Conceptual Framework for Modeling, Assessing and Supporting Competencies within Game Environments, Tech., Inst., Cognition and Learning, Vol. 8, pp. 137–161, 2010. MOEN, Ronald. Foundation and History of the PDSA Cycle. 2009. Retrieved, June 10, 2016, from https:// www.deming.org/sites/default/files/pdf/2015/ PDSA_History_Ron_Moen.pdf. WICKERT, Maria Lúcia Scarpini. Referenciais educacionais do Sebrae. Brasília: Sebrae Nacional, 2006. 129 p. REFERENCES