TOWARDS AN OBJECTIVE EVALUATION OF TEACHER PERFORMANCE Página web: http://epaa.asu.edu/ojs/ Artigo recebido: 25/1/2018 Facebook: /EPAAA Revisões recebidas: 13/5/2018 Twitter: @epaa_aape Aceito: 3/6/2018 Special Issue EdTech and Policies of Human Formation arquivos analíticos de políticas educativas Revista acadêmica, avaliada por pares, independente, de acesso aberto, e multilíngüe Arizona State University Volume 26 Number 119 September 30, 2018 ISSN 1068-2341 Open Educational Resources’ Impact and Outcomes: The Essence of OpenKnowledge and its Social Contribution Santiago Mengual-Andrés & Andrés Payà Rico University of Valencia Spain Citação: Mengual-Andrés, S., & Payà, A. (2018). Open Educational Resources’ impact and outcomes: The essence of OpenKnowledge and its social contribution. Education Policy Analysis Archives, 26(119). http://dx.doi.org/10.14507/epaa.26.3667 This article is part of the special issue, Edtech e Políticas de Formação Humana, edited by Lílian do Valle, Daniel Mill and Aldo Victorio Filho. Open Educational Resources’ impact and outcomes: The essence of OpenKnowledge and its social contribution Abstract: This article identifies the main social contributions of Open Educational Resources (OERs), their impact and the results of several initiatives related to them. Based on a theoretical and reflexive framework, we analyze the evolution of business models around OERs, their contribution to open knowledge and the common educational policies related to this phenomenon. Finally, we point out the new horizons and models of knowledge based on OERs. Design/methodology/approach: This study utilizes a theoretical and conceptual framework based on references coming from journals with an impact factor, complemented with quotes by important authors and OER representatives. The review offers a conceptual and applied http://epaa.asu.edu/ojs/ http://dx.doi.org/10.14507/epaa.26.3667 Open Educational Resources’ impact and outcomes 2 overview regarding the business models around OERs, their contribution to open knowledge and their impact on current society. Because the study is not empirical, the authors justify their statements on existing scientific literature with high academic impact. Practical implications include knowledge about the OER phenomenon, its social contributions, and the economic models established around it. Keywords: OER; Open Knowledge; business model; pedagogy; research Impacto y resultados de Recursos Educativos Abiertos: La esencia del conocimiento abierto y su contribución social Resumen: Este artículo identifica las principales continuaciones sociales de los recursos educativos abiertos (REA), su impacto y los resultados de diversas iniciativas relacionadas con los mismos. A partir de un marco teórico y reflexivo se analiza la evolución de los modelos de negocio en torno a los REA, su contribución al conocimiento abierto y las políticas educativas más comunes relacionadas con el fenómeno. Finalmente se señalan los nuevo horizontes y modelos de conocimiento basados en los REA. Este estudio pretende ofrecer un marco teórico y conceptual basado en referencias procedentes de literatura con factor de impacto. Dicho marco es complementado con citas de autores relevantes y representantes del fenómeno REA, ofreciendo un marco conceptual integral. la presente revisión ofrece una visión conceptual genérica y aplicada de los modelos de negocio entorno a los REA, su contribución al conocimiento abierto y su impacto en la sociedad actual. El presente artículo no es empírico por lo que los autores justifican sus afirmaciones sobre la literatura científica existente de alto impacto académico. Implicaciones prácticas: conocimiento del fenómeno de los REA, de su contribución social y de los modelos de negocio establecidos a su alrededor. Palabras-clave: REA; conocimiento abierto; modelos de negocio; pedagogía; investigación Impacto e resultados de Recursos Educacionais Abertos: A essência do conhecimento aberto e sua contribuição social Resumo: Este artigo identifica as principais continuações sociais de recursos educacionais abertos (OER), seu impacto e os resultados de várias iniciativas relacionadas a eles. A partir de um referencial teórico-reflexivo, analisa-se a evolução dos modelos de negócios em torno da REA, sua contribuição para o conhecimento aberto e as políticas educacionais mais comuns relacionadas ao fenômeno. Finalmente, os novos horizontes e modelos de conhecimento baseados nos REA são apontados. Este estudo tem como objetivo oferecer um referencial teórico-conceitual baseado em referências bibliográficas com um fator de impacto. Essa estrutura é complementada por citações de autores relevantes e representantes do fenômeno REA, oferecendo uma estrutura conceitual integral. A presente revisão oferece uma visão conceitual genérica e aplicada dos modelos de negócios em torno da REA, sua contribuição para o conhecimento aberto e seu impacto na sociedade atual. O presente artigo não é empírico, então os autores justificam suas afirmações sobre a literatura científica existente com alto impacto acadêmico. Implicações práticas: conhecimento do fenômeno REA, sua contribuição social e os modelos de negócios estabelecidos em torno dele. Palavras-chave: REA; conhecimento aberto; modelos de negócios; pedagogia; investigação Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 3 Introduction Today, technology permeates almost all of the activities of the world we live in. Information and Communications Technologies (ICT) have penetrated the economic and political markets, our workplaces, the way in which we communicate and interact, and, in short, the overall functioning of our societies. The impact of ICTs has changed the speed of knowledge production and distribution, the ways in which we consume said information, and learning modalities, and without realizing it, this impact has been a highly significant disruptive element for our educational systems. The information society is sustained by the digital, with the bit being considered as the most elemental raw material in any scenario of interaction. In this scenario, it is not uncommon to envisage the synergy among what we venture to call layers of information or knowledge. A virtual geographical layer, or digital map, may be interspersed with a physical and real layer—an example of a specific location—that allows for interaction with the various elements of the physical environment, to which one or several digital layers—such as the social layer—are added, expanding the potential of the physical layer and allowing us access to more information than we could conceive of two decades ago. This example serves only to set the scene for the changes that society is experiencing and the reflection required of social agents and consultants in order to take social advantage of technological innovations. Aware of the legion of scientific literature regarding technology’s contributions to society, as well as its disadvantages, the digital divide may be the leading example but it is still worth analyzing, from an educational and social perspective, the repercussion that the Open Knowledge movement has, had, and continues to have on post-industrial societies and how Open Educational Resources (OER) have become the central element of knowledge’s contribution to society. In 2001, the Massachusetts Institute of Technology (MIT) announced in an unprecedented move that its courses would be openly accessible online. At the beginning of that decade, the number of educational institutions that decided to join that initiative expanded, giving rise to the organization of the 1st Global OER Forum in 2002. The term Open Educational Resources (OER) was formally adopted there to refer to the “provision of educational resources, enabled by information and communication technologies, for consultation, use and adaptation by a community of users for non-commercial purposes” (UNESCO, 2002, p. 24). They were established as minor (modular) components that should be understood as small units of learning. However, in light of the lack of a specific consensus regarding the definition of the term OER (Rodriguez & Cueva, 2010), we can define them using three fundamental elements: a) resource, generally digital; b) educational or aimed at learning by the acquisition of knowledge and/or competencies; c) open, frame of reference that allows the object to be consulted, modified and distributed freely and under any circumstance. To educate is to share, to open. We speak of OER, of open content, but perhaps we should start speaking of Open Learning, a much broader concept, and one that resonates with the methodology of the digital world. OERs base their potential on a simple but powerful idea: knowledge and learning resources are offered freely so that anyone may use them. In short, they are open sources of assets (Tuomi, 2005). Although OERs are only one essential element, their goal is not to make large blocks of organized knowledge per se. The empirical research has determined that the typical uses of OERs are much less structured in general (Masterman & Wild, 2011) and may be used to cover specific learning needs, thus offering opportunities for supplementary learning and developing skills in a different way than in traditional teaching. In this sense, and although the OpenCourseWare (OCW) initiatives worldwide have created blocks of formal learning based on OER, a recent study made Open Educational Resources’ impact and outcomes 4 available to MIT’s OCW made clear that, in the majority, access to information is the general purpose of OERs. It is more focused on informal learning, and therefore its educational intention is more about improving knowledge, offering a supplement, and exploring interests outside of the formal educational environment (MIT OCW, 2011). The beginnings of OERs based on free access to knowledge and the acquisition of knowledge through informal education have created a much larger movement known as Open Knowledge. OERs represent a milestone in this movement and have encouraged discussion and reflection on their use and adoption in different dimensions (pedagogical, political, institutional, etc.), dimensions that involve a reading and reflection that we will attempt to address throughout these pages. From Economic Model to Knowledge Model The goal of OERs is to be an element of balance in the access to knowledge and education worldwide, thus fostering a more egalitarian and real access in all regions of the planet. The expanded models of free access to knowledge are largely in agreement with the idea of a social, egalitarian, inclusive and equitable education. Numerous studies have made clear the potentialities of OERs in developing countries, both at the governmental and scholastic levels. Open Knowledge— in general—favors the expansion of economies and scientific productivity (Duart & Mengual, 2014) making access to knowledge and education a less unbalanced reality among the regions of the world in which we live. However, OERs have not been immune to the tendencies of the so-called neo-liberal educational models. Or, in short, to the trends in a world mediated by globalization, or universalization, depending on the discussion. In that sense, it is normal to find said tendencies in all emerging models derived from Open Knowledge, such as in the case of the MOOC phenomenon, and the growing business model associated with knowledge that has been developing over the past year (Mengual, 2013). Analyzing the fact of the business models around OERs is not easy and is not free from interpretations. Understanding education from the perspective of individual freedoms or social guarantee is a personal exercise that respects individual rights and freedoms, while also being valid within what the supranational organizations express regarding freedom in education. However, and although it may seem dichotomous, OERs may be combined with business models regarding their use that do not create dissonance with the philosophy of the movement. In that sense, there are many authors that justify the use of OER from an economic point of view; for example, the fact that they tend to reduce the cost of developing teaching programs (Conrad, Mackintosh, McGreal, Murphy & Witthaus, 2013; Downes, 2007), or the decrease in studying costs for students (Conole, 2013; Wiley & Gurrell, 2009). These arguments from the educational business model perspective continue to be valid, but, at the same time, understood from the perspective of a model aimed at accountability and not at guaranteeing educational quality. In that sense, it must be understood that education is not, nor should be understood as, a business, and therefore the application of business models to OERs is more in line with the policies of accountability, profit, and management than with those of equitable education, learning and diffusion of knowledge. In this sense, if, one hundred and fifty years ago, we had applied the different business models that we currently apply to OERs, there would very possibly be no public university left. Although we can find numerous texts in the scientific literature on OER business models and case analyses (e.g. McGill et al., 2008; Tuomi, 2013), in the framework of Open Educational Resources: impact and outcomes organized by the FHSM in Paris (FHSM, December 2014), the business Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 5 models around OERs were analyzed and discussed in depth. Stéphan Vincent-Lancrin, representative of the OECD, presented three basic models: a) Community-based model; b) Philanthropy-based model ; c) Revenue-based model. The Community-based model bases the development and financing of OERs on a practical community, with the goal of offering experience to the members of the community. In said model, its sustainability will depend on the dynamism of the community, its expansion, and content updating, as well as on secondary income sources such as Crowdfunding campaigns. There are several examples of this model in the French context (e.x. Sesamaths, Livrescolaire), but most likely one of its greatest challenges is to keep the community active and increase the number of members in order for the model to be sustainable. The philanthropic model is preferably to be adopted by foundations, governments and companies. Its financing is limited to donations and grants, with its sustainability linked to the financing received or the strategies anticipated by the possible donors. However, said model aims to highlight the social value of the project and repository. This model faces the challenge of not influencing the content of OERs according to the interests of the financing source, as well as spreading the use of the objects created. Some examples can be found in the Khan Academy or OpenLearn platforms, or in the Hewlett Foundation’s initiatives. Finally, the revenue model is usually developed by an organization that bases its revenue model on third-party services that it offers to end-users. In short, its success depends on its ability to attract users and its information sales models. It is common to find freemium strategies or marketing based on the sale of additional services, and not charging for the learning objects, which are free. That is, a model of open educational resources but not-so-open educational services. In practice, it is common to find hybrid or mixed models, as in the case of MIT Open CourseWare (community + philanthropy) or generally philanthropic models with revenue. In short, the dynamics of OER-based business models do not establish a charge for the contents, which are generally free (completely or partially), and focus on the revenue from supplementary material, updates of said material, added-value services, etc. One of the companies that best represents this last model in the French and European contexts is OpenClassroom, based on the Freemium model since 2013 (Dubuc, 2014). So all signs indicate a movement of OERs towards a model of open contents and closed, high-profit services, which in a certain way perverts the initial philosophy of the initiative. OER business models may be applied to numerous resources (textbooks, software, digital learning objects, etc.) and are important to decision-making in terms of educational policies, business learning strategies and end users. In this way, they may help reduce the costs of educational management (without undercutting the quality), foster the creation of learning communities, expand access to quality educational resources, encourage change in learning roles (the content curating process as a disruptive element of traditional teaching), and, in short, strengthen the phenomenon of Open Knowledge. However, the business models must make us reflect on the profit-making aims in the OER phenomenon and the business opportunities that numerous organizations are seeing under the umbrella of OpenKnowledge. There are also other uncertainties underlying the business models, such as the imposition of a given culture in disadvantaged regions that use OERs as a means of access to quality information that, on numerous occasions, are not free of ideological or political connotations, or the impartiality that a good OER must preserve. In the same way, the curricular and cultural adaptation of OERs continue to be problems that both organizations and governments must face in terms of making responsible use of the learning objects. Faced with this scenario, we can speak of a necessary transition from the OER business models to knowledge models. The transition of this model promotes open learning, which creates greater benefit (in social terms) than the traditional business models. In this way, in agreement with Open Educational Resources’ impact and outcomes 6 Bacsich (2014), Open Access—among its many contributions—opens knowledge to society, fosters learning, helps reduce the costs of access to information and research, increases the scope of the diffusion of knowledge and/or research results, promotes a better-informed citizenry and improves democratic decision making. This type of model for use of OERs has an intangible value for organizations, institutions and governments that support non-profit use of OERs in their policies. In short, they help build a more informed, better-trained, more productive and democratic society. One example is the OER collection of The Metropolitan Museum of Art, which is not only a contribution to the scientific and social community, but also a free-access educational and learning supplement that fosters didactic itineraries and the expansion of the boundaries of the physical landscape (Thillay, 2014). Another example is the knowledge contribution model offered by iTunesU. Through iTunesU, Apple not only makes clear its commitment to OpenKnowledge, but also strengthens its image as a company and gives added value to the users of its products, without a business model focused on OERs. In this case, OERS serve as a supplement that can be consumed with or without its products, and that benefits the global society in the end. Role of OERs in Educational Policies and Formal, Non-formal and Informal Education Numerous supranational organizations have taken a stance on the role of OERs, Open Knowledge, Open Access and Open Science in the current global context. The 2012 Paris OER Declaration, approved at the World OER Congress held at UNESCO in 2012, expresses the benefits of OERs worldwide. This is justified not only for its contribution to the right to education established in the Universal Declaration of Human Rights, but also for its great contribution to various Declarations (such as the Millennium Declaration, Dakar, 2000) or the Declaration of Principles of the 2003 World Summit on the Information Society. Said declaration established that OERs contribute to the consolidation of the objectives of numerous international declarations, and that the different underwriting states must therefore, within their capacities and authority, foster the use of OERs by means of ICT environments that promote the use of open licensing; create quality materials and foster strategic alliances for OERs and encourage research on this trend. In the same way, the 2012 UNESCO declaration specifies the necessary reformulation of strategies and policies for the use of OER, promoting the development of specific policies for the production and use of these resources in order to advance education. Numerous investigations and reports have exemplified the commitment of numerous institutions and states to OERs and Open Knowledge (e.g. Atkins, Brown, & Hammond, 2007; Carson, 2009; Han & Liu, 2010; Hood & Littlejohn, 2017), the impact of OERs on formal education systems (e.g. Masterman & Wild, 2011), and even specific state programs such as the China Quality OCW program. There are numerous OER initiatives all over the world, and politicians are actively developing policies regarding this issue (Hylén et al., 2012). It is therefore not surprising to find initiatives in numerous regions of the planet: Brazil, China, Korea, Slovenia, South Africa, Turkey and Vietnam (Butcher & Hoosen, 2014; Dhanarajan & Porter, 2013; Jacobi, Jelgerhuis & van der Woert, 2013). The worldwide education agenda emphasizes the necessary educational demand that continues to grow across the world. However, the worldwide financial crisis that the majority of states are undergoing, and the increasingly restrictive economic policies have led states to see OERs as a potentially appealing solution for their Governments, while simultaneously being considered Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 7 important for democratizing access to education and promoting widespread access to permanent learning (Life Long Learning; James & Bossu, 2014). In that sense, OERs are becoming a viable alternative for developing countries where traditional models cannot meet the needs of the students, pupils and institutions (Butcher & Hoosen, 2014). There have been numerous debates and recent political initiatives regarding OERs that have focused on free access to learning materials and the economic viability of OERs (de Langen, 2011). In the EU, for example, the POERUP (Policies for OER Uptake) project, financed by the EU Lifelong Learning Program 2011-2014, has carried out numerous case studies with the aim of advising governments on how to stimulate the use of OER (Bacsich, 2014). OERs and new technologies are, without a doubt, transforming education (Tuomi, 2013), and educational institutions are thereby obliged to reconsider their priorities and functions (Butcher, 2011; McAndrew, 2010; Schaffert, 2010). In this scenario, governments, by means of their educational policies, must be the guarantors of promoting access to an equitable and quality education, fostering access to the resources deemed necessary in order to guarantee the comprehensive learning of the individual, and thereby the availability of all the resources that the state can guarantee. These (centralized) policies must not be the only ones to set the pace of the inclusion and use of OER in formal education (e.g. Danon, 2014). Decentralized policies also contribute to establishing the directives that allow those organizations (regions, educational institutions, etc.) to implement the use and creation of OERs into their educational management systems. Both policy management models aim to improve lifelong learning, promote equal opportunities, educational flexibility and diversification, the increase of support resources, etc., as well as to establish the frameworks, guidelines, regulations, and elements of quality that serve to execute an open education based on OER. Above all, it is clear that states and their governments, as guarantors of access to the right to education, must promote the creation of public policies (centralized or decentralized) that take advantage of both the social and economic benefits of Open Knowledge; an open society becomes a more democratic, equitable, informed and productive society. Understanding that access to knowledge is not a privilege, but rather a right, means that governments must support open-access policies in harmony with the information society and the four social functions of education that are far-removed from the traditional model inherited from the industrial society (Tuomi, 2013). Supporting the socio-educational implementation of OERs means understanding the educational model from a paradigm of continuous learning, online, where the productive component is based on continuous learning, information processing, and the creation of knowledge. The use of OERs educational models involves changes in educational enculturation, safeguarding cultural diversity and the transition of educational agents in communities and learning networks. At the same time, in terms of the socio-educational function of personal development, OERs safeguard the acquisition of individual and social skills, as well as increasing the pupil’s decision-making skills. We agree with Ferrari & Traina (2013) in thinking that accepting the use of OER in education means modifying our educational perspective towards the search for creative solutions for prescriptive methods that have essentially emanated from the publishing industry’s pedagogic models. OERs have started appearing significantly and globally in education – and especially in higher education – due to the significant reach and expansion of the learning models, the models based on knowledge and online learning. Nevertheless, this adaptation has occurred unequally in different countries and at different educational levels (McKerlich, Ives & McGreal, 2013), and in numerous situations it has occurred very slowly (Conole, 2012; Pegler, 2013). Given the movement’s beginnings, higher education is perhaps the level that has seen the most expansion in terms of adoption. In our view, the technological resources available at institutions of higher learning, together with greater levels of teaching freedom and flexibility in terms of creating programs, have Open Educational Resources’ impact and outcomes 8 been most responsible for making this adoption possible. Another aspect to consider is the great possibility of accessing teaching resources in various languages (Keller & Mossink, 2008), the possibilities of worldwide collaboration in teaching practice (D’Antoni, 2008), and the possibilities of offering pedagogy focused on the student (Bossu, Bull & Brown, 2012). However, the levels of basic and mandatory formal education have seen a smaller, more gradual adoption. Numerous investigations have linked the teaching staff’s use of technology in the classroom with the attitudes and beliefs about that technology (e.g. Almekhlafi & Almeqdadi, 2010; Li-Ping & Jill, 2009; Parker & Burnie, 2009; Ward & Parr, 2010). The stages of technological adoption proposed by Rogers (1985) indicated that it is necessary for the teaching staff to have sufficient technological competencies in order to achieve a real and effective integration of ICT into the classroom. Substituting the practical pedagogical model offered by the publishing industry is not an easy task, and it often requires the use of educational policies that motivate the teaching staff towards said change. There are numerous benefits of OER use in basic mandatory education, but there are various challenges that educational agents must face. The teaching staff’s use of OERs involves a change in the teacher’s role, and therefore a methodology change in the teaching-learning process. Using methodologies in line with the knowledge society by means of OER means that the teacher must assume the role of content curator, information manager, facilitator in the classroom, etc. However, this first initial and desirable phase must lead to a more in-depth phase, where the teacher not only selects OERs suitable for learning, but also collaborates actively in the development and distribution of OER. This final, complex phase requires digital competencies on the part of the teaching staff, which inevitably require specific training policies in the use and integration of ICT in the classroom. The business models of teaching book publishers, aware of their purchasing power loss due to the multiple school initiatives for the use of open learning materials or Open Access and collaborative schoolbooks, have outlined a mixed, appealing business model. Given the necessary time investment on the teacher’s part in selecting suitable OERs and properly integrating them into a program, or the creation of specific OER, the publishers propose tailor-made itineraries that combine their own materials with previously selected OER. However, we must reflect on whether the education of the information and knowledge society should be at the mercy of the contents previously selected by a publishing house, or, on the contrary, whether it is the educational agents’ function to program the didactic itineraries with entirely open resources. In our view, both the economic and social benefits are clear. The materials must respond to the needs of the classroom, and the classroom should not have to adapt to the specifications of the textbook. Online learning allows for an individualized curricular adaptation that no textbook or pre-selected OER can offer. Therefore, the teacher’s role and function as mediator is the key to success in the educational integration of OERs. From a different perspective, far from formal education, OERs have amounted to a revolution in terms of the possibility of continuous learning for society. Today, it is possible for any citizen to learn with the same materials as a student at Stanford or Harvard University and, at the same time, to see the master lectures (either live or pre-recorded) stored openly online. The goal of using OERs in said educational modalities (non-formal and informal) is not academic recognition or accrediting the acquisition of competencies, but rather the satisfaction of a basic human need: learning. In this way, a student that is dissatisfied with the lectures he/she receives at his/her institution may freely learn the same material with OERs used at other institutions, and thus supplement his/her learning. From the informal learning perspective, OERs allow for the supplementary acquisition of knowledge and competencies that enrich a citizen’s learning. Whether out of personal interest, motivation, or need, OERs and Open Knowledge broaden the professional horizons of societies’ Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 9 human capital, creating better-prepared citizens and offering educational possibilities that a state sometimes does not address in its formal education, and that, in addition, due to socioeconomic barriers, a citizen cannot afford to attend in person outside of his/her country. It is not surprising that this scenario and the use of OERs and the possibilities of ICT have developed initiatives such as the University of the People, a free online higher education institution that broadens the limits of knowledge that the state guarantees to a citizen in its territory. In the same way, OERs have given rise to new modalities of online education, as in the case of MOOC (Massive Open Online Courses) and their different variations (xMOOC, LMOOC, etc.); emerging educational modalities that respond to a social need, to learn more, better, in collaboration, openly, and with quality materials. In any case, while access to knowledge is of the utmost importance, we must not forget how the receivers assimilate this knowledge. New Horizons OERs have not only been an element of educational innovation (e.g. Thiriet, 2014) at all education levels, but they have also been able to respond to numerous problems originating from the school context. Agreeing with Majada (2014), OER initiatives help combat early academic failure, allowing for the early detection of the possible special educational needs of the students. In the same way, they allow us to pay attention to the diversity of the student, offering personalized learning itineraries and continuous learning by means of distance learning modalities. In Majada’s own words, OERs simplify educational access to the most vulnerable groups, promoting better access to persons with disability. In the same way, it fosters the international mobility of the students, contributing to overcoming language barriers and encouraging interculturalism. In addition, OERs contribute to the preservation of cultural heritage and the development and promotion of lesser-known disciplines. As we mentioned previously, to speak of OER is, undoubtedly, to recognize the social need to learn through non-formal education contexts. OERs are not only contributing to the pedagogical development of educational systems, but also to their management style and educational policies; open educational resources offer numerous new horizons. In particular, new research and project financing trends are being framed around Open Knowledge and Open Education. Some examples of projects financed by the European Union and linked to OERs include Scale CCR, OEREU, OpenEdu and COM4EDU (dos Santos, 2014). Said projects are framed not only within European Policies on Open Education, but also around a social, economic, and political context at the supranational level; recent studies have also made clear that Open Education acts as a bridge for formal education and fosters a more open mentality towards sharing, remixing and redistributing (Weller, 2014). At the same time, the phenomenon of Massive Open Online Courses (MOOC), essentially a compendium of OERs focused on Higher Education, has brought about a pedagogical and research revolution in the past three years. Both universities and higher education agents have seen in MOOCs a new tool for the democratization of learning, for displaying their methodologies, teaching staff, courses, etc., as well as for demonstrating their socio-educational commitment to Open Knowledge. MOOCs have not only been a pedagogical and educational revolution; they have been a model for the transformation of higher education and of the way in which university knowledge is consumed. They have not been a re-invention of OERs, but rather a technological and pedagogical layer linked to OERs, offering added-value tools for learning – such as the connectivist MOOCs, or xMOOC – and allowing for new policies and ideas regarding the modalities for the accreditation of competencies (Mengual, 2013). MOOCs, intimately linked to the Open Education Movement, could be a turning point in higher education modalities. In that sense, the massive online courses may Open Educational Resources’ impact and outcomes 10 increase the number of university graduates both in developing and developed countries, by means of much more affordable accreditation systems than on-site teaching modalities. Thus, universities are seeing the potentiality of these new modalities, which undoubtedly benefit all educational players. In short, MOOCs create opportunities for innovation in teaching and learning; they transform learning, not only in formal education, but also in the non-formal contexts derived from lifelong learning, the development of professional competencies in employed and unemployed social capital, as well as the fostering of self-learning (Lesko, 2014). It is clear that Open Education has become part of the political-educational agenda of numerous International organizations and Governments. It is estimated that in 2025 the demand for higher education will increase by 98 million new students (OECD, 2009), leading us to question the ability of governments to face and finance the costs of public education. In this scenario, ICT, the OER Movement, the OA Movement, the Open Education Movement and others are proposed as tools for transformation that may help in the educational challenges to come (Lesko, 2014). This scenario makes clear the importance of the MOOC phenomenon to science. In that sense, 2014 has been a flagship year for research regarding the movement. The majority of specialized scientific journals have focused their interest on studies on the trend from different perspectives. The more than 400 articles collected between 2013-2015 in journals from the WOS and SCOPUS selective database make this fact clear, and they indicate the relevance within national and international contexts. However, according to some authors, although the phenomenon is important, the number of high-impact publications continues to be very low compared to other emerging fields; having said that, the universities and countries that indicate the greatest scientific repercussion in this regard are the USA, Australia, Canada, the United Kingdom, and Spain (Lopez- Meneses, Vazquez-Cano & Román, 2015). Conclusion The expansion and growth of OER initiatives, as well as the amount of students they attract, together with their social benefit, may be an indicator of a coming revolution in education and learning (Tuomi, 2013). One of the fundamental roles of OERs is to bring education to the highest number of people possible, but also, and not less importantly, to transform education and educational systems. OERs require open educational resources, but also open technology and open practices, and especially new learning methodologies and a significant change in the role of the teacher. And when we speak of a new role, we should not forget that we are speaking of a feeling of loss of power, and this is always difficult to accept, although in reality, the power of the teacher in the classroom is not lost, it is transformed. Educational technology and the Open Knowledge movement have promoted the adoption of new teaching roles from a variety of perspectives (León-Urritia,. Cobos, & Dickens, 2018). Nonetheless, this way of learning without the mediation of the teacher can promote the non-formal learning what implies a necessary education revision. The teaching roles do not disappear but are transformed beyond self-learning, for example, as a guide (Torres, Santa & Lorenzo, 2018). Debate on OERs involves much more extensive reading regarding the broad strokes outlined in this paper. Studying the potential revolution of OERs and OpenKnowledge involves carrying out an analysis of the social, economic, and historical contexts into which OERs are attempting to integrate. But as Hal Plotkin stated, it is extremely difficult to think of policies and strategies and act in the middle of a revolution. In any case, more in-depth analysis of OERs is necessary. The lack of theory on open contents and their consequences causes confusion, disagreements and bad practices. OERs are contents. In no case are they an educational model or practice per se. In any case, OERs are a lever of change for models and practices. They are criticized Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 11 for their lack of pedagogy and methodology; perhaps now is the time to associate them with an open learning that includes, by necessity, these elements. In its document entitled, “Giving Knowledge For Free?” (2007), the OECD already considered this issue, and we have made very little progress. Almost a decade ago, that report was already speaking of reconfiguring the educational systems and economic models of higher public education, and it posed questions that still have not been answered, not because they are difficult to respond to, but rather because they represent facing said changes, and it is more convenient to avoid them and look the other way: How open are European universities? How open are European Open Universities? Is OER Open enough? Is OER targeted at me? Is OER in my language? Is free “cheap enough”? If we had attempted to find an answer in recent years, everything today would most likely be clearer and we would have progressed. The problem is that when an innovation emerges, traditional institutions and public policies do not pick them up until 10 or 15 years later. Finally, the ways in which OERs are used will mediate the abilities to make effective quality learning that responds to the social and emerging needs of a country’s education. References Almekhlafi, A. G., & Almeqdadi, F. A. (2010). Teachers' perceptions of technology integration in the United Arab Emirates School Classrooms. Educational Technology & Society, 13(1), 165- 175. Bacsich, P. (2014). Purity or profit: Flexing the OERbusiness model to achieve sustainability of open education. Presented at Open Educational Resources: Impact and Outcomes, Paris, France. Bossu, C., Bull, D., & Brown, M. (2012). Opening up Down Under: The role of open educational resources in promoting social inclusion in Australia. Distance Education, 33(2), 151-164. https://doi.org/10.1080/01587919.2012.692050 Butcher, N. (2011). A Basic Guide to Open Educational Resources (OER).Vancouver: Commonwealth of Learning. Butcher, N., & Hoosen, S. (2014). A Guide to Quality in Post-Traditional Online Higher Education. Dallas: Academic Partnerships. Castells, M., Tubella, I., Sancho, T. & Roca, M. (2007). La transición a la sociedad red. Barcelona: Ariel. Conole, G. (2012) Fostering social inclusion through open educational resources (OER). Distance Education, 33(2), 131-134. https://doi.org/10.1080/01587919.2012.700563 Conole, G. (2013). Designing for learning in an open world (vol. 4). Nueva York, NY: Springer. https://doi.org/10.1007/978-1-4419-8517-0 Conrad, D., Mackintosh, W., McGreal, R., Murphy, A., & Witthaus, G. (2013). Report on the assessment and accreditation of learners using OER [Research Report]. Vancouver: Commonwealth of Learning. D’Antoni, S. (2008). Open Educational Resources: the way forward, deliberations of an international community of interest [research report]. Vancouver: Commonwealth of Learning. Danon, C. (2014). Ressources éducatives libres Enjeux d’une politique nationale. Presented at Open Educational Resources: Impact and Outcomes, Paris, France. de Langen, F. (2011) There is no business model for open educational resources: A business model approach. Open Learning: The Journal of Open and Distance Learning, 26, 209-222. https://doi.org/10.1080/02680513.2011.611683 Downes, S. (2007). Models for sustainable Open Educational Resources. Interdisciplinary Journal of Knowledge and Learning Objects, 3, 29-44. https://doi.org/10.28945/384 Open Educational Resources’ impact and outcomes 12 Duart, J. M., & Mengual, S. (2014). Impact of the knowledge society in the university and in scientific communication. RELIEVE, 20(2), art. M4. https://doi.org/10.7203/relieve.20.2.4343 Dubuc, P. (2014). Rel : Le Point De Vue D’un Éditeur The Oer Business Model. Presented at Open Educational Resources: Impact and Outcomes, Paris, France. Ferrari, L., & Traina, I. (2013). The OERTEST Project: Creating Political Conditions for Effective Exchange of OER in Higher Education. Journal of e-Learning and Knowledge Society, 9(1), 23-35. FMSH. (2014). Open Educational Resources: Impact and Outcomes. Retrieved October 23, 2017, from: http://www.fmsh.fr/en/node/26958 Gajaraj Dhanarajan, G., & Porter, D. (Eds.) (2013). Perspectives on Open and Distance Learning: Open Educational Resources: An Asian Perspective. Vancouver: Commonwealth of Learning and OER Asia. Hood, N., & Littlejohn, A. (2017). Knowledge typologies for professional learning: Educators’ (re)generation of knowledge when learning open educational practice. Educational Technology Research and Development, 65(6), 1583-1604. https://doi.org/10.1007/s11423-017-9536-z Hylén, J., Damme, D., Mulder, F. & D’Antoni, S. (2012) Open Educational Resources: Analysis of responses to the OECD Country Questionnaire (No. 76). Paris: OECD. Jacobi, R., Jelgerhuis, H., & van der Woert, N. (Eds.) (2013). Trend Report: Open Educational Resources 2013. Nijmegen: SURF. James, R., & Bossu, C. (2014). Conversations from south of the equator: Challenges and Opportunities in OER across Broader Oceania. RUSC. Universities and Knowledge Society Journal, 11(3), 78-90. https://doi.org/10.7238/rusc.v11i3.2220 Keller, P., & Mossink, W. (2008). Reuse of material in the context of education and research. Países Bajos: Creative Commons Nederlands. León-Urritia, M., Cobos, R., & Dickens, K. (2018). MOOCs and their Influence on Higher Education Institutions: Perspectives from the Insiders. Journal of New Approaches in Educational Research, 7(1), 40-45. https://doi.org/10.7821/naer.2018.1.252 Lesko, I. (2014). Open Education Consortium – advancing openness in education worldwide. Presented at Open Educational Resources: Impact and Outcomes, Paris, France. Li-Ping. T., & Jill, M. (2009). Students’ perceptions of teaching technologies, application of technologies, and academic performance. Computers & Education, 53, 1241- 1255. https://doi.org/10.1016/j.compedu.2009.06.007 López, E., Vázquez, E. & Román, P. (2015). Analysis and Implications of the Impact of MOOC Movement in the Scientific Community: JCR and Scopus. Comunicar, 44, 73-80. https://doi.org/10.3916/C44-2015-08 Masterman, L. & Wild, J. (2011) JISC Open Educational Resources Programme: Phase 2. (OER Impact Study: Research Report). Retrieved from http://www.jisc.ac.uk/media/documents/programmes/elearning/oer/JISCOERImpactStu dyResearchReportv1-0.pdf McAndrew, P. (2010) Defining openness: Updating the concept of ‘open’ for a connected world. Journal of Interactive Media in Education, 1-13. McGill, L., Currier, S., Duncan, C., & Douglas, P. (2008). Good intentions: improving the evidence base in support of sharing learning materials. (Research Report). Retrieved from http://ierepository.jisc.ac.uk/265/1/goodintentionspublic.pdf. McKerlich, R., Ives, C., & McGreal, R. (2013). Measuring use and creation of open educational resources in higher education. The International Review of Research in Open and Distance Learning, 14(4). https://doi.org/10.19173/irrodl.v14i4.1573 http://www.jisc.ac.uk/media/documents/programmes/elearning/oer/JISCOERImpactStudyResearchReportv1-0.pdf http://www.jisc.ac.uk/media/documents/programmes/elearning/oer/JISCOERImpactStudyResearchReportv1-0.pdf http://ierepository.jisc.ac.uk/265/1/goodintentionspublic.pdf Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 13 Mengual, S. (2013). Rethinking the role of Higher Education. Journal of New Approaches in Educational Research, 2(1). https://doi.org/10.7821/naer.2.1.1-2 OECD (2007). Giving Knowledge for free? The emergence of Open Educational Resources. Paris: OECD. Parker, B., & Burnie, D. (2009). Classroom Technology in Business Schools. AACE Journal, 17(1), 45-60. Pegler, C. (2013). The Influence of Open Resources on Design Practice. In H. Beetham & R. Sharpe (Eds.), Rethinking Pedagogy for a Digital Age: Designing for 21st Century Learning (pp. 306-333). New York: Taylor & Francis. Rodriguez, G., & Cueba P. (2010). OER, estándares y tendencias. RUSC. Universities and Knowledge Society Journal, 7(1). https://doi.org/10.7238/rusc.v7i1.361 Rogers, E. M. (1985). Diffusion of innovations (4ª ed.). Nueva York: The Free Press. Schaffert, S. (2010) Strategic integration of open educational resources in higher education. In U.-D. Ehlers & D. Schneckenberg (Eds.), Changing Cultures in Higher Education (pp. 119 -131). Berlin: Springer. https://doi.org/10.1007/978-3-642-03582-1_11 Thillay, A. (2014). Quels modèles économiques pour permettre la production de ressources éducatives libres par les enseignants du secondaire? Presented at Open Educational Resources: Impact and Outcomes, Paris, France. Thiriet, P. (2014). Technologie 3D, ressources libres et réseaux sociaux pour promouvoir, apprendre et enseigner une science. Presented at Open Educational Resources: Impact And Outcomes, Paris, France. Torres, R., Santa, A., & Lorenzo, C. (2018). Resignification of Educational E-innovation to Enhance Opportunities for Graduate Employability in the Context of New University Degrees. Journal of New Approaches in Educational Research, 7(1), 70-78. https://doi.org/10.7821/naer.2018.1.263 Tuomi, I. (2005). The future of open source. In M. Wynants & J. Cornelis (Eds.), How Open Is the Future? (pp. 429 – 459). Brussels, VUB Brussels University Press. https://doi.org/10.1111/ejed.12019 Tuomi, I. (2013). Open Educational Resources and the Transformation of Education. European Journal of Education, 48(1), 58-78. Ward, L., & Parr, J. (2010). Revisiting and reframing use: Implications for the integration of ICT. Computers & Education, 54 (1), 113-122. https://doi.org/10.1016/j.compedu.2009.07.011 Weller, M. (2014). Institute of Educational Technology OER Research Hub team. Presented at Open Educational Resources: Impact and Outcomes, Paris, France. Open Educational Resources’ impact and outcomes 14 About the Authors Santiago Mengual-Andrés University of Valencia santiago.mengual@uv.es https://orcid.org/0000-0002-1588-9741 Senior Lecturer at the Comparative Education and History of Education Department. Faculty of Philosophy and Educational Sciences, University of Valencia. Andrés Payà Rico University of Valencia andres.paya@uv.es Senior Lecturer at the Comparative Education and History of Education Department. Faculty of Philosophy and Educational Sciences, University of Valencia. About the Guest Editors Lílian do Valle Universidade do Estado do Rio de Janeiro (UERJ) lilidovalle@gmail.com https://orcid.org/0000-0002-8694-9297 Professora titular de Filosofia da Educação da Universidade do Estado do Rio de Janeiro, doutora em Educação pela Universidade de Paris V – René Descartes. Daniel Mill Universidade Federal de São Carlos (UFSCar) mill.ufscar@gmail.com https://orcid.org/0000-0002-8336-3645 Professor do Programa de Pós-Graduação em Educação da Universidade Federal de São Carlos (UFSCar). Doutor em Educação (UFMG). Gestor em Educação a Distância. Aldo Victorio Filho Universidade do Estado do Rio de Janeiro (UERJ) avictorio@gmail.com https://orcid.org/0000-0002-7132-8615 Professor do Programa de Pós-Graduação em Artes da Universidade do Estado (PPGArtes). Doutor em Educação (UERJ). Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 15 Special Issue EdTech and Policies of Human Formation education policy analysis archives Volume 26 Number 119 September 17, 2018 ISSN 1068-2341 Readers are free to copy, display, and distribute this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, it is distributed for non- commercial purposes only, and no alteration or transformation is made in the work. More details of this Creative Commons license are available at http://creativecommons.org/licenses/by-nc-sa/3.0/. All other uses must be approved by the author(s) or EPAA. EPAA is published by the Mary Lou Fulton Institute and Graduate School of Education at Arizona State University Articles are indexed in CIRC (Clasificación Integrada de Revistas Científicas, Spain), DIALNET (Spain), Directory of Open Access Journals, EBSCO Education Research Complete, ERIC, Education Full Text (H.W. Wilson), QUALIS A1 (Brazil), SCImago Journal Rank, SCOPUS, SOCOLAR (China). Please send errata notes to Audrey Amrein-Beardsley at audrey.beardsley@asu.edu Join EPAA’s Facebook community at https://www.facebook.com/EPAAAAPE and Twitter feed @epaa_aape. http://www.doaj.org/ mailto:audrey.beardsley@asu.edu https://www.facebook.com/EPAAAAPE Open Educational Resources’ impact and outcomes 16 education policy analysis archives editorial board Lead Editor: Audrey Amrein-Beardsley (Arizona State University) Editor Consultor: Gustavo E. Fischman (Arizona State University) Associate Editors: David Carlson, Lauren Harris, Eugene Judson, Mirka Koro-Ljungberg, Scott Marley, Molly Ott, Iveta Silova (Arizona State University) Cristina Alfaro San Diego State University Gene V Glass Arizona State University Susan L. Robertson Bristol University, UK Gary Anderson New York University Ronald Glass University of California, Santa Cruz Gloria M. Rodriguez University of California, Davis Michael W. Apple University of Wisconsin, Madison Jacob P. K. Gross University of Louisville R. Anthony Rolle University of Houston Jeff Bale OISE, University of Toronto, Canada Eric M. Haas WestEd A. G. Rud Washington State University Aaron Bevanot SUNY Albany Julian Vasquez Heilig California State University, Sacramento Patricia Sánchez University of University of Texas, San Antonio David C. Berliner Arizona State University Kimberly Kappler Hewitt University of North Carolina Greensboro Janelle Scott University of California, Berkeley Henry Braun Boston College Aimee Howley Ohio University Jack Schneider University of Massachusetts Lowell Casey Cobb University of Connecticut Steve Klees University of Maryland Noah Sobe Loyola University Arnold Danzig San Jose State University Jaekyung Lee SUNY Buffalo Nelly P. Stromquist University of Maryland Linda Darling-Hammond Stanford University Jessica Nina Lester Indiana University Benjamin Superfine University of Illinois, Chicago Elizabeth H. DeBray University of Georgia Amanda E. Lewis University of Illinois, Chicago Adai Tefera Virginia Commonwealth University Chad d'Entremont Rennie Center for Education Research & Policy Chad R. Lochmiller Indiana University Tina Trujillo University of California, Berkeley John Diamond University of Wisconsin, Madison Christopher Lubienski Indiana University Federico R. Waitoller University of Illinois, Chicago Matthew Di Carlo Albert Shanker Institute Sarah Lubienski Indiana University Larisa Warhol University of Connecticut Sherman Dorn Arizona State University William J. Mathis University of Colorado, Boulder John Weathers University of Colorado, Colorado Springs Michael J. Dumas University of California, Berkeley Michele S. Moses University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Kathy Escamilla University of Colorado, Boulder Julianne Moss Deakin University, Australia Terrence G. Wiley Center for Applied Linguistics Melissa Lynn Freeman Adams State College Sharon Nichols University of Texas, San Antonio John Willinsky Stanford University Rachael Gabriel University of Connecticut Eric Parsons University of Missouri-Columbia Jennifer R. Wolgemuth University of South Florida Amy Garrett Dikkers University of North Carolina, Wilmington Amanda U. Potterton University of Kentucky Kyo Yamashiro Claremont Graduate University Education Policy Analysis Archives Vol. 26, No. 119 SPECIAL ISSUE 17 archivos analíticos de políticas educativas consejo editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editores Asociados: Armando Alcántara Santuario (Universidad Nacional Autónoma de México), Jason Beech, (Universidad de San Andrés), Angelica Buendia, (Metropolitan Autonomous University), Ezequiel Gomez Caride, (Pontificia Universidad Católica Argentina), Antonio Luzon, (Universidad de Granada), José Luis Ramírez, Universidad de Sonora) Claudio Almonacid Universidad Metropolitana de Ciencias de la Educación, Chile Ana María García de Fanelli Centro de Estudios de Estado y Sociedad (CEDES) CONICET, Argentina Miriam Rodríguez Vargas Universidad Autónoma de Tamaulipas, México Miguel Ángel Arias Ortega Universidad Autónoma de la Ciudad de México Juan Carlos González Faraco Universidad de Huelva, España José Gregorio Rodríguez Universidad Nacional de Colombia, Colombia Xavier Besalú Costa Universitat de Girona, España María Clemente Linuesa Universidad de Salamanca, España Mario Rueda Beltrán Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Xavier Bonal Sarro Universidad Autónoma de Barcelona, España Jaume Martínez Bonafé Universitat de València, España José Luis San Fabián Maroto Universidad de Oviedo, España Antonio Bolívar Boitia Universidad de Granada, España Alejandro Márquez Jiménez Instituto de Investigaciones sobre la Universidad y la Educación, UNAM, México Jurjo Torres Santomé, Universidad de la Coruña, España José Joaquín Brunner Universidad Diego Portales, Chile María Guadalupe Olivier Tellez, Universidad Pedagógica Nacional, México Yengny Marisol Silva Laya Universidad Iberoamericana, México Damián Canales Sánchez Instituto Nacional para la Evaluación de la Educación, México Miguel Pereyra Universidad de Granada, España Ernesto Treviño Ronzón Universidad Veracruzana, México Gabriela de la Cruz Flores Universidad Nacional Autónoma de México Mónica Pini Universidad Nacional de San Martín, Argentina Ernesto Treviño Villarreal Universidad Diego Portales Santiago, Chile Marco Antonio Delgado Fuentes Universidad Iberoamericana, México Omar Orlando Pulido Chaves Instituto para la Investigación Educativa y el Desarrollo Pedagógico (IDEP) Antoni Verger Planells Universidad Autónoma de Barcelona, España Inés Dussel, DIE-CINVESTAV, México José Ignacio Rivas Flores Universidad de Málaga, España Catalina Wainerman Universidad de San Andrés, Argentina Pedro Flores Crespo Universidad Iberoamericana, México Juan Carlos Yáñez Velazco Universidad de Colima, México javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/819') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/820') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/4276') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/1609') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/825') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/797') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/823') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/798') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/555') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/814') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/2703') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/801') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/826') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/802') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/3264') javascript:openRTWindow('http://epaa.asu.edu/ojs/about/editorialTeamBio/804') Open Educational Resources’ impact and outcomes 18 arquivos analíticos de políticas educativas conselho editorial Editor Consultor: Gustavo E. Fischman (Arizona State University) Editoras Associadas: Geovana Mendonça Lunardi Mendes (Universidade do Estado de Santa Catarina), Marcia Pletsch, Sandra Regina Sales (Universidade Federal Rural do Rio de Janeiro) Almerindo Afonso Universidade do Minho Portugal Alexandre Fernandez Vaz Universidade Federal de Santa Catarina, Brasil José Augusto Pacheco Universidade do Minho, Portugal Rosanna Maria Barros Sá Universidade do Algarve Portugal Regina Célia Linhares Hostins Universidade do Vale do Itajaí, Brasil Jane Paiva Universidade do Estado do Rio de Janeiro, Brasil Maria Helena Bonilla Universidade Federal da Bahia Brasil Alfredo Macedo Gomes Universidade Federal de Pernambuco Brasil Paulo Alberto Santos Vieira Universidade do Estado de Mato Grosso, Brasil Rosa Maria Bueno Fischer Universidade Federal do Rio Grande do Sul, Brasil Jefferson Mainardes Universidade Estadual de Ponta Grossa, Brasil Fabiany de Cássia Tavares Silva Universidade Federal do Mato Grosso do Sul, Brasil Alice Casimiro Lopes Universidade do Estado do Rio de Janeiro, Brasil Jader Janer Moreira Lopes Universidade Federal Fluminense e Universidade Federal de Juiz de Fora, Brasil António Teodoro Universidade Lusófona Portugal Suzana Feldens Schwertner Centro Universitário Univates Brasil Debora Nunes Universidade Federal do Rio Grande do Norte, Brasil Lílian do Valle Universidade do Estado do Rio de Janeiro, Brasil Flávia Miller Naethe Motta Universidade Federal Rural do Rio de Janeiro, Brasil Alda Junqueira Marin Pontifícia Universidade Católica de São Paulo, Brasil Alfredo Veiga-Neto Universidade Federal do Rio Grande do Sul, Brasil Dalila Andrade Oliveira Universidade Federal de Minas Gerais, Brasil References About the Authors