© 2003 Current Issues in Comparative Education, Teachers College, Columbia University, ALL RIGHTS RESERVED Current Issues in Comparative Education, Vol. 6(1) 17 Institutional Evaluation, Knowledge and Multiculturalism: Some ways ahead in Brazilian higher education Ana Canen Faculty of Education, Universidade Federal do Rio de Janeiro Introduction In the contemporary world, the pivotal role of technology for knowledge production and fostering a competitive edge among countries in a highly globalized scenario has been widely recognized. The impact of information technology on the way knowledge is conceived has also been broadly discussed. Authors such as Polsani (2002) highlight the extent to which a new "rhizomic network" approach has been replacing a hierarchical and centered approach to knowledge building as a result of information technology development. However, while the importance of technology in knowledge building has been recognized, consensus has not been reached about what knowledge really means and what kind of knowledge should be fostered for educational policies. Such debates are informed by differing underlying concepts, each with serious consequences for educational goals, practices and knowledge production itself. As argued elsewhere (Canen & Grant, 1999), tensions exist between definitions of knowledge which regard it as directly linked to market demands only versus those transformational, multicultural views which assign to knowledge categories of citizenry development, appreciation of cultural diversity, as well as critical and independent thinking. These varied definitions impinge on educational policies and practices aimed at outlining and assessing the worth and merit of knowledge building goals and strategies. Similar dichotomous tensions have also underpinned discussions about the role of technology, viewed either as another source of overall inequality and exclusion or as a means by which developing economies have a chance to overcome constraints and speed up their development process. As centers of knowledge production, higher education institutions have been at the core of this volcanic debate. This is particularly evident in Brazil, where the rise of a newly elected Labor party government whose electoral claims that ensured a major victory in the elections of 2002 were mainly built on proposals of policies (including educational ones) dedicated to social inclusion, democracy, citizenry building, valuing of local cultures, and an overt stand against marketization and neoliberalism, at least at the discourse levels. Taking into account the above mentioned scenario, this present paper aims to discuss the recent proposal for a National System of Evaluation of Higher Education in Brazil (Sistema Nacional de Avaliação do Ensino Superior- SINAES, 2003), presented by a government Committee for Evaluation of Higher Education set up by the Ministry of Education in April 2003. The document analyses and makes recommendations of criteria, theories and strategies for changing processes and politics of evaluation in Higher Education. The proposal has been submitted to the academic community for discussions and should be sent to the Brazilian National Congress before the end of 2003 for deliberations about legislation concerning institutional evaluation in Brazil. Ana Canen December 17, 2003 18 I contend that a national institutional evaluation policy proposal can give precious insights as to the thrust and tensions surrounding the concept of knowledge, the role of technology and the way government perceives universities' missions towards those. This is due to the fact that such a proposal reflects government directions and criteria for judging the merit and worth of knowledge produced by higher education institutions with consequences for their resourcing, financing, accreditation and related implications. This case study has comparative relevance, as it illustrates dilemmas inherent to educational policies in countries beset by social disparities and cultural diversity that are eager to move away from educational goals linked to market perspectives but still struggle to conceive of alternative models that ensure educational inclusion without compromising educational quality and relevant knowledge for the contemporary scenario. The article is based on what I call a critical multicultural evaluation perspective that values institutional cultural diversity but nevertheless challenges relativistic approaches to it (i.e., those evaluation perspectives that claim institutional evaluation should stop short of making evaluative judgments with reference to values alien to the goals and mission spelled out by the institution itself). I argue that institutional evaluation policies that try to strike a balance between strategies that value culturally distinct institutional approaches to knowledge and those that ensure a regulatory control for quality assurance and institutional accountability are in a better position to help institutional growth. I also suggest that the will to go beyond what is generally perceived as product- oriented approaches to knowledge should not mean the demise of the search for technological and academic excellence in higher education institutional lives. The ways by which those issues have been addressed in the discourse of the Brazilian SINAES document will be discussed, as well as possible consequences and implications of such a proposal for knowledge building in multicultural countries. Institutional Evaluation of Higher Education in Brazil: What Potentials of a Critical Multicultural Model? Institutional evaluation in higher education represents a crucial process by which judgments about the worth and merit of institutional practices and knowledge building are gauged. In fact, as already claimed by Smith (1980) at the very beginning of the 1980s, evaluation is subject to myriad value judgments of what is good or ideal. I suggest institutional evaluation has mostly been perceived through dichotomous lenses, being either viewed as an objectivist process aiming at arriving at evaluative conclusions based on objective criteria and values (Scriven, 1994), or in more "ethnographic", relativistic and constructivist views generally identified as those that stop short of proposing criteria for making value judgments and evaluative conclusions and rather focus on evaluation as resulting from negotiation and dialog with institutional actors (Penna-Firme, 1995), taking into account specific institutional cultural values and missions. These latter views are far from being circumscribed to evaluation, being rather linked to postmodern and poststructural assumptions that "there is no single 'truth' - that all truths are but partial truths" (Lincoln & Guba, 2000, p.185), which leads to the impossibility of having aprioristic and pseudo-universal criteria for judging the merit and worth of any enterprise, including the evaluative one. Institutional Evaluation, Knowledge and Multiculturalism: Some Ways Ahead in Brazilian Higher Education Current Issues in Comparative Education, Vol. 6(1) 19 From the early 1990s up to the beginning of 2003, Brazilian debates and policies concerning institutional evaluation have been mostly marked by this dichotomous approach. The phase between 1994 and 1998 saw the flourishing of the Program of Institutional Evaluation of Brazilian Universities (PAIUB). This program was conceived as a process aimed at evaluating higher education institutions according to their values and missions based mostly on internal and external evaluative assessment dynamics geared towards pinpointing strengths and areas in need of strengthening (Ristoff, 1996; Sobrinho, 1996). As explained elsewhere (Canen & Oliveira, 2000), that process received government financing and was carried out in several universities, because of the general perception that it enhanced institutional growth due to its diagnostic and formative perspective, and its basis on general principles and indicators dedicated to respecting and valuing institutions' culturally plural identities. Such a program can be said to demonstrate the thrust towards more ethnographic, constructive and relativistic approaches to institutional evaluation in that it stopped short of making evaluative conclusions that could overstep the institution's own mission and values. As such, PAIUB was both lauded and criticized because it produced long reports that were perceived as useful internally, but that provided little clarity externally about the kind of knowledge universities produced and the merit and worth of the institution for society at large. A slow dismantling of the PAIUB for lack of continued governmental financial support and a growing thrust towards classificatory, product-oriented institutional evaluation have marked the second phase of institutional evaluation spanning 1998 and 2002 in Brazil. This type of evaluation is mainly translated into institutional ranking based on students' performance on National Course Exams--the so-called Provões (Brazilian nickname for "big exam")--which all university students must take at the close of their courses. While such classificatory policies have been lauded by a few who extol its efficacy as a means to provide accountability for society about knowledge produced in the universities, they have been seriously attacked by most of the academic community on the grounds that: they reduced institutional evaluation to students' evaluation; they did not take into account cultural diversity and plural institutional identities; and they claimed objectivity but were perceived as seriously biased towards measuring market demands and competencies deemed necessary in a globalized society. That type of institutional evaluation policy can be said to have silenced multiculturally-oriented perspectives in favor of homogenized, centralized and product-oriented evaluation processes under the guise of objectivity and accountability. At this point, it is important to note that academics have generally pointed to the influence of international agencies such as the World Bank in the shaping of educational policies, and indeed in the concept of knowledge that underpins those policies in countries like Brazil, which are heavily financially indebted to more developed countries. In fact, as argued elsewhere (Canen & Grant, 1999), the privileging of the 'knowledge for productivity' ideology has been to the detriment of more multiculturally oriented perspectives, and has had consequences on institutional evaluation, particularly from 1995 to the end of 2002. Since the inauguration of a new labor-oriented government whose ideological premises challenge market educational perspectives and whose rhetoric conveys defiance towards the influence of international agencies, it is Ana Canen December 17, 2003 20 relevant to evaluate the present thrust of institutional evaluation and the role of knowledge and technology underpinning it. I suggest that the will to go beyond a product-oriented approach to knowledge should not mean the perpetuation of a dichotomous approach that puts knowledge for democracy and knowledge for technological and academic excellence in opposition. In fact, I argue that a multicultural critical approach to institutional evaluation (Canen, 2001a) and to knowledge building should go beyond such dichotomies to try and work out hybridizations and syntheses that derive from the dialog between culturally diverse institutional identities and missions and those espoused by democratic societies committed to equity and excellence in knowledge building. At this point, it is important to note that a critical multicultural approach is that which not only values cultural diversity but also challenges racist, sexist and other discriminatory perspectives and practices in education, bearing in mind transformational pedagogies and evaluation approaches turned towards democratic tenets of society. Whilst democracy and inclusion deserve discussions of their own (Epstein, 1996; McGinn, 1996), as does multiculturalism and its many meanings and controversies (Canen, 2001b, 2003; Canen & Grant, 1999; Glazer, 1997; Grant, 2000; McLaren, 2000), a critical multicultural perspective as understood here should help view institutional evaluation as a complex process in which formative, diagnostic evaluative instruments aimed at valuing institutions' cultural distinctiveness and identities could be articulated with summative external evaluation processes aimed at providing clear criteria and goals for higher education institutions. Those evaluations should include indicators that make institutions accountable for the kind of knowledge they provide. However, that such knowledge be consistent with the democratic tenets of that society arguably should not be a hindrance towards its search for technological excellence. Brazilian Current SINAES Proposal: What Relationship with Knowledge Production and Technology? The current proposal of the National System for Evaluation of Higher Education in Brazil, called SINAES (2003), has tried to reedit the PAIUB principles (which were in place between 1994 and 1998, as discussed above) with respect to institutional identities, a global view of institutional performance and a commitment to inclusionary and democratic education, as opposed to the last government's evaluation policy that was perceived as an exclusively product-oriented approach to knowledge and evaluation. However, it is important to point out that the first challenge for a national system within the context of a society as multicultural and highly socially unequal as Brazil-a society in which regional, cultural and ethnic plurality prevail-is how to value cultural diversity and indigenous knowledges without falling into a relativistic approach to knowledge building and evaluation. Inextricably linked to that challenge, a second challenge refers to the plural scenario of higher education in Brazil. In fact, Brazil cannot be said to have a single system of higher education, but many systems, which include: public universities (federal and state universities run respectively by the federal and state governments with no student fees); private universities; and alternative private higher education institutions outside the university model (i.e. Higher Institutes for Teacher Education; University Centers; Institutional Evaluation, Knowledge and Multiculturalism: Some Ways Ahead in Brazilian Higher Education Current Issues in Comparative Education, Vol. 6(1) 21 Isolated Faculties; Technologic Courses and others, which have also had a boom in the last few years). In this respect, Amaral & Polidori (1999) point out that the expansion of the higher education system in Brazil has been mostly the result of the expansion of the private sector, with mixed results and uneven performance. Indeed, the expansion of the private sector in higher education has also been perceived by many as a direct consequence of the influence of international agencies in Brazilian educational scenario that deem public higher education as "not-profitable". The consequence is that institutions committed to research and ground breaking academic thought (most of the public universities and a few of the private ones can be said to fit into this profile) coexist side by side with others committed to alternative values and missions, mostly dedicated solely to teaching rather than research. This exceptionally pluralistic and complex scenario is paved with uneven academic results and impacts. In this sense, the sole use of relativistic strands of evaluation might risk losing objective standards for assessing the worth and merit of knowledge produced in this diverse scenario, even though they respect cultural institutional identities. Bearing the above in mind and building on the critical multicultural approach previously discussed, I suggest that the current proposal of the SINAES (2003) has gone more in the way of striking a balance between formative, culturally diverse evaluation strategies and external summative evaluation for accountability purposes than had the previously discussed institutional evaluation policies. In fact, the SINAES (2003) document demonstrates an aim to go beyond a dichotomized view of evaluation as discussed before in that it spells out its intention of valuing cultural diversity and plural institutional identities while at the same time defending regulation and quality control by central government. The following excerpt supports this assertion and describes the policy's main approach to institutional evaluation:"to have a far-stretching view of the evaluative processes without dissociating them from the necessary regulation and control by the State in fostering and supervising the system in its whole (...), which recognizes the importance of diversity but as long as it is compatible with the demands of quality, social relevance and autonomy" (SINAES, 2003, p.7). However, I also argue that in order to accomplish that, the document proposes a systematic evaluation approach consisting of some processes and structures that raise concerns about the transparency of evaluative criteria and to the way knowledge production and technology are conceived. Firstly, the regulatory perspective could be seriously menaced by the fact that the evaluation system proposed in the SINAES (2003) stops short of providing any evaluative grade, but rather culminates in long descriptive reports. One can infer that the resistance to providing any clear category or weights to the various indicators and aspects designed to evaluate institutions of higher education in more global terms stems from the opposition to league tables and classificatory assessment (as espoused in the "Provões") which the document associates with the market-oriented approach to education that the new government is keen to challenge. However, associating objectivist evaluation process (Scriven, 1994) as a whole to market or neoliberal strands could be an undue overgeneralization that risks being perceived as a denial to offer to society clear and concise evaluative statements about the quality of knowledge produced within the plurality of higher education institutions. That denial could further be reinforced by the kind of instrument deemed to replace the "Provões" for student assessment. In fact, by being targeted at a sample of students that could be Ana Canen December 17, 2003 22 voluntary and by putting the evaluative results within a framework of "areas" rather than "courses", as proposed in the document, that instrument could possibly fail to give a clear picture of institutional performance per faculty, department or other institutional unit, mitigating their approach to knowledge as well as their singular projects and impacts. Secondly, I suggest that the way knowledge and technology are perceived in the document has been colored by a dichotomous approach that tends to place in opposition the concept of knowledge for "market demands" (in which the role of technology is enhanced) and the concept of knowledge for citizenry, democracy and multicultural sensitivity (in which either the role of technology is silenced or is limited to the impacts of information technology in knowledge building and in promoting more equitable access to information). That seems to be implied, for instance, in one of the indicators for students' assessment within the proposed self-evaluative schedule of institutional performance within the referenced document. It spells out that institutional evaluation should "evaluate the social responsibility of the institution relative to its policies of expansion of entrance vacancies; [and] it should consider whether such policies are committed to criteria defined by public policies and knowledge development needs or rather to market demands"(SINAES, 2003, p.88). As inferred from the above excerpt, a dichotomous approach that separates social responsibility and democracy from what is perceived as "market demands" (without making it clear what such categories mean and with an implied negative view towards the latter) might risk sending a mixed message to higher education institutions that technical excellency and "market" relevancy of knowledge are negative aspects, thereby compromising projects and institutional cultural identities attuned to such values in knowledge building. Though other interpretations could be made concerning the message delivered in the document, its ambiguity could render it vulnerable to such misconstructions as suggested here. Rosenfield (2003) contends that government has colored its stand against financial investment in technological development and research with ideological propaganda that associates such an investment with market oriented approaches to knowledge. It is interesting to point out that although there is no explicit discussion about knowledge within the document, each time the categories "science" or "technology" are mentioned they are associated with expressions and ideas related to: their "inclusive nature" (as in the proposed evaluative indicators to assess curricula and programs); the extent to which the community has access to them (as in the proposed evaluative indicators to assess extension and socially geared institutional activities); the extent to which science is linked "to the needs of the population"; and "the scientific and social relevance of science" (as in the proposed evaluative indicators to assess the staff's scientific productivity). That seems to highlight the relationship between the SINAES, knowledge production, and technology as mainly built around the "usefulness" of the latter two to promote social equity and democracy. In fact, as can be noted, the SINAES (2003) seems to have substituted regulation through market values to regulation based on the extent to which higher education institutions adhere to democratic, inclusionary values. Nevertheless, as pointed out by Epstein (1996) and McGinn (1996), democracy and inclusion are complex issues that need to be clearly addressed and defined. However noble and coincidental those regulatory ideals Institutional Evaluation, Knowledge and Multiculturalism: Some Ways Ahead in Brazilian Higher Education Current Issues in Comparative Education, Vol. 6(1) 23 espoused by the SINAES (2003) proposal may be to a multicultural perspective defended here, the lack of clarity about what those categories mean and how they are to be translated into higher education institutions leave a vacuum that may be filled by misconstrued views that pit technological development for the purpose of academic excellence against purposes of democratic consolidation, to the detriment of the former. That such a dichotomous approach should prevail in the document becomes even more worrisome when one looks at the centralizing approach to the management of the evaluation process as a whole. For instance, the SINAES proposes a national central evaluation committee of twelve members chosen by the President of Brazil. This committee would be comprised of experts in education and evaluation whose duties would include planning, supervising, collecting documentation and giving a final say in the evaluation process before sending the results to the Ministry of Education for appropriate actions. The homogeneous composition of the centralized national evaluative committee and its worrisome direct and close link to central power could arguably mean a preponderance of the regulatory and controlling dimension of the evaluation to the detriment of its multiculturally-oriented one. Likewise, even though claiming there should be an institutional self-evaluation dimension, as an intrinsic part of the institutional evaluation process as a whole, so as to recognize and appreciate a plurality of cultural identities among institutions and to allow a formative approach to evaluation to flourish, the fact that such a self-evaluative dimension should be done based on a checklist previously elaborated by the National Central Committee might be harmful to the initial multiculturally oriented goal. In order to ensure representation of the academic community and civil society at large on that committee and, more importantly, to provide a space for alternative models of knowledge building and technology enhancement disparate from a pre-determined ideological straight-jacket, other criteria for both the selection of committee members and institutional self- evaluation should be sought. By underplaying the role of technology development for knowledge building in higher education, government rhetoric seems to be sending a message of an apparent dissociation of national educational values from international agencies' "productivity oriented" proposals, conveying a sense of a "nationalistic", democratic and inclusive approach to knowledge. However, paradoxically, such a reduction of the concept of knowledge could well mean an increasing dependence on those very agencies for access to new technologies and "developmentally useful" knowledge (King & McGrath, 2002), since national minds and projects would be stifled by a lack of financial resources and support. In a critical multicultural approach (Canen, 2001b, 2003; Canen & Grant, 1999; Glaser, 2000; Grant, 2000; McLaren, 2000), I have argued that we should be wary of dichotomies and rather glean the constructed meanings of knowledge in their multiple hybridizations and loci, lest we risk falling into a paradox of stifling institutional diversity and technology enhancement projects under the very flag of multiculturalism, democracy and inclusion. Conclusions The present paper discussed institutional evaluation in Brazil under a critical multicultural approach. It contended that even though the current document with a proposal for a national institutional evaluation system has moved more in the direction Ana Canen December 17, 2003 24 of trying to balance respect for cultural diversity and regulation for quality control in evaluation of higher education institutions, some imprecisions and imbalances should be addressed so as to ensure that that aim is achieved. It has particularly highlighted the need to spell out the expected objectives for higher education in Brazil as well as to clearly state the meaning of knowledge and the criteria to judge its merit and worth in accordance not only with cultural diversity and democratic values, but also with technological development and the recognition of its impact on knowledge building. It has spelled out concerns about a misconstrued and pervasive view within the document that dichotomizes technological development for knowledge for excellence and knowledge for democracy, to the detriment of the first one. In this sense, it has also argued for the need to challenge a homogenizing and centralizing dash in the structures and processes of the proposed institutional evaluation process, so as to ensure a plural representation deemed central for a multiculturally oriented institutional evaluation. It was contended that such a step would allow higher education institutions to give air to alternative models of knowledge building and technology enhancement not only away from the dichotomous approaches discussed above, but also from ideological straightjackets that stifle knowledge building and technology enhancement under the flag of vague and ill-defined notions such as those of "usefulness" or "social relevancy" of knowledge. It should be pointed out that discussions of institutional evaluation in Brazil within the context of the new government are still in the "first half of the game", as stated by the representative of the Secretary of State for Higher Education in Brazil during a recent meeting with the academic community to discuss the issue. Much is surely going to pop up, particularly in the wake of the several debates the SINAES proposal has been engendering inside and outside the academic community. The current proposal is certainly a step forward in the new Brazilian government efforts of seeking alternative models of knowledge and institutional evaluation that keep well away from neoliberal strands perceived as exclusive and that value cultural diversity and democratic societal values. Rather than insurmountable obstacles, the risks and challenges of such an enterprise as pointed out within the limits of the present article should be thought of as stepping stones leading to new, more multiculturally-oriented and significant models of knowledge for the contemporary world. Such models should value cultural diversity and democratic values but nevertheless not stifle technological enhancement and alternative approaches to knowledge building. Brazilian challenges to achieve that aim could offer precious insights to other multicultural countries and provide possible options to be worked out for a new perspective in evaluation and knowledge building for the present millennium. In memory of Professor Nigel D.C. Grant Institutional Evaluation, Knowledge and Multiculturalism: Some Ways Ahead in Brazilian Higher Education Current Issues in Comparative Education, Vol. 6(1) 25 References Amaral, A. & Polidori, M. (1999), Quality Evaluation in Brazil: a competency based approach? Higher Education Policy, V. 12, pp. 177-199. Canen, A. (2001 a), Avaliação da Aprendizagem em Sociedades Multiculturais. Ed. Papel & Virtual, www.papelvirtual.com.br Canen, A. (2001b), Universos Culturais e Representações Docentes: subsídios para a formação de professores para a diversidade cultural, Educação e Sociedade, V. 77, pp. 207 - 227. Canen, A. (2003), Child Education and Literacy Learning for Multicultural Societies: the case of the Brazilian National Curricular References for Child Education (NCRs), Compare, V. 33, N.2, pp. 251-264. Canen, A. & Grant, N.D.C. (1999), Intercultural Perspective and Knowledge for Equity in the Mercosul Countries: limits and potentials in educational policies, Comparative Education, V. 35, N. 3, pp. 319 - 330. Canen, A. & Oliveira, M.T. (2000), Institutional Evaluation in the Mercosur: transformation or control?, International Journal of Educational Development, N. 20, pp. 277 - 286. Epstein, I. (1996), Educating Street Children: some cross-cultural perspectives, Comparative Education, V. 32, N.3, pp. 289-302. Glazer, N. (1997), We Are All Multiculturalists Now. Massachusetts: Harvard University Press. Grant, N. (2000), Multicultural Education in Scotland. Edinburgh: Dunedin Academic Press. King, K. & McGrath, S. (2002), The Globalization of Development Knowledge and Comparative Education, Compare, Vol. 32, No.3, pp. 281-284. Lincoln, Y. & Guba, E.G. (2000), Paradigmatic Controversies, Contradictions, and Emerging Confluences. In: Denzin, N.K. & Lincoln, Y.S. (eds.), Handbook of Qualitative Research. California: Sage Publications, pp. 163-188. McGinn, N. F. (1996), Education, Democratization and Globalization: a challenge for comparative education, Comparative Education Review, V.40, N.4, pp. 341-357. McLaren, P. (2000), Multiculturalismo Revolucionário.Porto Alegre: ArtMed Editora. Penna-Firme, T. (1995), Avaliação de Inovações Educacionais, Seminário Internacional de Avaliação da Educação, pp. 67 - 74. Rio de Janeiro: Funcação CESGRANRIO (printed). Ana Canen December 17, 2003 26 Polsani, P. R. (2002), Education in the Network: knowledge flows and learning nodes, Ristoff, D. I. (1996), Princípios do Programa de Avaliação Institucional, Avaliação, N.1, pp. 47-53. Rosenfield, D. L. (2003), Aliança: MST & Monsanto, O Globo, 1 December 2003, p.7. Scriven, M. (1994), Evaluation as a Discipline, Studies in Educational Evaluation, V. 20, N.1, pp. 147 - 166. SINAES (Sistema Nacional de Avaliação da Educação Superior) (2003), Comissão Especial de Avaliação, Secretaria do Ensino Superior (SESu), Ministério da Educação, Brasília, Brazil. Smith, N. (1980), Evaluation Studies: sources of values influencing educational evaluation, Studies in Educational Evaluation, V. 6, pp. 101-118. Sobrinho, J. D. (1996), Avaliação Institucional: marco teórico e campo político, Avaliação, N.1, pp. 15-24.