7 issn 1822 - 8402 european integration studies. 2009. no 3 R&D poLICY IMpACT oN RESEARCH EXCELLENCE Kristina Babelytė Birutė Mikulskienė Mykolas Romeris University Abstract facing the worldwide economic recession, the public research and development (R&D) policy undergo the increasing pressure to economize and improve research performance levels in this way complying with public accountability. Thus the design of R&D policy has to integrate an appropriate set of research performance criteria together with the balanced choice of relevant indicators for the allocation of scarce funds for the very purpose of pursuing scientific advancement in the frames of international rivalry. During several last decades the discourse of the Eu politicians expanded with the term of “research excellence”, which became conceptual background for further Eu R&D policy development. often excellence in research is used as synonym of “research quality”. Nevertheless, rivalry for the highest evaluation of research performance has not yet provided with a single and fixed definition of “research excellence”. This paper aims to outline the content of excellence in research and, investigating the existing Lithuanian experience to measure it, discuss the R&D policy intervention impacts on the process and the results of pursuing high level research. The methods of research used are grounded on the investigation of Lithuanian legal acts, valid in the field of R&D and conditioning the performance of public R&D institutions. Two levels of policy intervention: institution and individual are taken into consideration. Also, in the light of research excellence measurement R&D output assessment methodology used by the Government as a basic institutional funding instrument is analyzed. The conclusions mark that due to the conditions of ambiguous definition of “research excellence” the need for the methods to recognize and identify excellence in research are vital. However, its pro-active state- level management practices should contain an umbrella of flexible, well-balanced and situation receptive procedures. Therewith, the research excellence encouragement instruments within the framework of the state’s R&D policy should cover the individual and institutional level endorsed and promoted by sustainable and diversified measures. Keywords: Research excellence, research performance, R&D policy. Introduction Present policy discourse most frequently presupposes a notion of “excellence” in research as a synonym of high quality research, consequently to the perception of qualitative higher education. Increasing public pressure to find a comprehensive solution between world-class quality research and development (R&D), optimum allocation of funds, and accountability, influences the design of policy, the issue to measure the research performance at the same time pursuing scientific advancement, especially on international arena. Even though excellence in research is generally understood to be desirable, it is rarely defined in policy in detailed and even more rarely measured. (Tijssen, 2003). The result of research is under high level of uncertainty and risk – with regard to the choice of a method of research, expected outcomes, raised hypothesis, etc., at the same time pursuing for the better, higher result. May the notion of “research quality” be the synonym of the “research excellence”? The “quality” in research pertains to the conformity of the standard requirements embodied with the ambition for the highest level of performance. Therefore, under the conditions of such uncertainty, specific instruments or methods to discover higher level of quality in research is necessary. The latter pertains to the creation and development of the perception of “quality” or even “excellence” in research, especially on the state’s policy level. From R&D policy-makers’ perspective, however, modeling and measuring are sine qua for decision-making, thus, generally, R&D policy impact on research quality or excellence directly correlates to the set R&D policy objectives. As E. Garfield and A. Welljams-Dorof states, ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 8 government policy-makers, corporate research managers, and university administrators use R&D indicators for a variety of purposes, like measuring the effectiveness of research expenditure, prioritizing strategic planning, etc. (Garfield, 1992) in this way intervening the very input, process and output of research. At the same time, research quality management (basically, planning, control and improvement) is indispensible constituent of the whole R&D policy framework (Vukovic et al., 2007). The aim of the article is to outline the content of excellence in research and, investigating the existing Lithuanian experience to measure it, discuss the R&D policy intervention impacts on the process and the results of pursuing high level research. The methods of research used are grounded on the investigation of European initiatives as well as Lithuanian legal acts, acting in R&D field. The focus on Lithuanian R&D policy lays in the deeper analysis of the way R&D outputs are assessed and measured. The qualitative standards to researchers, who conduct the R&D, are investigated in the light of excellence of R&D performance. The Formal Assessment Methodology of Research Production of Research and Higher Education Institutions and the Order of Minimum Qualification Requirements for the Positions of Scientific Workers, Other Researchers and Lecturers at Research and Higher Education Institutions are studied. The findings of such analysis presuppose consecutive results and are followed by the conclusions, forming the essential tendencies of Lithuanian R&D policy with regard to encouraging “excellence” in research. Is excellence the objective for research policy? Even though the importance of excellence in research has never been questioned, academic attention to the very perception of “excellence” was paid in the 1970s and 1980s by sociologists and psychologists, seeking to explore the origins of the stratum of scientific elite (Jackson, 1987). Today research excellence has gained quite an exceptional status and has become one of the leading themes and/or aims of research and development policy, invoking the utilitarian and economic attribute with the foreground of “competitiveness”. (Hicks et al., 2000). “Governments need systematic evaluations for optimizing their research allocations, re-orienting their research support, rationalising research organizations, restructuring research in particular fields, or augmenting research productivity. In view of this, they have stimulated or imposed evaluation activities.” (Moed, 2005, p. 15). In other words, the concept of “excellence” on public R&D policy discourse gained the form of impetus for forming the framework of R&D policy decisions originated, related or designated within frames of the demand of “excellence” in research, in a way converting one of the constituents of the collection of a certain quality management perception. (Vanagas, 2004). However, it should be stressed that “excellence“ in research has never been defined as a legal concept. The very perception that it refers to the measurement of research perfomance exists across the world (Patton, 2002). Within the perspective of the EU, the term “research excellence“ often overlaps with the perception of “research quality“, surveying the linkage between measurement of scientific performance and science policy decisions. (Leydesdorff, 2005). With the Green Paper on the European Research Area (ERA) (2007), the European Commission launched a broad institutional and public debate on what should be done to create a unified and attractive ERA, which would fulfil the needs and expectations of the scientific community, business and citizens. “The right balance should be found between competition and cooperation. Researchers and research institutions should be stimulated by higher levels of competition on a European level to develop world-class excellence. At the same time, they should be engaged in stronger cooperation and partnerships across Europe and beyond to effectively address issues of common concern.” (Green Paper on the European Research Area, 2007, p. 12). Furthermore, the search of unified guidelines on the policy level in providing clear steps for universities to enhance the quality (or excellence) of their country’s research area, and of the ERA as a whole, stipulated the European Commission to head for the overview of the member state’s policies to promote quality/excellence in research. CREST OMC 3% working group on “Mutual learning approaches to improve the excellence of research in universities” recently provided with the draft report (February, 2009). It offers a great variety of information relevant for the national policies to promote research excellence: best practices, lists of advantages and disadvantages of certain instruments, inventories, etc. It should be noted that no common definition for the “quality” or “excellence” in research is offered, since the very objectives of pursuit of “quality” or “excellence” frame their understandings, either by the requirement of research funders, disciplines or criteria of existing assessment instruments. Nevertheless, the report states that there appears to be a common set of instruments widely implemented in all or most member states. These core set of desirable, basic instruments include ensuring excellence promotion in research through competitive and performance-oriented funding (institutional and individual), some form of evaluation or accreditation, an infrastructural development plan, and the promotion of institutional quality management. ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 9 perception of research excellence at the state’s level Analyzing Lithuanian R&D policy in the light of encouragement of research performance in the research and higher education institutions with the specific objective to pursue research excellence, it should be noted that none of the legal acts uses “research excellence” as a legal concept. Notwithstanding, Lithuania is not unique in this respect. This is not to state that in Lithuania research excellence has never been as a pursuit of either academic society or R&D policy making level. However, speaking in the general terms “research excellence” is more regarded as R&D policy measure. Basically, two legal acts form the perception of “quality” in research on the policy-decision level in Lithuania. Two different approaches could be distinguished, namely institutional and individual level. From the institutional perspective, the Formal Assessment Methodology of Research Production of Research and Higher Education Institutions, approved by the Order of the Minister of Education and Science of the Republic of Lithuania No. ISAK-1215, 30th April, 2008 (OJ, 2008, No 56-2128) (hereinafter – Formal Assessment Methodology), outlines the framework of measurement of scientific production, based on any possible research outputs. The incentive targeted at individual level is the Resolution of the Government of the Republic of Lithuania No. 906 on Approval of the Order of Minimum Qualification Requirements for the Positions of Scientific Workers, Other Researchers and Lecturers at Research and Higher Education Institutions; of the Order of Organization of Competitions for the Positions of Scientific Workers, Other Researchers and Lecturers at Public Research and Higher Education Institutions and of the Order of Awarding of Pedagogical Names at Universities (OJ, 2005, No 102-3786) (hereinafter - Minimum Qualification Requirements for the Personnel of Research and Higher Education Institutions). Research excellence acknowledgment: institutional level The most prominent policy making tool in respect to “research quality” at institutional level outstands in the form of the Formal Assessment Methodology. This methodology is mainly based on the internationally acknowledged quantitative principles. The number of publications, monographies, patents, cooperation with business, governmental sectors, and participation in R&D projects locally and internationally are under consideration. According to the internationally agreed scientific criteria’s, such as average citation index, with regard to the categories of scientific periodicals, the ISI database and other international databases containing scientific information, research outputs are measured every single year. It should also be noted that in the Formal Assessment Methodology the scientific production within the field of social sciences and humanities (SSH) is assessed separately from physical, biomedical and technological (PhBT) science fields, with different attributes for every assessment item of scientific production. Considering of the principle that “the measurement of excellence can be based on indicators (quantitative) and “descriptors” (qualitative)” (Luukonen et al., 2006, p. 250), it may be stated that the Formal Assessment Methodology includes expert assessment of monographies; significant scientific works, their translations into other languages; material of presentations read in the scientific conferences, scientific reviews in periodicals and other continuous scientific publications, etc. In general terms, the procedural tools of the assessment of research production according to the Formal Assessment Methodology ensure ex-post evaluation and are basically connected to the distribution of state funds (the so-called basic institutional funding). At the same time this methodology seeks to improve efficiency of research and higher education institutions and increase their international rivalry. In order to seek the policy impact of the assessment of research production of research and higher education institutions, let us analyze the dynamics of publications during the period 2000-2008. The dynamics of the number of ISI articles (Fig. 1) reflects the dramatic increase of scientific publication tendencies for both SSH and PhBT. The set of Lithuanian research journals became listed in ISI database during this time. 0 200 400 600 800 1000 1200 1400 2000 2002 2004 2006 2008 Year N o of IS I a rt ic le s biomedical, technical and physical sciences social sciences fig. 1. publications of articles of Lithuanian research and higher education institutions in ISI database Data source: Prepared referring to the Report on the Activities of the Centre for Quality Assessment in Higher Education in 2008. Vilnius: Kopa, 2009. ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 10 Putting aside the assumption that the citation data depend on many factors, including research field, scientific journal, a kind of an article and author/ reader dependent factors (Bornmann et al., 2008), the first glimpse at Fig. 2 below indicate that the Formal Assessment Methodology does not efficiently encourage higher quality of the published research articles – higher citation index in the ISI database. 0 1 2 3 4 5 6 1998 2000 2002 2004 2006 2007 Year A ve ra ge c ita tio n N o du rin g th e pu bl ic at io n an d tw o su bs eq ue nt ye ar s biomedical, technical and physical sciences social sciences fig. 2. Average citation index of articles of Lithuanian research and higher education institutions in ISI database Data source: Prepared referring to the Report on the Activities of the Centre for Quality Assessment in Higher Education in 2008. Vilnius: Kopa, 2009. If the main impact of the Formal Assessment Methodology was the burst of research production via publications, it should be named that the miner effect was that national journals increased their quality and were granted by ISI citation index. From the data above it emerges that the Formal Assessment Methodology encourages the research and higher education institutions produce more scientific publications without addressing themselves to seek for the higher number of the citations of the articles, event though the latter data indicates higher quality of the scientific production. Concisely and with regard to the perception of “quality” in research, it should be stated that the Formal Assessment Methodology presupposes the network of the standard requirements to be identified as belonging to the “qualitative research” dimension. Since grounded mostly on quantitative principles, it is quite substantial supporting start for the bibliometric analysis of the research production of research and higher education institutions. Present Formal Assessment Methodology encourages the researchers, their groups or research and higher education institutions to seek that their articles are printed in the ISI cited publications, however, since this provision was strengthened only in 2007, clear reflection of its successive results would be slightly too forward. The Formal Assessment Methodology does not hint about the exact number of certain kinds of articles or likewise, it might be stated that it serves as the reference for R&D producers on what is assessed and based in certain methods on the state’s level. Accordingly, the Formal Assessment Methodology may be considered neither a stimulus for “research excellence” nor a declaration of “research quality”. Therefore consecutive suggestions may follow that in order to balance the assessment of the research production of research and higher education institutions necessary changes may include, for example, assessment of scientific production of the higher level only, not every single unit of the scientific production; comprehensive expert evaluation of scientific monographies; or number of patents issued to Lithuanian authors and registered in the European Patent Office, US Patent and Trademarks Office or Japan patent Office (Daujotis et al., 2002). The latter idea may be reasoned by the continuous process of pursue of scientific advancement “at all stages of research process, from basic to applied research as well as in collaboration and partnership with the business community as part of research and innovation ecosystems within non-linear complex innovation processes.” (Diversified Funding Streams for University-based Research: Impact of External Project-based Research Funding on Financial Management in Universities. European Commission Expert Group Report, 2008, p. 5). Certainly, differentiated assessment procedures with regard to separate science field should be retained (van Raan, 2003; 2005), perhaps even differentiating between social sciences and humanities, in this way encouraging the quantitative as well as qualitative dimension of separate scientific production. It should be stressed that on the ground of quantitative results from annual application of the Formal Assessment Methodology, the basic institutional funding is distributed. Tight correlation between research output assessments and the funding scheme is considered as a resulting factor to strive for the research and higher education institutions for higher research performance, what, sequentially, could be named as the pursue of research quality. Diversification of the research funding sources with more emphasis on the competitive research funding share by no doubts influences the strategies of the research and higher education institutions externally (Flick, 2006). Competitive nature of such research funding accommodates the concept of “quality” in research with a dimension of ex-ante evaluation of non- existing, yet hypothetical research output, when dealing with good intentions and brilliant innovative ideas. The competitive research funding is based on different criteria than the so-called basic institutional funding. ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 11 With regard to the concept of “quality“ or “excellence“ in research, the problem of research measurement arises. The essential principle of competitive research funding obliges to measure not the “quality“ or “excellence“ of research output, but the ability of a researcher to produce “excellent“ research. In other words, the application for the competitive research funding does not supply the funder with the research output; it only contains the plan to pursue it. Forasmuch as mentioned above, the strategies of the research and higher education institutions include instruments complying with the standard requirements to receive funding during competition, but at the same time, they pertain the ambition to show the highest performance indicators among the competitors. In succession, the scientific productivity plays a vital role within this research funding scheme. Therefore, clear formulation and application of the relation of autonomy and accountability of research and higher education institutions shape the background for integrated system of organizational policies and management practices. According to M. Gibbons, C. Limoges, H. Nowothy et al. “quality control has two main components: one is institutional and concerns the spatial position of a particular research activity in the cognitive landscape; the other is cognitive and pertains to the social organization in which such research is performed.” (Gibbons et al, 1994, p. 32). Freedom to decide upon the direction of the institution’s research performance results in every possible warranty of open mind of its scientific community. However, regardless restless mind of a researcher, the research and higher education institution should form the boundaries for the constructive outcomes of unequivocal drive for research. The principle of accountability, either to the research funder(s) or the society itself, should comprehend to the reason for research itself. Research quality in this frame plays the role of conjunction between the researcher, research and higher education institution and the end-user, i.e., society (Fujigaki et al., 2000). Research excellence acknowledgment: individual level Human capacity building derives as one of the most important research quality factors. The Minimum Qualification Requirements for the Personnel of Research and Higher Education Institutions indicates the perception of “quality” in research with regard to individual level. This legal act sets out compulsory minimum requirements for the persons, participating in public competition and seeking to occupy the positions of scientific workers, other researchers, lecturers at public research and higher education institutions. The minimum qualification requirements are differentiated by the scientific field, but mostly depend on the level of the position – professor, docent, chief/senior/young/ researcher, etc. In general, the number of scientific publications in ISI and other international scientific databases, recognized by Lithuanian Research Council, the number of international research projects, scientific conferences and scientific traineeships, the number of prepared monographies, textbooks and other methodological means, the number of scientific reviews, etc. are taken into account. Additional specific criteria may be applied by the research and higher education institutions as employers, but the inclusion of the set minimum standards, indicated on the state’s level, is compulsory. Within the frames of the Minimum Qualification Requirements for the Personnel of the Research and Higher Education Institutions the perception of “quality” deviates as a starting position for improvement. This legal act provides with the lowest obligatory limit for an individual, but at the same time allows possibilities to attain higher level. Therefore it forms the basics of the perception of “quality” regarding a researcher, but surpasses the higher expectations from him/her. Matching human capacity to research excellence The first attempt to match human capacity with research excellence by the research output and bring the dimension of coordination of the centres of excellence on national level and mapping of them in Lithuania was first analyzed by the group of researchers in 2008 (Bar auskas et al., 2008). The study provided with detailed international practice on the methods of coordination of the centres of excellence and revealed the possible scenarios for the optimization of national financing scheme of the centres of excellence. A priori the study stated that due to immense research potential in Lithuania, significant to the country and abroad, the created national R&D funding system with special attention to research excellence would allow strengthening the research quality and extent of the research groups, their international significance and, most importantly, use of their results to the state’s economic and social welfare. An attempt to identify the advanced and internationally competitive groups of researchers was made. Within the frames of the same study advanced and internationally competitive groups of researchers among Lithuanian academic society were identified and assessed during the so-named ”call for centres of excellence”, carried out in 2008. It was based on the mapping of the groups of researchers working on similar or the same research issue as a team or a group of researchers, seeking for common result on one issue or in the same or similar field. For identification of originally formed centres of excellence the basic set of indicators were sorted according to the methodology, picking up just the most significant ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 12 indicators of research excellence (the number of PhDs, identifying themselves within the centre of excellence; number of scientific monographies or publications equal to them; number of publications in the periodicals, included in the list of database approved by the Lithuanian Research Council; number of publications in foreign journals and publications of scientific references, works of applied research). Certainly, additional indicators like number and size of projects, international expertise, etc. were also paid attention to. Besides, the list of applied general criteria for the assessment of the centres of excellence were competitiveness, conformity between the centre’s themes and tasks, dissemination of the centre’s results and international acknowledgement. Notwithstanding, it should be noted, that even though the assessment of the centres of excellence was differentiated between the technical, social sciences and humanities, it was performed one time. The permanent feature of this kind of evaluation of the centres of excellence in Lithuania, like performed on a yearly basis, may supply with better coherence of the R&D policy- making measures and presuppose the solid impetus for the centres of excellence themselves to further actively perform research and strive for the highest-rated results on national and, especially, international scene. Despite this, the mapping of the Lithuanian research potential allowed identification of the advanced and less advanced research fields on national context at the same time enabling the segregation of the researchers themselves. Conclusions Similarly to every other member state of the EU or worldwide, the perception of ‘quality’ in research on Lithuanian R&D policy level is a non-legal concept. However, its impact in Lithuanian R&D policy discourse determines the formation of R&D policy and may be considered as the analogue constituent of the collection of certain quality management. The legal framework in R&D field conditions the very notion of pursuit of quality in research as an administrative instrument, i.e., the justification for the public funding, and at the same time refers to the purpose of effectiveness of research results. In other words, the notion of research quality on Lithuanian R&D policy- making landscape pertains to the requirements of the research funding and accountability dimensions defined by the existing assessment methods or criteria, applied both – on institutional and individual level. However, policy intervention (annual assessment of research outputs) does not define the upper limit for the scientific advancement. Either Lithuanian research and higher education institutions or researchers and groups of them are allowed to pertain to the conformity of the standard requirements embodied with the ambition for the highest level of performance. Successively, the concepts of “quality” and “excellence” in research should be differentiated. This is not to say that these two concepts are not inter-related, on the contrary, but with regard to the analysis described in this article, the concept of “excellence” within the R&D policy discourse is more likely to be identified as the subsequence of the “quality” standards and denotes the higher dimension of the “quality” itself. Herewith “research excellence” plays the role of impetus for the higher research performance. Nevertheless, it should be noted that coherent outlook of R&D policy with regard to the subject of quality in research, addressing both institutional and individual level, and in this way forming the basis for the state’s competitive potential in the this sphere, simultaneously influences the research input, output and process. Undoubtedly, competitive research funding schemes must prevail versus the institutional funding. In succession to this, the research performance assessment methods in Lithuania should be reviewed with the aim to include research carried out previously, the proposed research as well as potential for renewal in relation to the international research frontline. In other words, excellent research requires sound and pro-active management practices. References Baršauskas P., et.al. Išskirtinių mokslinių tyrimų vadyba. Lietuvos mokslo taryba, ISM Vadybos ir ekonomikos universitetas, 2008. Bornmann L., Mutz R., Neuhaus Ch., Daniel H.D. Citation Counts for Research Evaluation: Standards of Good Practice for Analyzing Bibliometric Data and Presenting and Interpreting Results. /Ethics in Science and Environmental Policies/ Vol. 8, 2008. P. 93-102. Daujotis V., Radžvilas V., Sližys R., Stumbrys E. Lietuvos mokslo politika Europos kontekste. Vilnius: Justitia, 2002. Diversified Funding Streams for University-based Research: Impact of External Project-based Research Funding on Financial Management in Universities. European Commission Expert Group Report. Brussels, 2008. http://www.eurosfaire.prd.fr/7pc/ doc/1236263443_eg_external_research_funding.pdf [2009-03-20] Flick, U. Managing Quality in Qualitative Research. SAGE Publications Ltd., 2006. Fujigaki, Y., Leydesdorff, L. Quality Control and Validation Boundaries in a Triple Helix of University-Industry-Government: ‘Mode 2’ and the Future of University Research. Vol. 4, No. 39, Social Science Information, 2000. P. 635-655. ISSN 1822-8402 EUROPEAN INTEGRATION STUDIES. 2009. No 3 13 Garfield E., Welljams-Dorof A. Citation Data: Their Use as Quantitative Indicators for Science and Technology Evaluation and Policy-Making. /Science and Public Policy/ Vol. 5, No. 19, The Scientists, 1992. P. 321-327. Gibbons, M., Limoges, C., Nowothy, H. et al. The new production of knowledge – the dynamics of science and research in contemporary societies. SAGE, 1994. P. 179. Green Paper on the European Research Area, Brussels, 4.4.2007 COM(2007) 161 final. http://ec.europa.eu/ research/era/pdf/era_gp_final_en.pdf [2009-03-18] Hicks D., Breitzman A., Hamilton K., Narin F. Research excellence and patented innovation. /Science and Public Policy/ Vol. 27, No. 5, Beech Tree Publishing, 2000. P. 310-320. Jackson D.N., J.P. Rushton J.P. Scientific Excellence: Origins and Assessment. Newbury Park: Sage, 1987. Kristapsons J., Martinson H., Dagyte I. Baltic E&D Systems in Transition: Experiences and Future Prospects. Zinatne, Academic Publishers. 2003. Leydesdorff L. Evaluation of Research and Evolution of Science Indicators. /Current Science/ Vol. 89, No. 9, 2005. P. 1510-1517. Luukkonen T., Nedeva M., Barre R. Understanding the dynamics of networks of excellence. /Science and Public Policy/ Vol. 33, No. 4, Beech Tree Publishing, 2006. P. 239-252. Moed, H. F. Citation Analysis in Research Evaluation. Springer, 2005. P. 346. Mutual Learning on Approaches to Improve the Excellence of Research in Universities. CREST 4th OMC Working Group Report (draft). Brussels, 2009. Order of the Minister of Education and Science of the Republic of Lithuania No. ISAK-1215 on Approval of Formal Assessment Methodology of Scientific Production of Research and Higher Education Institutions. OJ 2008, No 56-2128. Patton, M. Q. Qualitative research and evaluation methods. 3rd ed. SAGE, 2002. P. 598. Resolution of the Government of the Republic of Lithuania No. 906 on Approval of the Order of Minimum Qualification Requirements for the Positions of Scientific Workers, Other Researchers and Lecturers at Research and Higher Education Institutions; of the Order of Organization of Competitions for the Positions of Scientific Workers, Other Researchers and Lecturers at Public Research and Higher Education Institutions and of the Order of Awarding of Pedagogical Names at Universities. OJ 2005, No 102-3786. Report on the Activities of the Centre for Quality Assessment in Higher Education in 2008. Vilnius: Kopa, 2009. http://www.skvc.lt/files/2008_ataskaita/ SKVC_ataskaita-galutine.pdf [2009-04-04] Tijssen, R. J.W. Scoreboards of Research Excellence. /Research Evaluation/ Vol. 3, No.2, Beech Tree Publishing, 2003. P. 91-103. van Raan A. F. J. Measurement of Central Aspects of Scientific Research: Performance, Interdisciplinarity, Structure. /Measurement/ 3 (1). Lawrence Erlbaum Associates, Inc., 2005. P. 1-19. van Raan A. F. J. The Use of Bibliometric Analysis in Research Performance Assessment and Monitoring of Interdisciplinary Scientific Developments. / Technology Assessment – Theory and Practice/ 12 (1), 2003, P. 20-29. Vanagas P., Visuotinės kokybės vadyba. Kaunas: Technologija, 2004. Vukovic A., Pavletic D., Ikonic M. Basic Approaches to Total Quality Management and the Fundamental Concepts of Excellence. 27-2, 2007. P. 71-81. hrcak. srce.hr/file/41658 The article has been reviewed. Received in March, 2009; accepted in April, 2009.