87 The concept of quality in the public realm The concept of quality has started to become an important factor for public organizations starting with the ‘90s. Before the World War II quality was only used in order to describe the physical characteristics of an industrial product. More recently, the concept has been applied to the service sector as well. This trend is accompanied by a shift in perspective as well. At the beginning one may talk about conformity in relationship to existing standards. Then the conformity is evaluated against the expectations of the clients, costs, and more recently against latent expectations (the ability to anticipate demand). Today the concept of quality includes both technical aspects (focus is on the client) as well as functional aspects (focus is on how the interaction with the client is carried out). Several of the most important functional factors include1: 1. Tangible assets: Buildings, equipments, personnel, communications; A FRAMEWORK FOR THE SELF- EVALUATION OF THE FUNCTIONING OF PUBLIC INSTITUTIONS: INCENTIVE TOWARD AN INCREASED QUALITY OR MERELY A FORMALITY? Sorin Dan ŞANDOR Associate professor, The Department of Public Administration, the Faculty of Political, Administrative and Communication Sciences, The “Babeş- Bolyai” University, Cluj-Napoca Public administration can no longer ignore quality. Total Quality Management is trying to change the philosophy of the area. There are many TQM-inspired tools that can help this process: quality (or excellence) awards, ISO standards, Common Assessment Framework (CAF). CAF is a tool developed by the European Union in order to assess the capacity of public institutions to produce quality. Comprised of two main dimensions, enablers and results, nine criteria and 27 sub-criteria, the instrument allows a self-evaluation of the state of a specific institution. Romania intends to apply this framework in public administration. We have tried to see how this can work and several problems did occur: essential ingredients are missing – mainly systematic planning and performance measurement. Beside this, subjectivity and lack of commitment and knowledge can undermine such assessments. Romanian Public Administration Reform Strategy needs to start from the beginning, involving the entire organization towards quality, planning, developing and improving constantly performance measurement systems, without those being impossible to speak of quality management. Transylvanian Review of Administrative Sciences, 15 E/2005, pp. 87-95 1 Valarie A. Zeithaml, A. Parasuraman, Leonard L. Berry, Delivering Quality Service: Balancing Customer Perceptions and Expectations, NY, The Free Press, 1990, p.226. 88 2. Credibility: The ability to act upon your obligations as rigorously as possible; 3. Promptness: The willingness to help clients, to be as prompt as possible; 4. Expertise: Abilities and knowledge; 5. Politeness: Respect, consideration, and cordiality; 6. Honesty: Correctness and rigor; 7. Surety: The lack of danger, of risk, and uncertainty; 8. Communication: Informing the client in a timely manner while using an easy to comprehend language; 9. Access: How easy it is for the client to get in touch with the provider; 10. The understanding of the client: efforts made to understand the client and its needs. The change in attitude with regard to quality is due mainly to a model called Total Quality Management (hereafter TQM). TQM is comprised of a set of principles, tools, and procedures that help accomplish the mission of the organization both from a qualitative and quantitative standpoint. TQM is a managerial philosophy that is accomplished within the framework of a managerial system that promotes a continuous improvement with regard to all the activities within an organization. The process of continuous improvement involves three key dimensions: focus o the client; betterment of processes; and total involvement. In this context one should also refer to a change in the approach with regard to the relationship that exists among management, employees, and clients. The client becomes extremely important and in order for his needs and expectations to be fulfilled the employee is the key actor. The employee is to be considered the main provider of quality. The role of management is to support and engage employees so that they obtain the desired outcomes. Employees need to enjoy autonomy, the decision- making process is decentralized, and the leadership of the organization promotes collegiality and it is not based on hierarchy. An organization that functions according to the principle of TQM is characterized by flexibility, which is necessary to adapt to an ever-changing environment; within this environment the organization functions as a cohesive entity that is not based on hierarchy and rigid rules but it rather implies the adhesion of all members to the goals and values of the organization, whom they are able to establish and adjust themselves. In 1988 14 major corporations have created the European Foundation for Quality Management with the declared goal of promoting quality management in Europe. In 1991 the European Award for Quality2 has been created; it promoted the Model of the European Award for Quality- this name has been changed in the 1997 into the European Model for Excellence, known also by its short name of EFQM. The success of this model is the result of its simplicity; it is also due to the fact that it provides enough guidelines with regard to the organizational dimensions, however it is not normative that makes it in turn flexible. As the EFQM model has a pre-determined set of criteria underlying its formulation, this has somehow limited up to a detain extent the need for determining which part of the TQM philosophy should be adopted by the organization3. Another model can be defined in connection with the ISO standards 9000:2000 that also deal with the systems of quality management. These standards are not substantially different from the models 2 Similar awards exist in the US (Malcolm Baldridge National Quality Award of 1987) as well as in Japan (Deming Award as of 1950s). In Romania the Foundation for the Romanian Award Of Quality J. M. Juran has been created in 1999; the first award has been granted in 2001 but few people know about it and its meaning. Under the Romanian award there is a section for public institutions. 3 Dotun Adebanjo, TQM and business excellence: Is there really a conflict? In Measuring Business Excellence, nr. 3, volume 5, p.39; 89 presented so far. There are 8 principles underlying the quality management: 1. Customer focus 2. Leadership 3. Involvement of people 4. Process approach 5. System approach to management 6. Continual improvement 7. Factual approach to decision making 8. Mutually beneficial supplier relationships This standard is extremely popular maybe also because an ISO certification may lead to an increase in the business profit. At the end of 2003 there were over 500,000 ISO certifications in 149 countries4. Currently attempts to develop a guideline for the application of the standard 9001:2000 for the field of public administration5 are made. Common Assessment Framework (hereafter CAF) Based on the EFQM model, at the initiative of the European Union, a Common Assessment Framework for the public administration has been developed. A similar concept can be found in the Romanian legislation. An initial version of this model has been presented for the first time at Lisbon, Spain, during the first European Conference on Quality within the Public Administration of the Member States. A revised version was officially lunched in 2002. CAF draws upon the logic and structure of EFQM, trying to provide in the same time the necessary adjustments of this model to the public sector. The most important adjustments include: the distinction between a citizen and a client, focus on the management of change, and the impact on the society. In addition, there are specific examples for each sub-criterion that should guide evaluation. The following graphical structure replicates the logic underlying the model: CAF Enablers Results Innovation and Learning 1. Leadership 5. process and change management 3. Human resources management 2. Strategy and planning 4. Partnerships and resources 7. People results 6. Customer/citizen- oriented results 8. Society results 9. key performance results 4 The ISO Survey of ISO 9001:2000 and ISO 14001 Certificates, http://www.iso.org/iso/en/iso9000-14000/pdf/ survey2003.pdf 90 Enablers within an organization determine the way in which the organization reaches its goals; for each goal there are sub-criteria with regard to a practice meant to lead to excellence. Evaluation is made based on PDCA (Plan-Do-Check-Act) or Deming’s wheel, a tool borrowed from the TQM model. The aforementioned sub-criteria are listed in the table below: Table 1: Table for the evaluation of enablers Does not know how to answer, does not understand the question, or this question does not apply to the organization 0 There is no action or significant project that would contribute to the achievement of this goal 1 An action plan P has been defined Plan 2 An action plan has been defined and implemented; PD Do 3 An action plan has been defined and implemented; its implementation is monitored; PDC Check 4 An action plan has been defined and implemented; its implementation is monitored; also corrective actions has been undertaken where necessary; PDCA Act 5 Plan definition, implementation, monitoring, and the adaptation of its realization are all part of a continuous cycle (process of continuous improvement). P A D C For the results of an organization there are also 3 sub-criteria in place. They are listed in the table below: Table 2: Table for the evaluation of results Does not know how to answer, does not understand the question, or this question does not apply to the organization 0 No result is measured. 1 Key results are measured and show negative or stable trends. 2 Results show modest progress. 3 Results show substantial progress. 4 Excellent results are achieved and positive comparisons to own targets are made. 5 Excellent results are achieved, positive comparisons to own targets are made and positive benchmarks against relevant organisations are made. In the end we have the following structure: 1. Leadership; what is its role in relationship to the following tasks: 1.1 Give a direction to the organisation: develop and communicate vision, mission and values; 1.2 Development and implementation a system for managing the organization; 1.3 Motivate and support the personnel, and act as a role model; 1.4 Manage the relations with politicians and other stakeholders; 2. Strategy and planning; what does the organization do to: 1.1 Gather information relating to present and future needs of stakeholders; 1.2 Develop, review, and update strategy and planning; 1.3 Implement strategy and planning in the entire organization; 3. Human resources management; what does the organization do to: 3.1 Plan, manage, and improve human resource with regard to strategy and planning; 3.2 Identify, develop, and use competencies of the employees aligning individual, team and organizational targets and goals; 3.3 Involve employees by developing dialogue and empowerment; 91 4. Partnerships and resources; what measures have been taken in order to be sure that the organization: 4.1 Develops and implements key partnership relationships; 4.2 Develops and implements partnerships with the clients/citizens; 4.3 Manages knowledge; 4.4 Manages finances; 4.5 Manages technology; 4.6 Manages buildings and assets; 5. Process and change management; to take into consideration how the organization: 5.1 Identifies, creates, manages and improves processes; 5.2 Develops and delivers products and services and products by involving the customers/ citizens; 5.3 Plans and manages modernization and innovation; 6. Customer/citizen-orientated results; to what extent, the organization succeeds to satisfy customers/ citizens’ needs and expectations by: 6.1 Results of the customer/citizen satisfaction measurements; 6.2 Indicators of customer/citizen oriented measurements; 7. People results: 7.1 Results of satisfaction and motivation measurements; 7.2 Indicators of people results; 8. Society results: 8.1 Results of societal performance; 8.2 Results of the environmental performance; 9. Key performance results: 9.1 Goal achievement; 9.2 Financial performance. CAF is implemented through a process similar with some versions of Delphi and it has the following stages: 1. At the institutional level, the consensus regarding the implementation of CAF is established; 2. A person is assigned to manage the CAF implementation; 3. A certain number of members are selected from the organization (most commonly between 10 and 15); 4. The members of the group should be aware of the goal of the assessment and about the CAF method; 5. Each assessor evaluate each criterion; 6. The assessors try to get a consensus regarding the evaluation of each criterion; 7. New methods to improve the performance of the institution are identified; 8. Improvement strategies are selected; 9. An action plan is developed and implemented; 10. The results are monitored and the CAF process is restarted. In order to evaluate the progresses, the process will be repeated at certain time intervals (2-3 years). TQM needs continual improvement. In general, the implementation of CAF is considered to be a step ahead. Some specialists think that CAF is a first step in the transition to EFQM, ISO or other quality assessment methods. 92 CAF in Romania In Romania, the first attempts to implement CAF occurred in the public administration reform. In the context of European Union criticism regarding the weak administrative capacity, using this instrument for evaluation and diagnosis was considered necessary. In this sense, the institutional twinning project between the Romanian Ministry of Administration and Interior, French Ministry of Public Service, Reform of the State and Spatial Planning and Presidency of Council of Ministers from Italy. Until now, the translation of the documents was completed and a pilot project was implemented (but its results are not available yet). The problems have started with the translation. The Romanian translation is based on the French version of CAF. Many problems, that could have been avoided, arise from it. The Romanian translation uses the word administration instead of management, idiosyncrasies that is specific to french language, but not to Romanians. Another interesting option would be to change the expression clients/citizens with the word customers. An even more serious problem regards the translation of the word leadership by the expression nivele de conducere (an Romanian expression that has means management), which comes from French word encadrement. But in the same time, CAF ask for a correct and complete understanding of leadership notion and not to limit its understanding to management6. The TQM philosophy implies that in order to achieve quality all the hierarchical levels are involved and the initiative can occur at any of these levels. Also, at criterion 3, Human Resources Management, sub-criterion 3.3 regarding the empowerment of the employees, it is translated by responsibility and autonomy. This tendency is even more obvious, in the way the first example at this sub-criterion is translated. Instead of “promoting an open, non-hierarchical culture of communication and dialogue” the translation is “to promote a climate of openness, communication and dialogue and establishment of hierarchical non-authoritarian relations”. It is true that the EU member states agreed that each country should adapt CAF to the national context7, but this does not imply that the base concepts of CAF and TQM should be misinterpreted. Calin Popescu Tariceanu government will continue the reform of public administration drafted during the governance of Social Democrat Party, because CAF is included in the strategy of the new government. In the implementation calendar of the measures and actions for achieving the provisions of the Governmental Program 2005-2008 from April 20058, it is stated that CAF will be extended at the level of pubic authorities and services by the end of 2007 (action 1.13.2, p.197). The Central Unit for Public Administration Reform (CUPAR) is responsible for achieving this goal. Training sessions regarding the total quality and CAF will be organized. This activity is the major component of action 1.13, “Establishment of the performances evaluation and monitoring system”, which has as deadline the first semester of 2006, under the control of the Department for Deconcentrated Public Services, within the Ministry of Public Administration and Interior. CAF is included also in another part of the calendar for the implementation of the Governmental Program, being referred by the goal Institutional Capacity, action 3.11, further implementation of CAF. Deadline for this action is December 2005. 5 http://www.iso.org/iso/en/commcentre/pressreleases/2005/Ref951.html; 6 Christian Engel, Common Assessment Framework, The state of affairs, in Eipascope nr. 1/2002, p.36 7 European Institute for Public Administration, Study of the Use of the Common Assessment Framework (CAF) in the European Public Administration, Synthesis Paper Prepared for the 41th Meeting of the Directors_General of Public Administration (Rome, December 2003), p.8 93 The conclusions of a pilot study In May 2005, we conducted a pilot study within Faculty of Political and Administrative Science, Babes-Bolyai University. The goal of this study was to test the reaction of the civil servants when they act as CAF evaluators. We selected 9 high rank and executive civil servants, who were in the same time enrolled in the Master Program organized by the faculty in collaboration with National Institute for Administration. First, we made sure that they know the goal and the CAF method. Then we asked them to evaluate their own institution and to explain for each criterion why they gave a certain grade and also the difficulties that they encountered. In the end, we organized a focus group in order to evaluate the chances to implement CAF in Romania. The results that we obtained are the following: Criterion Grade 1 - Leadership 2.64 2 – Strategy and planning 2.47 3 – Human resources management 2.69 4 – Partnerships and resources 3.26 5 – Process and change management 3.11 6 – customer/citizen-orientated results 3.18 7 – People results 3.24 8 – Society results 3.35 9 –Key performance results 3.83 Mean 3.04 The average result that we obtained is very good, 3.04 comparatively with 2.42 that the European Institute for Public Administration obtained in 125 institutions from 14 countries of Western Europe. We have to mention that the higher score we obtained regards the key performances, and lower score regards strategies and planning. Can we trust these results? Is the Romanian public administration doing so well? A more detailed analysis of the answer sheets and of the answers from the focus group shows that we should not completely trust the results. First and maybe the most important conclusion is that the tables for evaluation of enablers/results weren’t taken into consideration. This is explained by the fact that very few institutions have actions plans, not to mention other stages of the Deming’s wheel, and the performance evaluation systems almost do not exist. In the absence of this information at the institutional level, the civil servants based their evaluations on more subjective criteria that would make the results to sound in a more desirable form. In many cases, the respondents did not answer to all the addressed questions, which confirmed the supposition of lack of interest in giving meaningful information to the researcher. The ignorance of the responsibilities of the other departments within the institution that they work for represented another major difficulty encounter by the respondents in answering to all the questions. Another problem regards deconcentrated institutions from state level at the county level, which are limiting their activity in implementing the directions received from governmental level. These institutions are less interested in adapting these general directions to the specific characteristics of a county. In this context, the deconcentrated institutions do not perform such task. Another observation refers to the fact that as the participants to the research have a higher position within the institution they work for, they tend to have a more favorable perspective of the institution that they belong to and seeing the opportunity to participate to the research as an opportunity to praise 94 their institution. The executive civil servants tend to be more critical and objective in evaluating the institution they work for. Conclusions CAF can be a very useful tool in evaluating the institutional capacity (which is considered by the reports of the European Union to be still an important problem for Romanian public administration) and for improving it. If we forget the context, there are several problems that can occur. Common Assessment Framework can be applied only in an environment characterized by high level of consensus regarding the implementation of TQM. As long as the public administration environment is not open to the ideas and the philosophies of the total quality, CAF will be regarded as an unpleasant and formal procedure and people will use subjective criteria in evaluating the institution. There are major barriers in the Romanian public administration that make the implementation of this method very difficult. Firstly, there are no action plans to guide the activity of public institutions, and in consequences there are no implementation, evaluation and improvement efforts (Deming’s wheel – PDCA). Secondly, the Romanian public institutions do not have a culture orientated toward evaluation. Public institutions do not conduct evaluations of their programs, public policy analysis, impact studies, customers/employees’ satisfaction measurement, and they lack performance measurement systems. Public institutions do not acknowledge these weaknesses and there are no strategies to strengthen them. Analyzing the strategy for the reform of public administration, we understand that it is aimed to improve the formulation of the public policies, and to develop the competence of the public institutions to implement CAF. But in the same time we can observe that it lacks some important elements. The strategy does not elaborate on evaluation - a key element.9. According to Lascelles and Dale (D.M. Lascelles, B.G. Dale, The Road to Quality, IFS, 1993), the Romanian public institutions can be characterized as organizations that do not support the idea of quality (first stage out of seven identified by these two authors). These institutions are characterized by a strong emphasis on legal requirements that specifies exactly when the quality evaluation initiatives should take place and by a vague understanding of the basic concepts regarding TQM. In this moment, the major focus is to get the support of the public institutions for the quality assessment procedures, at least regarding the management. Otherwise, CAF or any other instrument (ISO, EFQM) will be regarded as another formal procedure imposed from outside, and which will have a low support from the institution. The process toward a management of quality widely used takes time (Lascelles and Dale approximates that it takes approximately 10 years). In order to achieve this goal, the following requirements are needed: involvement of the institution and of all its customers, strict performance measurement systems that should be improved continuously and a change in the way quality is understood from a formal understanding to quality as a way of life. 8 http://www.guv.ro/obiective/pg2005-2008/calendar-actiune-pg-guv.pdf 9 The scheme proposed by the government to analyze the formulation of public policies (http://www.mai.gov. ro/index18.htm) has as the final objective the implementation of public policies, and ignores the last stages of the process regarding the evaluation and assessment. 95 References: 1. Adebanjo, Dotun, TQM and business excellence: Is there really a conflict, în Measuring Bussiness Excellence, nr. 3 vol 5, p. 37-40 2. Bovaird, Tony, Elke Loffler, Moving from excellence models of local service delivery to benchmarking `good local governance’, International Review of Administrative Sciences, nr. 1 (68)/2002, pp. 9-24 3. Dale, B.G., M. Zairi, A. Van der Wiele, A.R.T. Williams, Quality is dead in Europe - long live excellence - true orfalse, Quality Focus, Vol.4, nr. 3, pp. 4-7 4. Drăgulănescu, Nicolae, Magdalena Drăgulănescu, Managementul calităţii serviciilor, AGIR, 2003 5. Engel, Christian, Common Assessment Framework: The state of affairs, Eipascope nr. 1/2002, pp. 35-39 6. European Foundation for Quality Management, Introducing EFQM, V2.1/En, 2003 7. European Foundation for Quality Management, The Fundamental Concepts of Excellence, V2/En, 2003 8. European Institute for Public Administration, Study on the Use of the Common Assessment Framework (CAF) in European Public Administrations, Synthesis Paper Prepared for the 41th Meeting of the Directors-General of Public Administration (Rome, December 2003) 9. Guvernul României, Calendarul de aplicare a măsurilor şi acţiunilor pentru îndeplinirea prevederilorProgramului de guvernare 2005-2008 (proiect), aprilie 2005, http://www.guv.ro/obiective/pg2005-2008/caledaractiuni-pg- guv.pdf 10. Lascelles D.M., B.G Dale, The Road to Quality, IFS, 1993 11. Loffler, Elke, Defining and Measuring Quality in Public Administration, Bristol Bussiness School Teaching and Research Review, nr. 5/2001, pp. 1-21 12. Stanciu, Ion, Managementul calităţii totale, Cartea Universitară, 2003