202 Abstract Using a principal-agent framework and multi-country survey data of over 400 public sec- tor organizations, this article examines the effect of result control on the use of financial manage- ment techniques in public sector organizations. In order to avoid invalid conclusions, we test for heteroskedasticity and model residual vari- ance using a heterogeneous choice model. This model yields important insights into the effect of result control that would be overlooked in a mis- specified ordered logit model. Our findings reveal that result control matters, although size and pri- mary task of the organization also prove to be determinants of the use of financial management techniques. Within the context of the continuous attempts being made to improve public sector performance, policy makers should thus devel- op different strategies for different (individual) agencies, while relying on a strong ex-post result control, when they want to stimulate the use of financial management techniques. Keywords: financial management tech- niques, heterogeneous choice model, result control. USING FINANCIAL MANAGEMENT TECHNIQUES WITHIN PUBLIC SECTOR ORGANIZATIONS, DOES RESULT CONTROL MATTER? A HETEROGENEOUS CHOICE APPROACH Jan WYNEN Jan WYNEN Dr., KU Leuven Public Governance Institute, Leuven, Belgium E-mail: jan.wynen@soc.kuleuven.be Tel.: 0032-16-320.143 Transylvanian Review of Administrative Sciences, Special Issue/2014, pp. 202-222 203 1. Introduction A well performing public sector is considered to be a prerequisite for the economic and democratic performance of countries. Recent public sector reforms have been at- tempted in individual countries to achieve a better performing public sector. As such, the organization of the public sector in OECD countries has been subject to some ma- jor trends and shifts during the last 20 years (Pollitt and Bouckaert, 2004, p. 95; Hesse and Toonen, 1997, p. 26; Halligan, 2002, pp. 20-50). New Public Management (NPM) is the best known example of these reforms and led to several changes of the public sector in many countries. NPM was a reaction to correct the irretrievable failures and moral bankruptcy of the ‘old’ public management (Hood, 1991; Keating, 1989). More in particular, NPM formed an answer to the lack of result and customer orientedness of public organizations that delivered services to the public or implemented policy (Verhoest et al., 2007). According to NPM doctrines public sector performance can be improved by im- porting concepts, tools and values from the private sector. Public sector organizations should be offered more managerial autonomy while being controlled by the govern- ment on the basis of results. By doing so public sector organizations are believed to be more likely to apply private-sector styles of management techniques, to be more customer-oriented and will be more likely accountable for results, leading to a higher efficiency and a better performing public sector organization. Therefore, one of the central arguments of NPM is that more managerial autonomy may enhance the use of private-sector style management techniques by public sec- tor organizations only under the condition of result control. This argument is based on principal-agent theory and asserts that because of goal incongruence (or conflict) and information asymmetry between the agency and its political principals, there is a considerable risk of opportunistic behavior by the agency (Pratt and Zeckhauser, 1991). Opportunism can take the form of adverse selection, moral hazard, and, in pol- icy settings, can lead to a subversive or deviant policy implementation by the agent (Waterman and Meier, 1998). Following this line of reasoning, public managers of autonomous public sector organizations will have no incentives to use management techniques within their own organizations. On the contrary, management techniques may enhance the information for the political principal about potential shirking or deviant behavior of the agency. The concept of management techniques is however very heterogeneous, and cap- tures a large variety of techniques. Sets of management techniques are frequently referred to as management systems or management capacity. Based upon literature (Pollitt, 1995; Ingraham et al., 2003; Flynn, 2002; Verhoest et al., 2010, pp. 50-73) four management subsystems can be distinguished: financial management, performance management, human resource management, and quality management. For each of these subsystems a number of techniques are proclaimed as good practice by NPM literature (Naschold, 1989; Pollitt, 1995). In this article we focus on financial manage- ment. Whereby, following Verhoest et al. (2010, p. 62), internal result-based allocation 204 of resources to organizational units and the development of a cost-calculation system are considered to be representative for the use of financial management techniques within public sector organizations. Using multi-country data this article studies to what extent result control affects the use of financial management techniques. However, the paper also has a method- ological goal, more precisely to make public sector researchers aware of the under- lying assumptions of ordered models. Indeed, most studies rely on Likert-type data, making ordered and multinomial models increasingly popular. This paper will show that it is useful to subject a regression analysis to some additional scrutiny because a misspecified model may lead to erroneous inferences. The remainder of this paper is organized as follows: a theoretical model is devel- oped in section 2 while in section 3 the data is described and descriptive statistics are provided. The methodological approach, discussion of the model and main findings are discussed in section 4, which is followed by some concluding remarks. 2. Using financial management techniques: a principal-agent approach The relationship between the public sector organization and the oversight gov- ernment is a typical example of a principal-agent relationship, whereby the public sector organization acts as the agent of their political and administrative principals. As such the agent, or in this case the public sector organization, enjoys some degree of managerial autonomy. This kind of autonomy can be defined as the level of decision making competencies (discretion) an organization has vis-à-vis superior levels, bodies and actors (Verhoest et al., 2004). Granting a public sector organization more mana- gerial autonomy involves shifting decision making competencies from external actors (i.e. parent ministries, ministers) to the organization itself by delegation or devolution. Principal-agent theory emphasizes that the agent may use his autonomy to behave opportunistically (Jensen and Meckling, 1976). Therefore public sector organizations are also subject to various degrees of result control. Result control is referred to in this article as the extent to which the CEO is accountable for results and whether or not this accountability is linked to sanctions or rewards. Whereas managerial auton- omy refers to the extent that public sector organizations have discretion on the choice and use of the inputs (resources) they use, result control refers to the mechanisms used by the political and administrative principals to control which outputs/results the public sector organization produces. According to Bouckaert (1998) result control is believed to temper opportunistic behavior by the public agency and to enhance its performance in several ways. First, the information asymmetry between the govern- ment and the public agency as to the performance of the latter is lessened by the use of information revealing instruments. Second, the goals of both parties are aligned more closely because clear objectives and targets are set and negotiated. Third, result con- trol instruments like performance contracts may set priorities among the objectives of the different involved ministers, reducing the ‘multiple principals’ problem’ (Bouc- kaert, 1998). Hence, the NPM-doctrine, which propagates the delegation of task to autonomous agencies and managerial autonomy, stresses the need for a strong result 205 control by the ministers and parent departments of such agencies. Agencies should thus be given clear objectives by ministers and their achievement should be moni- tored, evaluated and sanctioned in case of mal-performance (Bevan and Hood, 2004; Verhoest et al., 2012). Public agencies thus have to accept a rigid result control system, which includes performance indicators and performance monitoring and assessment in order to ensure that the agency uses its discretion to pursue the achievement of the objectives of its principal (Christensen and Lægreid, 2004, p. 102). Following Verhoest et al. (2010, pp. 50-55) we expect that agencies which are controlled to a large extent ex post for their results may be forced, or induced, to use management techniques when this is assumed to be beneficial for their overall performance. The use of financial management techniques can be regarded as some kind of monitoring, bonding. Since financial management techniques might thus be helpful for the oversight authorities to control the organization’s activities, we believe these techniques to be used to a larger extent in organizations with a high level of result control compared to organiza- tions with lower levels of result control. Although scarcely studied, the independent influence of external result control of agencies is consequently considered to have a positive influence on the use of financial management techniques within agencies. An element in the principal-agent relation, as described above, is that the agent receives sufficient autonomy in order to be able to implement the principals’ demands in an efficient, flexible and specialized way. Having autonomy or decision making competences in managerial affairs may thus be a facilitator, enabler or inducement of using management techniques (Lægreid et al., 2008; Christensen and Lægreid, 2007). By applying modern management techniques, agencies hope to increase overall per- formance in response to the demands of result control. This argument is also central to NPM doctrines, claiming that both managerial autonomy and result control are needed in order to stimulate the use of innovative management techniques. Manage- rial autonomy combined with result control provides public managers with both the possibility and the incentive to introduce these kinds of management practices. We thus expect that not only result control has a positive influence on the use of financial management systems, but that the interaction of management autonomy and result control has an equally positive effect on the use of financial management techniques. This relationship is visually presented in Figure 1. 3. Data source, variables and descriptive statistics We focus on a specific type of public sector organizations, which we refer to as ‘public sector agency’. Following Pollitt and Bouckaert (2004) and Verhoest et al. (2010) we focus on public agencies with the following features: (1) they are public law bodies, (2) they are structurally disaggregated from other organizations and core min- istries, (3) they have some capacity for autonomous decision-making with regard to management or policy, (4) they are formally under some control of ministers and min- istries, (5) they have some expectation of continuity over time, and (6) they have some resources on their own (either financial or personnel). Companies and corporations with a commercial focus which have to closely observe the laws regulating private 206 Internal use of financial management techniques ‘ManagerialAutonomy’ granted to the organization by ministerand parent department ‘External ResultControl’ of organization by minister and parent department Organizationallevel Level of political and administrative principals Figure 1: Visualization of the relationship between result control, managerial autonomy and the use of financial management techniques companies or which are registered under company law as company and governmen- tal foundations, trusts and charities are excluded from our understanding of agencies. Data used for the analysis have been provided by the COBRA-network (‘Compara- tive Public Organization Data Base for Research and Analysis’). The COBRA network aims to encourage and enable comparative research of public sector organizations1. It developed a common questionnaire in order to survey senior managers of public sec- tor organizations in particular, (semi)-autonomous agencies located directly beneath ministries and ministers. In each state the top management (Chief Executive Officers (CEOs)) of the state agencies was asked to fill in a web-based questionnaire contain- ing several types of questions (i.e. perceptions of autonomy and control, innovative activity, management and organizational culture). The joint dataset comprises unique agency-level survey data spread across 15 different countries. Yet, the goal of this ar- ticle is not to conduct a cross-country comparison of the effect of result control on the use of cost calculation systems but to examine this relation independently from coun- try characteristics. Consequently the selection of countries is not vested in theory but is based on maximizing the amount of data while maintaining a representative sam- ple. For this paper we will use data on agencies from 6 countries: Norway, Belgium, the Netherlands, Romania, Germany and Austria. The organizations that responded proved to be representative of the total populations in each state, with a broad distri- 1 For more information see: htt p://soc.kuleuven.be/io/cost/index.html. 207 bution across type of agency, primary tasks, ministries and policy fields. Out of the initial 669 observations, 238 organizations have missing data on the outcome, explan- atory, and/or control variables, leaving us with a sample size of 431 state agencies. 3.1. Measuring the use of financial management techniques The ordered variable ‘cost calculation systems’ refers to the use of financial man- agement techniques and is the main dependent variable in our different models to be estimated. We investigate the development of a cost-calculation system based on direct survey evidence. In particular, agencies were asked the following question: ‘To what extent is there in your organization a development of cost-calculation systems?’ Agencies were given the following three options: no/to a small extent, to some extent or to a large extent. The dependent variable is set to 0 if the agency indicated that it does not happen, to 1 if it only happens to some extent and to 2 if it happens to a large extent. We chose this management technique since we consider it to be a good example of financial management (see also Verhoest, 2010). Furthermore, since several manage- ment tools are included in the dataset, an explanatory factor analysis (using a poly- choric correlation matrix, in order to account for the categorical nature of the depen- dent) has also been carried out in order to verify the fact that ‘cost calculation systems’ load on the same factor as another financial management technique namely the inter- nal allocation of resources to organizational units on the basis of performance. Since it does, we can consider them closely related. Therefore, we consider the development of a cost calculation system to be a good and representative measure for the use of financial management techniques. We chose not to use an index of both management techniques in order to maximize the number of observations. Our dependent is thus based on a single-item measure, which can be perceived as a weakness of our data. Yet this is often not less reliable than multiple response items (e.g. Bergkvist and Rossiter, 2007; Gardner et al., 1998; Wanous and Hudy, 2001). Gard- ner et al. (1998) even point out that single item measures avoid the risk of aggregating multiple measures whose inter-item correlation is due to common method variance. Furthermore, we rely on self-reported indicators. The dependent is the perception of one person at a particular level of the organization (the CEO) within the agency; this also brings limitations since it can lead to an upward response bias due to social de- sirability bias, rationalizing myth (Meyer and Rowan, 1977) and hypocrisy (Brunsson, 1989). The motivation behind using the perception of the CEO within the agency is however the following: the perception of senior managers about the actual autonomy and control of their agency will heavily influence their actions and the way in which they manage their agency. In a previous and comprehensive case study by Verhoest (2002), it becomes clear that the de facto autonomy and control perceived by the CEOs may differ substantially from the formal autonomy and control of agencies (as set out in legislation and regulations). Other studies came to similar conclusions (Verhoest et al., 2004), or have shown that the behavior of senior management was guided by these perceptions (Verhoest et al., 2004). Consequently, perceptions of autonomy and 208 control give us insights about the room for maneuver that senior managers think they have, and hence about the actual functioning of agencies (Verhoest et al., 2010). In the following subsections we will first discuss the operationalization of result control and management autonomy after which the control variables are discussed in detail. 3.2. Operationalization of result control and management autonomy Result control is based on (i) the accountability of the agency CEO for agency per- formance (results) to the government and (ii) the extent to which the organization faces sanctions or rewards for its performance. When accountability is linked to sanc- tions or rewards, result control is high and the variable is set to one. If CEO account- ability is not linked to sanctions or rewards we consider result control to be low and the variable is set to zero. A high level of result control in this case equals a ‘hard’ form of performance contracting, in which under- or over-performance leads to not only the accountability of the agency CEO, but also to sanctions or rewards (see Verhoest et al., 2010, pp. 60-79). A score ‘0’ refers to no result control or ‘soft’ result control (mean- ing CEO is accountable for the results, but without sanctions or rewards being given). Following principal-agent theory, we take two types of organizational autonomy concerning managerial decisions into account; personnel management autonomy (Personnel Management Autonomy) and financial management autonomy (Financial Management Autonomy). Personnel management autonomy relates to the autonomy of an agency to make decisions concerning salary level, promotion, and evaluation of staff, in general (beyond individual decisions) without interference from ministries. Financial management focuses on three dimensions; the extent to which the organi- zation itself and without interference from superior bodies (ministries, ministers) is able to shift personnel and running cost budgets, to shift personnel-running cost and investment budgets, and set tariffs for services and products. In line with Wynen et al. (2013), a dummy score is calculated for each form of management autonomy based on the aggregation of the three items; whereby score 1 indicates full autonomy on all three items. 3.3. Control Variables Furthermore, we control for some other factors which are based on previous stud- ies (e.g. Verhoest et al., 2010; Wynen et al., 2013) believed to influence the use of innova- tive management techniques such as financial management techniques. By controlling for these factors we want to reduce the possibility that the found influences of result control on the use of financial management techniques are in fact due to the influence of other variables which are not in the model. However, a comprehensive discussion of all possible determinants of the use of financial management techniques lies be- yond the scope of this paper. Instead, we will focus on variables that are available in our dataset and which will be incorporated in the subsequent empirical analysis. • Measurability of primary organizational task: Task characteristics and the related tech- nical environment affect organizational practice, and hence the use of financial management techniques. These tasks may vary according to their measurability 209 (Wilson, 1989, p. 83), their policy/political environment (Bourdeaux and Chikoto, 2008; Dull, 2009) as well as their salience (Pollitt and Bouckaert, 2004, pp. 80-105). Financial management techniques can be expected to be used more in organizations with more measurable tasks and tasks which face pressures stemming from market forces or political salience. In the field of agency studies, a study with agencies from Belgium, Norway and Ireland found a positive effect of measurability of primary organizational task on the use of internal result-based allocation of resources to or- ganizational units; however, no significant effect was found for the development of cost calculation systems (Verhoest et al., 2010, pp. 105-120). Nonetheless, we include a dummy (Services) in order to examine the effects of primary organizational task. This dummy is set to one when the primary task exists of services (general public services and business and industrial services). • Type of agency: Also, the legal distance from government and the extent of struc- tural disaggregation from the parent ministry has consequences for institutional norms and organizational cultures. We expect a stronger inclination towards using financial management techniques in organizations further away from government, especially when they have their own legal identity, vested in public law or private law (Hammerschmid and Geissler, 2010; Lægreid et al., 2006). First, agencies which are structurally disaggregated from government and which have their own legal identity are more susceptible for demands of their customers and stakeholders and more visible for media and society, increasing the need for legitimacy. Such organi- zations need to secure their existence by building strong linkages with and support from these external actors. Hence, pressures to adopt and use modern management techniques which helps them to deliver services of a high quality and in an efficient way, are comparatively strong. Units directly under ministerial responsibilities are more politicized and less in direct contact with citizens, which is typically seen as hampering managerial instruments (Bogumil and Ebinger, 2008; Bouckaert and van Dooren, 2003, pp. 20-30; Bach and Jann, 2010; Hammerschmid and Geissler, 2010). Likewise, organizations further away from government are more likely to develop organizational cultures conducive to the use of financial management tech- niques. Therefore, a dummy that is ‘Public or private law based’ is included in the analyses. It is coded one if the agency is a public or private law based organization and is set to zero otherwise. • Organizational age and size: Based on sociological neo-institutionalism and focusing on path-dependencies and culture at organizational level, the history and culture of an organization can shape the extent to which it will use new management tech- niques (see cultural-institutional perspective by Christensen and Lægreid, 2007; Selznick, 1957). Although the survey used for data collection in this research did not measure organizational culture and institutional norms directly, we have data on aspects which can serve as proxies. In this respect, the age and size of the orga- nization are important, as older and smaller agencies will have a more enshrined culture, values and norms, which make it harder to introduce and effectively use 210 instruments which are at odds with the present, strongly institutionalized culture. Younger agencies will be more inclined to use financial management techniques, also because these younger agencies are still in the process of building a strong legitimacy and relations with their stakeholders, and have a more developmental culture (Quinn and Rohrbaugh, 1981). Furthermore, it may be speculated that larg- er organizations have higher internal information asymmetry. Larger organizations have a higher likelihood of having more hierarchical levels within the organization (e.g. Kimberly, 1976; Child, 1973; Blau, 1970), sequentially leading to possible prob- lems of information asymmetry and goal incongruence (Eisenhardt, 1989). In order to overcome these issues, more structured management techniques will be more likely applied in large organizations. Moreover, larger organizations will also have more capacity to implement management techniques than smaller organizations (e.g. Lægreid et al., 2007). Based on the above we expect that both size and age will have an effect on the use of financial management techniques. Size in terms of num- ber of staff, measured in FTE (Size (FTE)), as well as agencies’ age (Age), measured in years since founding (survey year minus year of set-up), are therefore included. Since the distributions of size and age are highly skewed, we use the square root in our models. 3.4. Descriptive statistics This subsection is devoted to a descriptive analysis aimed at investigating: (1) the representativeness of the subsample used in the regression analysis; and (2) possible occurrence of collinearity problems. Table 1 shows summary statistics for the main variables. The purpose of this table is to test whether the estimation subsample used in the regressions continues to be representative of the entire sample or is instead biased in one or more variables, because of an unbalanced distribution of missing val- ues. Consequently, we compare the averages and standard deviations of the variables introduced in previous sections. Overall, the values reported in Table 1 in the Original Sample and Used Sample columns are very similar. This suggests that missing values were randomly distribut- ed, and that the observations used to estimate the regressions constitute a representa- tive subsample of all the agencies that were originally included in the survey. Table 1 however indicates that public or private law based agencies are slightly underrepre- sented in our sample, yet the decrease in these types of agencies appears to be distrib- uted evenly over all countries, thereby suggesting that there is no geographical bias in the distribution of missing values. When examining variables, we notice that agencies in our sample employ on av- erage, in number of FTE, 662 employees. The median corresponds to 118 employees, indicating that organizational size ranges from very small (0 FTE) to very large (6,781 FTE). Agencies in our sample exist on average about 29.57 years. Again the median is lower: 14 years. When it comes to type, only 43.2% appear to be public or private law based. From the statistics with regard to the organizational task it becomes clear that 211 Ta bl e 1: D es cr ip tiv e st at ist ics O rig in al s am pl e Us ed s am pl e (N =4 31 ) To ta l s am pl e Hi gh re su lt co nt ro l= 0 (N =2 88 ) Hi gh re su lt co nt ro l= 1 (N =1 43 ) Di ff. Va ria bl e m ea n sd p5 0 N m ea n sd p5 0 m ea n sd m ea n sd Us e of c os t c al cu la tio n sy st em s 1, 84 2 0, 72 9 2 54 4 1, 85 8 0, 70 3 2 1, 78 1 0, 70 6 2, 01 4 0, 67 1 -0 ,2 33 ** * Hi gh p er so nn el m an ag em en t a ut on om y 0, 54 0, 49 9 1 57 2 0, 49 9 0, 50 1 0 0, 51 0, 50 1 0, 47 6 0, 50 1 0, 03 4 Hi gh fi na nc ia l m an ag em en t a ut on om y 0, 19 3 0, 39 5 0 55 3 0, 17 6 0, 38 2 0 0, 18 1 0, 38 5 0, 16 8 0, 37 5 0, 01 3 Hi gh re su lt co nt ro l 0, 31 7 0, 46 6 0 53 6 0, 33 2 0, 47 1 0 0 0 1 0 Pu bl ic or p riv at e la w ba se d 0, 56 4 0, 49 6 1 66 9 0, 43 2 0, 49 6 0 0, 41 0, 49 3 0, 47 6 0, 50 1 -0 ,0 66 Se rv ice s 0, 55 6 0, 49 7 1 56 1 0, 55 9 0, 49 7 1 0, 54 9 0, 49 8 0, 58 0, 49 5 -0 ,0 31 Si ze (F TE ) 17 ,1 6 23 ,3 5 10 ,5 4 55 9 17 19 ,3 4 10 ,8 6 16 ,4 7 20 ,6 1 18 ,0 5 16 ,5 1 -1 ,5 8* * Ag e 4, 61 2, 79 1 3, 60 6 62 6 4, 65 5 2, 81 5 3, 74 2 4, 91 6 3, 00 7 4, 13 2, 30 2 0, 78 6* ** Be lg iu m 0, 18 5 0, 38 9 0 66 9 0, 23 2 0, 42 3 0 0, 20 1 0, 40 2 0, 29 4 0, 45 7 -0 ,0 93 ** G er m an y 0, 10 9 0, 31 2 0 66 9 0, 14 2 0, 34 9 0 0, 18 8 0, 39 1 0, 04 9 0, 21 7 0, 13 9* ** Ro m an ia 0, 06 88 0, 25 3 0 66 9 0, 07 66 0, 26 6 0 0, 04 51 0, 20 8 0, 14 0, 34 8 -0 ,0 95 ** * Ne th er la nd s 0, 30 8 0, 46 2 0 66 9 0, 12 5 0, 33 1 0 0, 14 2 0, 35 0, 09 09 0, 28 8 0, 05 11 Au st ria 0, 09 72 0, 29 6 0 66 9 0, 10 7 0, 30 9 0 0, 11 8 0, 32 3 0, 08 39 0, 27 8 0, 03 41 No rw ay 0, 23 2 0, 42 2 0 66 9 0, 31 8 0, 46 6 0 0, 30 5 0, 46 1 0, 34 3 0, 47 6 -0 ,0 38 Si gn ific an ce le ve ls ar e ca lcu la te d us in g a W ilc ox on M an n- W hi tn ey te st fo r t he v ar ia bl es U se o f c os t c al cu la tio n sy st em s, A ge a nd S ize (F TE ), a ch i-s qu ar e te st w as u se d fo r t he re m ai ni ng v ar ia bl es : * ** p< 0, 01 , * * p <0 ,0 5, * p< 0, 1 212 a majority (56%) of agencies is involved in tangible tasks (services). Furthermore, we notice that most agencies have high personnel management autonomy (50%) while only 18% of them have high financial management autonomy. When looking at our main covariates of interest, we see that 33% of agencies indicate to be subject to high result control. Mean use of cost calculation systems equals 1.858 with a median of 2, meaning that most agencies use cost calculation systems to some extent. More pre- cisely, 141 (33%) agencies report to make no use of cost calculation systems, while 210 (49%) agencies report to use it to some extent and 80 (18%) indicate to make use of them to a high extent. The table also allows us to examine differences between agencies with high re- sult control and agencies without high result control. Differences were tested using a chi-square test, except for age, size (FTE) and the use of cost calculation systems a (non-parametric) Wilcoxon Mann-Withney test was used. Interestingly, when com- paring means, agencies with high result control prove to use cost calculation systems significantly more than agencies without high result control. Furthermore, agencies subject to high result control are significantly larger and younger than agencies with- out high result control. A possible explanation could be that small agencies have a more homogeneous culture and a more distinct identity, and are therefore able to modify signals from above (Verhoest et al., 2010). Also, agency age can be linked to organizational culture. The development of a distinct culture and tradition within an agency takes time; consequently, older agencies will be more able to resist pressure from outside actors. Finally, the table also shows that high result control differs sig- nificantly across countries. Agencies in Belgium or Romania are in general faced with lower levels of result control, while agencies in Germany are more often subject to high result control. The linear correlation analysis among the regressors is reported in Table 2, show- ing that the value of the correlation between two regressors is not >0.26 (between no high financial management autonomy and services as primary task). This suggests that no collinearity exists between the regressors. Table 2: Pairwise correlation matrix Variables (1) (2) (3) (4) (5) (6) (7) High personnel management autonomy (1) 1 High fi nancial management autonomy (2) 0,1715 1 High result control (3) -0,0329 -0,0157 1 Public or private law based (4) 0,0863 0,2483 0,0626 1 Services (5) 0,1008 0,2637 0,0302 0,1887 1 Size (FTE) (6) 0,0591 -0,053 0,0383 -0,0201 0,0018 1 Age (7) 0,1032 -0,0052 -0,1316 -0,1848 0,0353 0,15 1 The dummies for the 6 counties were not reported to save space, The largest correlation coeffi cient for one of these dummies and the variables reported in the table equals 0,36 and corresponds to the correlation between the dummy for Norway and high personnel management autonomy. 213 4. Methods and results Since the dependent variable ‘cost calculation systems’ is measured on an ordinal scale, we make use of an ordered logit model2. We add country dummies in order to take country clustering into account3. Based on the finding that agencies with and without a high result control differ significantly, we model the regression separately for both groups. Table 3 presents the results of the regression for agencies with and without high result control4. Table 3: Results of Logit regression predicting the development of cost-calculation systems for agencies with and without high result control Variables Low result control High result control High personnel management autonomy 0.161 0.583 (0.311) (0.443) High fi nancial management autonomy -0.000505 0.591 (0.340) (0.491) Public or private law based -0.289 0.202 (0.378) (0.580) Services 0.450* 0.859** (0.248) (0.382) Size (FTE) 0.0118* 0.0318*** (0.00632) (0.0111) Age -0.0157 0.0996 (0.0390) (0.0789) Country dummies Included Included Cut1 0.281 0.792 (0.396) (0.587) Cut2 2.539*** 3.899*** (0.427) (0.697) Observations 288 143 pseudo R² 0,0416 0,1413 Log Likelihood -282,034 -122,497 Standard errors in parentheses; ***p<0,01; **p<0,05;*p<0,1 When examining Table 3 we notice that size (FTE) and services (measurability of tasks) prove significant for both groups. Regardless whether there is high or low result control, size proves to have a significant positive effect on the use of financial manage- ment techniques. The larger an agency, the higher the likelihood that the agency will make use of this kind of management technique. The same holds for measurability of 2 Ordered probit models were also estimated and produced very comparable results. Further- more the Akaike Information Criterium (AIC) and Bayesian Information Criterium (BIC) were almost identical. 3 Other methods to deal with country clustering are the clustering of standard errors or the use of multilevel models. Since we only have six countries in our analysis this however proves to be impossible. 4 An equivalent method is to test for interactions between particular predictors and dummy variables representing the groups. 214 tasks, regardless of the degree of result control, the variable services and thus mea- surability of tasks appear to have a significant positive effect on the use of financial management techniques. Furthermore, Table 3 shows that personnel and financial management autonomy do not appear to influence the use of financial management techniques regardless whether the subsample is defined by high result control or not. This indicates that the combination of high result control and personnel or financial management autonomy does not lead to an increased use of cost calculation systems. This finding does not support the traditional NPM doctrine which claims that more managerial autonomy will lead to an increased use of innovative management tech- niques, such as cost calculation systems, under the condition of result control. Table 4: Ordered Logit and Heterogeneous Choice Models for estimating the development of cost-calculation systems Variables Ordered logit (1) Heterogeneous Choice (2) High personnel management autonomy 0.271 0.362 (0.250) (0.232) High fi nancial management autonomy 0.149 0.263 (0.272) (0.267) High result control 0.791*** 0.880*** (0.212) (0.202) Public or private law based -0.186 -0.309 (0.313) (0.305) Services 0.568*** 0.527*** (0.202) (0.193) Size (FTE) 0.0162*** 0.0152*** (0.00563) (0.00496) Age 0.00386 0.0255 (0.0346) (0.0328) Country dummies Included Included Cut1 0.599* 0.844*** (0.338) (0.317) Cut2 3.060*** 3.224*** (0.373) (0.414) Variance High result control -0.421*** (0.142) Belgium 0.499*** (0.175) The Netherlands 0.462** (0.210) Germany -0.387** (0.185) Observations 431 431 pseudo R² 0,0697 0,1014 Log Likelihood -412,38 -398,31 Model χ² 61,75 89,9 Model df 12 16 Standard errors in parentheses; ***p<0,01; **p<0,05;*p<0,1. A Likelihood Ratio test was estimated to test the fi t between the homoskedastic and heteroskedastic model, it showed that the heteroskedastic model has a signifi cantly better fi t (χ² (4) = 28.14***). 215 Column 1 of Table 4 presents the results of the regression for all agencies combined in order to examine the role of result control on the use of cost calculation systems in more detail. Following this table, result control clearly impacts the use of cost calcula- tion systems as do size and measurability of task. Yet before discussing this table in more detail, we have to stress that these results are based on an ordered choice model, making it inconsistent if the statistical assump- tions concerning unobserved terms in the model are incorrect (Smith, 1989). One of these assumptions states that the variance of the disturbances should be constant across observations, in other words they should be homoscedastic (Greene, 2003, pp. 100-150). For instance, unmeasured variables affecting the use of financial manage- ment techniques may be more important for agencies with low result control than for agencies with high result control or vice versa. The impact of this problem is however not limited to our analysis, but is linked to models using maximum likelihood models in general (Greene, 2003, pp. 302-400; Wooldridge, 2010, pp. 230-400)5. As Williams (2010) points out, one rationale for the ordered logit model is that there is an underlying latent variable y*. As agencies cross a threshold on y*, their val- ues on the observed value y change. For example if y* is smaller than 5, y will be equal to 0, but if y* is larger than 5, y will be equal to 1. Thus, Y tells us that y* falls within a particular range, but does not give us the exact value of y* (Williams, 2009; Long and Freese, 2006). The model for the underlying y* can be written as: (1) Where the x’s are the explanatory variables, the α’s are coefficients that give the effect of each x on y*, is a residual term with a logistic or normal distribution, and σ is a parameter that allows the variance to be adjusted upward or downward (Williams, 2010). Following Amemiya (1985), Allisson (1999) and Williams (2009, 2010) we esti- mate parameters called β’s, whereby the α’s and β’s are related in the following way: k=1,.., K (2) Normally, the residual variance is set to be equal to /3 (in logit models) or 1 (in pro- bit models). When residual variances differ across groups, probit and logit estimates 5 Examples of such models are the logit and probit model, the ordered logit, the ordered probit, multinomial models and the tobit model. The problem of heteroskedasticity is not limited to maximum likelihood models, Ordinary Least Square models (OLS) are also aff ect- ed. Yet the eff ect will be diff erent for these kinds of models since only the standard errors will be incorrect (Cameron and Trivedi, 2005). Contrary to maximum likelihood models, the problem of heteroskedasticity in OLS models can also be more easily corrected, namely by using robust standard errors. Furthermore group diff erences are only one source of het- eroskedasticity, as Williams (2009) points out heteroskedasticity can also be a concern with continuous variables, where it may be unreasonable to assume that errors are the same in magnitude no matt er how large the value of the independent variable is. For a more detailed description of the issue of heteroskedasticity we refer to Williams (2010). 216 however become unreliable. For example; two groups can have identical values on the α’s, but if their residual variance differs, their β’s will differ6. In such case, the model is suffering from heteroskedasticity and as Hoetker (2004) notes ‘even small differences in residual variation can indicate differences where none exist, hide differences that do exist, and even show differences in the opposite direction of what actually exists’. Consequently, standard errors will be wrong and the parameter estimates will be bi- ased. A good example of this problem is presented by Allisson (1999). When estimating the chances for academic promotion, he initially found that the effect of the number of articles differed significantly for women and men. Given the oddity of this result, he claimed that this was the result of differences in the degree of residual variation between men and women. When examining this further he indeed found that women have more heterogeneous career patterns than men, and that unmeasured variables have a different effect on men than on women, leading to the observation that the number of articles is more important for men than for women. Heteroskedasticity may also be a problem in our estimation. Subsequently, we test for the presence of heteroskedasticity using a likelihood ratio test7. Basically, the likelihood ratio test checks whether the heteroscedastic model has better fit than a homoscedastic model. The test confirms that this heteroskedastic model has a better fit than the homoscedastic model, χ² (4) = 28.14*** when the residual variance is mod- eled by high result control and the country dummies: Belgium, the Netherlands and Germany. This is achieved by applying a heterogeneous choice model8. Residual vari- ability in using financial management techniques consequently differs for agencies with and without high result control and for the above mentioned countries. A wide range of easy accessible literature concerning this model is available; see for instance Cameron and Trivedi (2005) and Williams (2010). The heterogeneous choice model is an improvement over the initial and often criticized approach of Allisson (1999), see for instance Keele and Park (2006), and overcomes its most significant weaknesses (Williams, 2009). These models allow for ordinal dependent variables and a much more flexible specification of the variance equation. As Keele and Park (2006) note, if this kind of model is correctly specified ‘the heteroskedastic ordered probit model can be given a clean bill of health, as both the level of overconfidence and coverage rates are close to ideal’. Consequently, a heterogeneous choice model is presented in column 2 of Table 4. We can note that the heterogeneous choice model produces a larger model chi-square (89.9 versus 61.75) but at the cost of 4 degrees of freedom. 6 This is a logical consequence since the β’s are calculated using α and σ, see (2). 7 More in particular we make use of Williams (2010) oglm’s stepwise selection procedure. Although stepwise selection procedures are often criticized as a model-building device be- cause they are atheoretical and can capitalize on chance, they can be useful as a means for identifying heteroskedasticity. See Williams (2010) for more information. 8 We make use of Stata (12.1) and Williams (2010) oglm command however, SPSS users can make use of the PLUM routine. 217 Moreover, both the BIC statistic as well as the AIC statistic favor the heterogeneous choice model9. When examining the heteroskedasticity term of high result control we notice a negative significant effect, which means that there is less residual variability for agencies with high result control than there is for agencies with low result control when it comes to the use of financial management techniques. The effect of unmea- sured variables on the use of financial management techniques is smaller for agencies with high result control than for agencies with low result control. In other words, other unmeasured factors have a stronger effect on the use of financial management techniques for agencies without high result control. Intuitively this makes sense. The higher the accountability, the less important other factors are, while the lower the ac- countability, the more important other factors become. When comparing coefficients of the homoscedastic (regular ordered logit model) and the heteroskedastic model we notice that results appear more or less the same. Nonetheless, the heterogeneous choice model yields important insights into the ef- fects of high result control that would be overlooked in the misspecified ordered logit model (column 1). An examination of marginal effects helps to clarify what the sub- stantive differences are between the two models. With marginal effects, all variables except one are set equal to their means, and we see how changes in the remaining variables affect the probability of each possible outcome occurring (Williams, 2009). For a dichotomous regressor, we measure the effect as the variable changes from 0 to 1. For continuous variables, the instantaneous rate of change is measured. Table 5 presents the marginal effects for the ordered logit and heterogeneous choice models. For the ease of interpretation only the variable with high result control is included. Table 5: Marginal effects for the ordered logit and heterogeneous choice models without interactions, focusing on the effect of high result control (1) (2) Ordered logit Heterogeneous choice NO USE OF COST CALCULATION SYSTEMS High result control -0.160*** -0.261*** (6.97e-05) (4.22e-08) SOME USE OF COST CALCULATION SYSTEMS High result control 0.0434*** 0.221*** (0.00435) (0.000221) HIGH USE OF COST CALCULATION SYSTEMS High result control 0.116*** 0.0398 (0.000581) (0.356) pval in parentheses. ***