Microsoft Word - AJIs10-2finala.doc AJIS vol. 10 no. 2 May 2003 81 THE AUSTRALIAN GOVERNMENT’S ABANDONED INFRASTRUCTURE OUTSOURCING PROGRAM: WHAT CAN BE LEARNED? Anne C. Rouse Department of Information Systems University of Melbourne Email: acrouse@forensys.com.au Brian J. Corbitt School of Information Systems Deakin University ABSTRACT Early in 2001, after a damning public report by the Auditor-General, the Australian Federal Government abandoned its highly promoted “whole of government” IT infrastructure outsourcing initiative. This about-face was greeted in the press with reports that the initiative was a “fiasco”. Yet a four-year case study conducted by the authors suggests a more complex picture. Like many other “selective” outsourcers of IT, the Federal Government had been led to believe that it was adopting a relatively low risk strategy that would, if well managed, lead to significant cost savings and operational benefits. Instead, despite having implemented many widely promoted “best practices”, the Federal Government found a substantial discrepancy between what outsourcing promised to deliver, and what was actually achieved. In this respect their experiences were no different from those of many other large IT organizations engaged in selective IT outsourcing, who responded to a substantial contemporaneous survey. This case study examines why the Government’s expectations were not achieved, and arrives at conclusions that have important implications for decision makers confronted with choices about sourcing IT service delivery. Keywords Outsourcing of IS EL07, IS Strategic planning EF04, Issues in organising IS, EG03 IS Risk management EL08, Selective outsourcing, Public sector, Government outsourcing, Case study, Survey INTRODUCTION On 24 July 2001 an article appeared in the influential IT pages of The Australian claiming that as a result of the “Federal Government IT outsourcing fiasco” many public and private sector organisations were rethinking their attitudes to IT outsourcing. The article stated that “concerns about the handling of federal outsourcing had turned private industry off the whole concept” (Mitchell, 2001a, p 31). This article highlights the controversy surrounding the Federal Government’s outsourcing of its IT infrastructure. Six weeks later, the issue again hit the headlines in The Australian with a report that the 2000 Senate Finance and Public Administration References Committee had found the initiative was seriously flawed (Mitchell, 2001b p 41). Between 1998 and 2001 the initiative was rarely out of the headlines. This paper reports a four year case study of the Federal government whole-of-government IT infrastructure outsourcing initiative. The paper provides a brief outline of the case; reports the data sources used; considers several key observations that can be made about the case, and presents several implications these have for decision makers. CASE SYNOPSIS In April 1997, the Australian Federal Government announced that by 1999 it intended to outsource the delivery of IT infrastructure (mainframe, server, and desktop services, and in some cases telecommunications) for most federal agencies. At the time it was suggested that this initiative would lead to saving of up to $AUD1billion; boost local industry development; and provide access to new technologies. Savings were to occur through the use of external vendors, and the consolidation of requirements across government agencies, which would, in some cases, be clustered to form cross-agency contracts. The decision created some controversy, and was examined by an ongoing Senate Inquiry (SFPARC, 1997). Expecting some internal resistance to the initiative, and anxious that projected cost reductions not be dissipated, in the 1997 budget Government deducted from line budgets the 15% savings expected to accrue from outsourcing IT. In 1999, the Federal Auditor-General independently decided to review the outcomes of the first four contracts under this initiative. His September 2000 Report (ANAO, 2000) found that the Government’s ambitious timetable had not been met, and that the costs of market testing and contracting had blown out substantially (by over $AUD26 million). Most of these cost blow-outs were attributed to fees paid to specialist international consultants, including ‘Big 5” accounting firms, outsourcing advisors, and the outsourcing legal specialists, Shaw Pittman. These advisors were engaged to facilitate the implementation of the initiative. The Auditor- General also found that, contrary to Government claims, only one of the first four contracts, the contract for “Cluster 3”, was likely to deliver any substantial savings, and that actual savings after twelve months were less AJIS vol. 10 no. 2 May 2003 82 than half of those projected at the time the contract was signed. At the same time the Auditor-General found that outsourcing had a number of negative consequences on the operational business of agencies. The Auditor-General’s Report re-ignited community opposition to the initiative, and led the Government, late in 2000, to commission an independent inquiry to examine the risks associated with the initiative (the Humphry Inquiry). In January 2001 the Federal Government released the Humphry Report (Humphry, 2000), which recommended that the centralised and consolidated approach to IT outsourcing be discontinued. However, agency line managers would still be required to evaluate the outsourcing of their IT infrastructure services and justify their decision if they chose not to outsource. Surprisingly, the Humphry Report did not address the issue of cost savings, but did find that the implementation risk (the risk to the agencies’ operations) associated with the consolidated approach were higher than had been expected. The Auditor-General reviewed the first four contracts for the initiative, listed below: Table 1: Agencies analysed by the Auditor-General’s Whole Of Government IT Infrastructure Outsourcing Initiative Review (ANAO, 2000) Agency Value Won by Date ‘Cluster 3’ – DIMA*, AEC* and four small agencies $160m CSC Mar-98 Department of Employment, Training and Youth Affairs and Employment National (DEETYA-EN) $300m Cancelled Jun-98 Australian Tax Office (ATO) $490m IBM/GSA Jun-99 ‘Group 5’ – five small agencies $90m Advantra Jul-99 *DIMA: Department of Immigration and Multicultural Affairs, AEC: Australian Electoral Commission Contrary to suggestions made elsewhere (e.g. Seddon, 2001; Willcocks’ submission to SFPARC, 2001), although relatively large in scope, the whole of government outsourcing initiative was not an example of “total outsourcing”. Instead, the initiative was an example of “selective outsourcing8”, involving only infrastructure IT services and accounting for around half of IT expenditure (SFPARC, 1997; 2001). METHODOLOGY This case study was analysed using hermeneutic textual analysis. Hermeneutic analysis is a process used to identify contextual explanations for apparent contradictions found in qualitative text-based data. The textual material is repeatedly sifted and compared to find cross-verification and contradictions. Understanding emerges from the comparison between new and previous information. The strategy is akin to the work of a historian sifting through competing accounts of past events and motivations. Text sources used by the authors included: • Notes from interviews conducted between 1997 and 2001 with 16 informants working for 9 Federal agencies, or for their IT service vendors. In most cases, informants were interviewed more than once. • A literature review based on press reports related to Australian IT outsourcing between 1997 and December 2001; • Detailed analysis of a number of Federal and State government reports, particularly the 1997, 1998 and 2000 Senate Finance and Public Administration References Committee Inquiries into outsourcing initiatives (SFPARC 1997, 2001), the Auditor-General’s Report (ANAO, 2000); and the Humphry Report (cited as Humphry, 2001); • Walker and Walker’s (2000) broad based analysis of government outsourcing in Australia; • A study of the case by Seddon (2001) that is drawn from the Auditor-General’s and Humphry Reports; • Analyses prepared for members of parliament by the Parliamentary Library Politics and Public Administration Group (Verspaandonk 2001a, 2001b); • Notes from a series of focus groups involving 56 respondents reporting on their experiences of IT outsourcing; and • Analysis (reported in Rouse, 2002). of a University of Melbourne IT outsourcing survey of 1000 large sites in Australia, distributed in 2000. This survey, which had a 24% response rate (n = 240) was 8 Lacity & Willcocks (1998) have defined “selective outsourcing” as outsourcing of between 20% and 80% of the IT budget, in contrast to “total outsourcing” in which over 80% of the IT budget is outsourced. They argue that total outsourcing is particularly problematic, and recommend strongly against it because of the risks. AJIS vol. 10 no. 2 May 2003 83 conducted by a team of five at the University that included Rouse, Seddon, and Willcocks (an initial report is found in Seddon et al, 2000)9. KEY FINDINGS The authors’ analysis of the case revealed several key findings (discussed below) that provide a lens through which the case can be interpreted. The 15% savings target Although the initiative encountered problems in a number of areas, it is unlikely that it would have been abandoned if it had delivered the 15% savings expected on the basis of expert advice received from vendors, industry specialists, consultants and academics. At the time vendor bids were assessed, detailed costings were carried out by OASITO (the Office of Asset Sales and IT Outsourcing, the agency tasked with managing the initiative) under the guidance of specialist consultants, and in consultation with agency management and IT staff. These bids in turn were sought only after extensive research had been undertaken of the services likely to be required, and the costs of internal service delivery. Projections at the time vendors were selected suggested savings of up to 28% were likely, and a target of 15% would have appeared achievable. However, the Auditor-General’s analysis (ANAO, 2000) revealed that despite expert advice, the cost and savings projections for two of the arrangements (ATO and Group 5) were flawed. The business model projections had failed to take into account additional cash streams associated with equipment that would be available at the end of the contract if the services were not outsourced. They had also under-estimated the costs of leasing risks absorbed by the agencies. According to the Auditor-General the consequence of these flawed projections was that only one agency (Cluster 3) was likely to make any substantial savings from the outsourcing arrangements. Reviewing the business model developed when the vendor was selected, the Auditor-General determined that projected savings for that group should have been 28%, of which 5% were associated with one-off retrenchment costs. However, the actual savings in the first 12 months of the contract were only 12%.  3% short of the Government’s apparently conservative 15% target, and less than half the ongoing savings the business model projected. Because Cluster 3 agencies were required to make up the 3% shortfall out of operating budget, this over-optimism had profound negative effects on their operational performance, and the 12% savings were seen in a negative light by all involved. However, examination of the empirical literature on outsourcing savings, as well as discussions with 56 IT practitioners in the focus groups (Rouse, 2002) suggests that savings of 12% are at the high end of what can realistically be expected from outsourcing IT. The contemporaneous survey of IT outsourcers conducted by the University of Melbourne in late 1999/early 2000 revealed a quite different picture to that expected by government decision-makers, and instead confirmed the information coming from the focus groups. In that study only a minority of selective outsourcing cases (42.4%) reported any savings at all, and only 8.7% reported “substantial” savings. These ratios were statistically no different to those for the sample as a whole  42.1% and 7.4% respectively (Rouse, 2002). The finding that most respondents failed to report any savings from outsourcing applied whether respondents were government or non-government organisations, medium, large or very large organisations, and whatever proportions of their IT budget was outsourced (Rouse, 2002). In light of this information, the experiences of the Federal initiative can be seen as typical, rather than aberrant. Other, smaller contemporaneous surveys  such as the two reported by Lacity & Willcocks, (2000, 2001) with a total of 77 outsourcing respondents; and that by Aubert et al (1999) with 70 respondents  also found that IT outsourcing did not result in substantial savings. Cost increases, on the other hand, occurred in 22% of the respondents to the University of Melbourne study, and in up to 49% of the respondents to Aubert et al’s (1999). Decision makers went to some trouble to follow recommended practices It has been suggested in the press and academic literature that the Federal Government initiative represents a failure by decision-makers to follow recommended practices. Seddon (2001), for example, juxtaposes an account of the initiative with what he labels “ten commandments” for good practice, implying by doing so that these prescriptions were not followed. Most of these prescriptions are based on a range of practices described as “proven” to lead to outsourcing success by Lacity and Willcocks in their most recent text (2001, 9 Referred to in this paper as the University of Melbourne survey. AJIS vol. 10 no. 2 May 2003 84 pp 6-16). These practices were articulated in the mid 90s, and arose largely from Lacity’s and Hirschheim’s (1993; 1995) series of case studies. These authors had observed that many organisations that experienced substantial problems with outsourcing engaged in what could be described as naive behaviour, such as failing to carry out thorough investigations of their own and market costs; and signing long contracts (10+ years) with a single vendor for substantial proportions of their IT budget, so increasing dependence on the vendor. As a result of these case studies Lacity and Hirschheim developed a set of prescriptions for good sourcing practices. These were further codified and augmented by Lacity and Willcocks on the basis of a post hoc examination of 61 case studies one or other author had conducted in Europe and the US (described in Lacity & Willcocks, 1998; 2001). Our investigation of the federal initiative suggests that “not following recommended practices” is an inadequate explanation for the failure. The fact that the government spent over $25 million on obtaining and implementing specialist consultant advice related to the design and implementation of the program (ANAO, 2000, p 17) indicates that decision-makers were keen to make the arrangement work well. Government had also received advice from the IT Review Group (ITRG); the IT&T Policy Advisory Committee, panels of industry specialists, and from the Australian Industries Commission. All of this advice suggested that if the process was carefully managed, the Government could succeed in achieving the targeted 15% savings (and possibly much more) without compromising operational effectiveness. This message is, incidentally, prevalent in current MIS textbooks, despite the fact that it has little empirical support. The evidence gathered by the authors is that the Federal Government spent considerable effort in planning for, and implementing the initiative, even though, in hindsight, the costs and time involved were greater than had been expected. In this respect they were typical of other Australian organisations (Seddon et al, 2000). The case study data gathered by the authors suggests that decision-makers followed almost all of the “proven practices” from the literature. Of Seddon’s “ten commandments” for IT outsourcing success, there is evidence that only one was not followed. This practice relates to allowing the internal IT services groups to bid against external vendors, which case informants suggested was an expensive and impractical proposition. To manage risks, government decision-makers adopted many practices recommended by Lacity, Hirschheim and Willcocks, and summarised in Lacity and Willcocks (2001). These included: • selective outsourcing (at around 50% of IT budget); • targeting services thought likely to be generic and “non core”, i.e. infrastructure services which Lacity & Willcocks (2001 p 9) and Seddon et al (2000 p 4) have suggested are frequently outsourced successfully; • use of short contracts (3 to 5 years); • crafting of detailed and comprehensive contracts that were prepared with the advice of specialists; • retaining highly-skilled specialist staff to manage the arrangement; • involving both line and IT management in the decision, even though, in the end, senior management (i.e. Cabinet) made the decision; • conducting detailed and extensive analysis of vendor offerings and both internal and market prices; and • designing and implementing post-contract management structures and processes. Regular formal and informal monitoring and SLA negotiations were instituted as part of this process. Looking at the management attention and resources devoted to the initiative, the case appears to be a textbook example of decision-makers responding to the best available advice. This observation raises a critical question: “How likely is it that organisations that do implement good management practices will “succeed” in IT outsourcing?”. Research, which influenced the decision, did not necessarily support the decision Examination of the Industry Assistance Commission’s report into contracting out, the Senate Inquiries and the citations used in discussion papers (e.g. Verspaandonk, 2000b) indicates that Government’s expectation, that 15% savings were realisable from IT outsourcing, were influenced by several key studies. Particularly important were findings by Domberger, Hodge, and Lacity & Willcocks. This research was cited by informants, and in public documents, as evidence that carefully-managed outsourcing frequently leads to substantial cost savings of 20% to 30% or more. However, closer analysis of this work reveals that a more complex relationship exists. For example, research by Domberger (CTC, 1999) suggested savings of 30%+ from outsourcing were common, and this research played an important role in setting Government expectations (Rouse, 2002, Walker & Walker, 2000). Walker and Walker (2000) identify several flaws in some of this research, and careful examination of the most recent data (CTC, 1999) reveals that while such savings were common for simple outsourced services such as cleaning and catering, increases of around 8% were observed when IT services were outsourced. AJIS vol. 10 no. 2 May 2003 85 Hodge (1996) carried out a meta-analysis of a large number of quantitative studies of general outsourcing by government agencies. His studies translated results from earlier studies to a common metric, so that data could be consolidated and re-analysed. He found savings of between 8% and 14% were possible but these were largely for simple services, like garbage collection, or cleaning (Hodge, 1996). However, savings for corporate services (which would include IT) increased on average by 5% (Hodge, 1999). In relation to IT services, Lacity and Willcocks (1998) reported cross case comparisons that suggested most outsourcing (70%+ of the 33 outsourcing cases with discernible outcomes) led to savings. They found that “selective” outsourcing of IT services was particularly successful. For their selective cases reporting cost savings  a large majority (85%) of the selective outsourcing cases that reported an outcome  savings were, on average, a substantial 23.8% (data from Lacity & Willcocks, 1998, re-analysed in Rouse, 2002). But the authors acknowledge in that paper that these cases are not statistically representative. The fact that these cases are not statistically representative means they shed no light on the success rates that might be expected in the wider population. Furthermore, researchers often have difficulty gaining access to case study sites involving problems or failure. So the positive odds of success represented by the cases studied by Lacity and Willcocks (1998; 2001), and the substantial cost savings reported for selective outsourcing (1998) are not those that typical outsourcing purchasers can expect. In spite of these limitations, the success ratios reported by these authors were cited by a number of informants involved in the initiative as justification for their strong belief that their selective IT outsourcing would succeed. Contemporary evidence indicates that selective, well-managed outsourcing still involves considerable risks In following the various recommended practices discussed above, the Australian Federal Government was typical of the large organisations studied in the University of Melbourne survey. Most of those respondents also followed recommended practices like selective outsourcing; outsourcing infrastructure services; employing short contracts; benchmarking IT services; and detailing services and service levels in their contracts. Despite this, most survey respondents were disappointed with their IT outsourcing experiences, with only just over 1/3 (36%) reporting a positive score on a measure of satisfaction and value. Less than a third of respondents reported positive scores for measures of strategic benefits, technology benefits, and economies of scale. When it came to cost savings, only 7.4% of respondents reported substantial savings, and only 42.1% reported any savings at all. One explanation for the problems experienced by the first four agencies in the Federal initiative is that IT outsourcing  even selective outsourcing involving short contracts and infrastructure services  is much riskier than has been recognised. The fact that previous research did not quantify levels of risks appears to have led to unrealistic expectations and over-optimism. Much of the research on risks has been based on case studies (e.g. Lacity and Hirschheim, 1993) or subjective-argumentative research (e.g. Earl, 1996). It may be that decision-makers discounted the risks described by researchers in the belief that they were rare, or only encountered by poorly managed organisations. That risks are substantial is evident in the responses to the University of Melbourne study. Analysis revealed that risks ranged from high to very high for both public and private sector organisations in that survey (Rouse, 2002). Negative outcomes like failing to get expected cost savings; inability to concentrate on core business, failure to avoid technology obsolescence, and inability to control costs were reported by the majority of organisations, while substantial problems (like unexpected cost increases, loss of organisational knowledge, and difficulties projecting future requirements) were encountered by many. The proposition that these risks were not expected is consistent with the low level of satisfaction expressed by respondents, as low satisfaction is usually the consequence when positive expectations are not confirmed by actual experiences (c.f. Rouse, 2002). In light of this contextual information, the negative experiences reported in the reviews conducted by the Auditor-General and by Humphry no longer appear to be examples of flawed management failing to achieve what other, better managed organisations were achieving. Instead they are relatively typical of the experiences of Australia’s top private and public organisations DISCUSSION AND IMPLICATIONS Attributions of the causes of outsourcing failure need to be examined carefully The Humphry Report recommended the abandonment of compulsory centralised and clustered IT outsourcing for future arrangements, because of the potential negative impacts this approach had on operational goals. But as a proponent of the benefits of outsourcing, Humphry did not suggest that IT outsourcing itself was inherently problematic, or that achieving significant cost savings was unlikely. The fact that the Auditor- AJIS vol. 10 no. 2 May 2003 86 General had found that two of the first three arrangements would never achieve substantial savings was not acknowledged. Instead, Humphry cited the substantial savings observed for Cluster 3 by the Auditor-General as evidence that outsourcing can be an economic success. Paradoxically, Humphry also recommended abandonment of the clustering strategy that contributed to these savings. The tenor of Humphry’s Report suggests he accepted the (largely vendor-driven) proposition that well- managed outsourcing usually succeeds. He argues (p 24) that “the debate over cost savings has tended to obscure other benefits, which can arise from properly implemented outsourcing [emphasis added] such as wider access to technology and technical skills, strategic partnership in a dynamic technical environment and an opportunity to manage capital expenditures more effectively”. A belief in this proposition (which does not have empirical support) would have influenced him to seek evidence of poor management as a cause of the failure to meet expectations, as would the political pragmatics of such an Inquiry. Humphry did indeed suggest that the initiative had been compromised by poor management on the part of the central management agency (OASITO). Humphry reported that decision-makers (within OASITO) had concentrated on contractual rather than operational issues, and had failed to ensure co-operation from line management. In addition, he suggested that agency management had been resistant and obstructive: “It is widely accepted that agencies’ inertia and resistance to change contributed significantly to the delays (p 9)... There is a need for [an] agency’s executive and its senior managers to demonstrate support for the process and commitment to its success (p 14). These explanations are not refutable, and so are not helpful. If decision-makers in OASITO had not concentrated on legal and contractual issues, they would have been rightly criticised. And how could decision- makers know “how much” attention to implementation risk was sufficient? Furthermore, any problematic strategy (where key expectations are not being met) is likely to result in resistance from some line managers, particularly if the strategy hampers their operational goals (the situation described by both the Auditor-General and Humphry). Hence the evidence of poor management and line management opposition becomes self- referential: because the initiative failed, decision makers must have failed to gain “sufficient” co-operation of stakeholders and must not have devoted “enough” attention to operational issues. And if line managers expressed concerns about the initiative, this “lack of buy in” (p 11) must have contributed to the failure to achieve the original goals. Yet when these propositions are examined, causal links are not strongly supported. Another factor implicated by Humphry was the clustering that occurred in two of the three contracts studied by the Attorney-General. The Attorney-General too had suggested that clustering had exacerbated difficulties encountered by agencies. Although Humphry defended the original clustering strategy, he suggested it had outlived its usefulness and was by 2000 no longer necessary (p 12). Consequently, a key element of his recommendations was the abandonment of requirements that agencies form grouped contracts (i.e. cluster). Yet clustering had allowed small agencies to benefit from larger agencies’ contract negotiation and management capacities, and attraction to vendors. Significantly, the only contract of those studied by the Auditor-General that was likely to succeed financially was Cluster 3, which chose to renew its clustered contract with CSC in August 2002. When this case is examined in depth, there is little evidence that it was clustering, the implementation flaws described by Humphry, or the concerns expressed by some line managers that resulted in the abandonment of the initiative. Much of the evidence cited by Humphry about risks and downsides of outsourcing had already been presented to the 1997 Senate Inquiry (SFPARC, 1997) but at that stage the Government had decided that the risks were manageable. What had changed by 2000 was that the Auditor-General had revealed that the downsides of IT outsourcing (including the managerial attention that had to be devoted to the arrangement, and the difficulties of implementing within a short timeframe) were substantial, whether or not agencies were involved in clusters. Furthermore, the risks that agencies would experience lock-in with the incumbent vendor at end of the contract were much more visible, as a result of the failure to attract more than one bidder to the fourth contract (DEETYA-EN). At the same time, the likelihood of commensurate financial and industry development benefits was much lower than had been expected. Consequently the risk/return ratio of the initiative was by then demonstrably, and publicly, poor. When the same arguments about risks and problems were presented to Humphry in 2000, together with evidence that impact of these risks were indeed widespread and damaging, he determined that these risks were by then so significant that the initiative should be reconfigured. Given the damaging findings of the Auditor- General’s Report, it is unlikely that he had any other choices open to him. It is tempting to see difficulties reported to Humphry as peculiar to the particular form of implementation (e.g. use of a prime-contractor model, clustering, or exercise of senior-management fiat), or the result of poor management. However, the difficulties can also be seen as examples of an important, and largely inherent, trade-off first identified by Lacity and Hirschheim (1995): certain strategies (like requiring the line to adopt common standards and to consolidate their IT requirements) whether implemented by vendors or internally, can lead to moderate cost savings, but these are often at the expense of the line’s operational goals. AJIS vol. 10 no. 2 May 2003 87 As an earlier supporter of outsourcing, it is unlikely that Humphry would have attributed the failure of the arrangement to an inherent trade-off between cost savings goals and operational needs, or to generally high risks that outsourcing benefits will fail to materialise (as revealed by Rouse’s 2002 analysis of the University of Melbourne survey). Humphry had been one of the industry specialists on the IT&T Policy Advisory Committee that encouraged the Government’s adoption of IT outsourcing and the clustering approach (Humphry, 2001, p 22). A reading of his report suggests a strong commitment to the intrinsic value of IT outsourcing, and its capacity to produce demonstrable benefits if well-managed. Humphry’s commitment to the potential benefits of outsourcing is understandable given the advice and research presented to Government at that time the IT&T Policy Advisory Committee made its recommendations (1997) The fact that the initiative failed to achieve goals that, in hindsight, appear very unlikely to have been achieved puts the case in a different light. So does the evidence (from the University of Melbourne survey) that many other organisations were also failing to achieve similar goals. This suggests that Humphry’s attributions, and his targeting of resistant line managers and a less-than-competent OASITO (which was consequently disbanded) could represent a search for scapegoats for the failure of a policy that was far riskier than was, at the time, recognised. Some organisations will never discover that their cost expectations are not being met Hodge (1996) observed that the more rigorous the evaluation studies of outsourcing, the lower the reported savings. This is because more rigorous studies included costs ignored by less rigorous ones, and controlled for other critical factors. An issue canvassed in Walker & Walker (2000) is the quality of cost and savings projections used to justify Government sourcing decisions. Yet when researchers gather data, where do informants obtain their figures when asked what savings (or dis-savings) they have made from a sourcing decision? As the Auditor-General’s Review reveals, to answer the question of what savings were actually obtained is complex, expensive, and requires substantial expertise. Rolf Jester, a Gartner spokesman, has suggested (in The Australian, 23 Jan 2001), that after as little as 12 months it is not even possible to establish with any certainty what the savings were, an observation also made by the Auditor-General. The process of determining savings requires comparison of four imprecise projections, each involving measurement errors and technical and financial assumptions. These four projections include: • Projections made at the time vendors tender about the service bundle that would be needed over the life of the contract; • Contemporary projections of the likely costs of supplying these services using in-house resources: • Contemporary projections of the chosen vendor’s likely costs to supply these services and the difference between this and those projected for in-house delivery; • Retrospective projections of what the services actually supplied by the vendor (which may be different to those projected at time of contract) might have cost if supplied in-house. Even the calculation of the actual costs and services provided under the outsourcing arrangements involve some imprecision. As the case study has illustrated, this calculation too can depend on assumptions and interpretation of both the contract and accounting conventions. The Auditor-General’s analysis revealed that there was a substantial difference between the services projected at the time the contract was signed, and those actually needed by Cluster 3 over the first 12 months, as well as differences between projected and actual service volumes. These were not the result of ignorance or failure to attend to the issues, but rather a result of unexpected business initiatives (ANAO, 2000, p 196). As Lacity & Hirschheim (1993) revealed, such differences can rapidly consume projected savings. There were also substantial differences between the costs expected for these services, and the actual costs, as a result of different assumptions about how staff would respond to voluntary retrenchment offers; how contractual terms would be interpreted, how quickly certain technology upgrades would occur; as well as some minor errors in the complex financial models used to evaluate tenders (ANAO, 2000, pp .196-197). This is an important lesson for decision-makers, as many organisations could be expected to encounter similar situations. Cluster 3 was the only agency in a position to even attempt the various comparisons discussed above. The clustered Group 5 argued that to carry out this analysis would be prohibitively time consuming and difficult, and might lead to distortions because of the high level of error involved. ATO was too early into its contract to measure savings at the time the Attorney-General reported. It is likely that similar circumstances will prevent most other IT outsourcers from conducting post-implementation reviews of actual cost savings achieved. This means that reported “savings” will often be based upon a very optimistic expectation, and that conflicting information may never be formally gathered to challenge the initial expectation. This is a particular problem for IT outsourcing researchers, because unless purchaser organisations have in fact carried out a comprehensive post-outsourcing review, many “savings” figures reported to researchers are likely to be those projected at the time the contract is signed, and so to significantly over-state actual savings AJIS vol. 10 no. 2 May 2003 88 The Auditor-General’s analysis of the Group 5 and ATO business cases revealed further, much greater error margins in projected costs associated with failure on the part of the decision makers to spot flaws in the assumptions that underlay projections made largely by specialist consultants. According to the Auditor- General these flaws meant that projected savings of 10% (Group 5) and 18% (ATO) would result in probable cost increases, or, at best, savings of 5.4% (ANAO, 2000, p 167) though these savings would quickly be eaten up by management costs and by unexpected changes. Analytic flaws like these are not likely to be revealed even when post implementation reviews are carried out, unless reviews are performed by staff with specialist financial skills, such as those held in the office of the Auditor-General. Without the Auditor-General’s independent (and to some extent, unwelcome) public review, Government decision makers might not have discovered that their expectations were not being met for some years, if at all. But this situation is likely to apply to many other organisations involved in IT outsourcing. Detailed analysis, like that conducted by the Auditor-General, may well reveal that savings are eaten up by the same factors that were at play with the initiative: changes in business requirements and volumes, misunderstandings about what is meant by complex contractual terms, delays associated with evolving technologies, incorrect assumptions, and in complex business cases, possibly errors. IT outsourcing is expected to lead to multiple outcomes, some of which are antagonistic An examination of trade and consulting literature in the last few years reveals a new theme: only naive decision-makers expect to make substantial savings from IT outsourcing: instead, sophisticated managers are seeking strategic benefits, like access to new skills and technologies and concentration on core competencies. This theme can be discerned by examining sites like the vendor-dominated Outsourcing Institute, or those of consultants advising on outsourcing. A cynic might be tempted to see this as a response to the failure over a decade to demonstrate widespread substantial savings from IT outsourcing. However, another possible interpretation is related to the findings of the Federal initiative. This is that IT outsourcing is usually initiated to achieve multiple goals, some of which may need to be traded off. The Federal government was seeking cost savings; better, or at least no worse impact on operational business; and industry development. The evidence from the various documents consulted was that they could achieve some of these goals, but not all. The ATO, for example, has reported substantial technology improvements as a result of its outsourcing arrangement, but reports of substantial savings are noticeable by their absence. According to the Auditor-General, these were always unlikely. Humphry reported a number of negative consequences of agencies’ attempts to meet industry development goals (which were later downgraded as a result of the Humphry Review), and noted that these goals conflicted with agency’s operational goals ( p 36). Elsewhere Lacity and Willcocks (2001) reported that the South Australian Government achieved substantial industry development, but evidence for substantial cost savings from that arrangement is slim. Rouse’s (2002) analysis of the University of Melbourne survey revealed that while the majority of purchasers reported obtaining new skills and positive vendor service from their outsourcing arrangements, this was at the expense of economic benefits (including cost savings); technology benefits, and strategic benefits. The noted economist Shlomo Maital (1994) has observed that only inefficient organisations can achieve organisational improvements without trading off different goals. The Federal Government had been led by advice to expect that they would not encounter unacceptable trade offs when outsourcing IT, but they were unpleasantly surprised. When cost savings were achieved, the operational impacts were quite negative, while arrangements (like, reportedly, the ATO) that succeeded in meeting technology and business goals did so at the expense of substantial savings. The “solution” now in place as a result of the Humphry Review (that is, requiring individual agencies to market test their service requirements) may not end up solving the problems revealed by the Auditor-General’s and Humphry Reports. Reducing complex, rich observations to simplified “recommended practices” has some limitations The failure of an organisation that went to such trouble to manage its risks raises questions about the “operationalisation” of the recommended practices OASITO followed. These practices were drawn largely from the case studies carried out by Lacity and Hirschheim in the early 90s, and the post-hoc cross-case analyses done when Lacity and Willcocks began reviewing their combined research (c.f. Lacity et al, 1996; Lacity & Willcocks, 1998). Examination of the original case descriptions (for example those in Lacity & Hirschheim, 1993) reveals complexity and richness, yet this richness was eventually distilled (Lacity et al, 1996) into a number of relatively simple prescriptions that may not have captured the underlying causal factors. One example is “selective outsourcing”, which has frequently been described as leading to success (Lacity et al 1996; Lacity & Willcocks 1998; 2001). Statistical analysis of the University of Melbourne survey had AJIS vol. 10 no. 2 May 2003 89 failed to find any differences in success measures between “selective” and “total” outsourcers (Rouse, 2002). In seeking an explanation for this, it became clear that the definition of “selective” outsourcing is malleable. While the label was originally used to describe outsourcing between 20% and 80% of IT budget (Lacity et al, 1996); more recently (2001) it has been used by the authors to describe outsourcing as low as 10% of IT budget. Informants in the focus groups conducted by Rouse (2002) tended to define “selective” outsourcing in terms of what it was not, that is “total” outsourcing. Total outsourcing has been presented in the academic literature as a risky and discredited strategy. Yet, as both Lacity & Willcocks (2000) and the University of Melbourne study identified that only around 7% of organisations engage in “total” outsourcing, there are very few decision-makers who would not perceive themselves as being involved in the recommended “selective” outsourcing. As another example, the recommended practice “involve line management and IT management in the decision” came about because some of the problematic cases studied by Lacity and Hirschheim (1993) excluded one or other of these groups from the decision. In the Federal initiative, the Government did involve both line management and IT in their decisions. There was extensive consultation and discussion amongst OASITO and line and IT staff in all the agencies involved. Not all agreed with the decision, but stakeholders had many opportunities to express their reservations through this consultation, and through the Senate Inquiry that took place during 1997. When seeking to reconcile this observation it becomes apparent that the term “involvement” is very general. It does not, for example, necessarily imply that active support from all parties is required for success. There are many examples in the IS and management literatures of senior management instituting ultimately successful organisational programs that encountered disagreements and dissent. Furthermore, such simplicity is not apparent in Lacity and Hirschheim’s original discussion of this practice, which was concerned, with the problems of excluding one or other viewpoint. Again, the impact of this imprecision is that it is likely to be only in those (probably rare) organisations that actively exclude one or other group that decision-makers would not perceive themselves as involving both IT and line management. On the basis of this case study, it appears that many of the recommended practices are necessary, but not sufficient for achieving success, particularly as the probability of their not being followed at all is low. For example, it is unlikely that an organisation involved in a multi-million dollar venture would not establish post management structures and processes of some kind, yet the Federal initiative has illustrated that such processes, even when based on “international best practice” advice do not guarantee success. A similar observation can be made about careful market testing and crafting of detailed contracts. The conclusion that such practices are not sufficient to guarantee success implies that decision makers may be unwittingly led to under-estimate their own risks when outsourcing IT, because most would be able to reassure themselves that they were following recommended risk-minimising practices. It is significant that there has been virtually no evidence provided about how well such practices protect against failure. The bulk of research conducted to date is either case study based, or subjective-argumentative, and so is not able to provide probabilistic estimates of success and risks, or the moderating effects that recommended management practices have on these. Quantitative studies have generally been small, so that the observed relationships could easily have occurred purely by chance. So although there is convincing qualitative evidence that failing to follow these practices can lead to problems, decision makers have no information about what risks they run even when such management practices are followed. CONCLUSIONS This paper has presented a very different view of the Federal IT Infrastructure case than has been presented in the press and some academic literature. In light of the contemporary survey data, it is our view that the case study is not an example of poor decisions and implementation. Instead it illustrates a decision predicated on research and advice that was overoptimistic about likely benefits if IT outsourcing, and unrealistically confident that good management could overcome the associated risks. Many of the outcomes for this “fiasco” are in fact typical of those of contemporary Australian organisations, and probably of many large organisations. A key implication of the case is that decision-makers need to frame their decisions in light of recent research about the relative risks and returns from IT outsourcing (eg Rouse, 2002; Aubert et al, 1999). The case reveals considerable uncertainty surrounding likely outcomes, and highlights the possibility that key assumptions about costs, savings, managerial effort, or the effects of outsourcing on operational performance might be incorrect. Framing IT outsourcing as a quite risky venture (in contrast to the comfortable and reassuring message provided by many vendors and consultants) demands similar strategies to those undertaken when examining any potentially risky venture. Such framing suggests careful examination of assumptions, recognition of the high levels of uncertainty involved in estimates, sensitivity analysis, “risk-boxing” of initial forays, and, as was done by the Auditor-General, early, and ideally, independent, evaluation of the achievement of goals set for the venture. AJIS vol. 10 no. 2 May 2003 90 REFERENCES ANAO Australian National Audit Office (2000). Implementation of Whole-of-Government Information Technology and Infrastructure Consolidation and Outsourcing Initiative. Audit Report No 9. Canberra. Aubert, Benoit A.; Rivard, Suzanne, and Patry, Michel (1999). Chapter 3.3: Impartition des services informatiques au Canada: Une comparaison 1993-1997. in Michel Poitevin (Ed) Impartition Fondements et analyse. Montreal: Canada: Presses de l’universite Laval, 203-220. CTC Consultants (1999) Government outsourcing: What has been learnt? Sydney: CTC Consultants. Earl, Michael J. (1996) The risks of outsourcing. Sloan Management Review; 37(3):26-32. Hodge, Graeme A. (1996) Contracting out government services: A review of international evidence. Montech. Hodge, Graeme A. (1999) Competitive tendering and contracting out: Rhetoric or reality? Public Productivity and Management Review. 22(4):455-469. Humphry, Richard (2000). Review of the Whole of Government Information Technology Outsourcing Initiative ('Humphry Report'). Canberra: Commonwealth of Australia Lacity, Mary C. & Hirschheim, Rudy (1993). Information systems outsourcing: Myths, metaphors and realities. Chichester, England: Wiley. Lacity, Mary C. & Hirschheim, Rudy (1995). Beyond the information systems outsourcing bandwagon: The insourcing response. New York: Wiley. Lacity, Mary C.; Willcocks, Leslie, & Feeny David F. (1996). The value of selective IT outsourcing. Sloan Management Review. Spring; 37(3):13-25. Lacity, Mary C. & Willcocks, Leslie (1998)An empirical investigation of information technology sourcing practices: Lessons from experience. MIS Quarterly. 22 (3):363-408. Lacity, Mary C. & Willcocks, Leslie P. (2000) Inside information technology outsourcing: A state-of-the- art report. Oxford, UK: OXIIM. Lacity, Mary C. & Willcocks Leslie P. (2001) Global information technology outsourcing: In search of business advantage. Chichester, UK: Wiley. Maital, Shlomo (1994). Executive economics. New York: Free Press. Mitchell, Selina. (2001a) Outsource suspicion hits firms. The Australian IT. 24 July : 31. Mitchell, Selina (2001b) Outsourcing’s still a problem, report says. The Australian IT 4 September : 41. Rouse, Anne C. (2002). Information Technology Outsourcing Revisited: Success Factors and Risks. Unpublished Doctoral Thesis. Melbourne: Department of Information Systems, University of Melbourne. Seddon Peter (2001). The Australian Federal government's clustered-agency IT outsourcing experiment. Communications of the Association for Information Systems. 5(13):1-33. Seddon, Peter; Cullen, Sara; Willcocks, Leslie P.; Rouse, Anne C., & Reilly, Colin (2000). Report on Information Technology Outsourcing Practices in Australia, 2000. Version 1, October. Melbourne,: Department of Information Systems, University of Melbourne. Available from Department. SFPARC Senate Finance and Public Administration References Committee (1997) Contracting out of Government Services: First Report: Information Technology. Canberra: Commonwealth of Australia. SFPARC (2001) Rebooting the IT agenda in the Australian Public Service: Final report on the Government's Information Technology outsourcing initiative. Canberra, Australia: Commonwealth of Australia; 2001. Verspaandonk, Rose. (2001a ). Politics and Public Administration Group: The whole-of-government IT outsourcing initiative [Web Page].; http://www.aph.gov.au/library/intguide/pol/IT_outsourcing.htm. Accessed July 1 2001 Verspaandonk, Rose (2001b) Outsourcing for and against: Current issues brief 18 2000/2001 [Web Page] . Available at: http://www.aph.gov.au/library/pubs/cib/2000-01/01cib18.htm. Accessed July 1 2001. Walker, Bob & Walker, Betty Con (2000). Privatisation: Sell off or sell out? Sydney: ABC Books. ACKNOWLEDGEMENTS The research for this paper was funded by Simsion Bowles & Associates, and by a University of Melbourne David Hays “writing up” award. The authors would like to express their gratitude to Professor Liz Sonenberg, Department of Information Systems, University of Melbourne, and to David Watson, Department of Marketing, Monash University, for their helpful comments on earlier drafts of the paper.