Microsoft Word - 35fakandu.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 53, 2016 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Valerio Cozzani, Eddy De Rademaeker, Davide Manca Copyright © 2016, AIDIC Servizi S.r.l., ISBN 978-88-95608-44-0; ISSN 2283-9216 Internal Auditing of Process Safety Lee Allford European Process Safety Centre,(EPSC), Davis Building, Rugby, CV21 3HQ, United Kingdom lallford@icheme.org This paper covers the major highlights from a revised member report from the European Process Safety Centre (EPSC) on the topic published in 2012 which included the results of a survey of member practice on process safety auditing as well as drawing on personal accounts on its critical success factors. During the course of 2014 further research was conducted on the related topic of internal auditing of process safety. The scope of the work included a literature review of the state of the practice with respect to the audit of process safety across the major hazard industries. In addition an online survey was conducted on current policies, standards and work practices associated with the internal audit of process safety. Finally survey respondents were invited to assess the commonly voiced barriers to successful auditing as well as propose areas for future research. 1. Background A safety audit according to EPSC (2001, p.7) is a “process of independent, systematic examination to assess the extent of conformance with defined standards and recognised good practice, to thereby identify opportunities for improvement”. Later in the same report EPSC (2001, p.11) describes a safety audit in glowing terms as “a positive and helpful force for improvement, owned and welcomed by management and conducted on a planned and regular basis”. Nevertheless investigations into major accidents in recent years have revealed flaws in process safety management systems and more specifically shortcomings in the audit process itself which is designed to maintain and improve the system. The irony of flawed audit assurance is not lost on Sutton (2009) who warns that one of the early signs of a degraded process safety culture is that of ineffective auditing. This shows itself in the softening of bad news to senior managers, a failure to find root causes of deficiencies, and an inadequate follow up to audit findings. In the aftermath of Texas City the BP U.S. Refineries Independent Safety Review Panel (2007, p. xv) expressed its concern that “the principal focus of audits was with compliance and verifying that required management systems were in place to satisfy legal requirements”. In one of its ten key recommendations the panel made explicit reference to process safety auditing and proposed that BP should establish and implement an effective system to audit process safety performance at its U.S. refineries. In a similar vein, the HSE (2001, p.5) found that at Buncefield that “auditing and monitoring arrangements focused on whether a system was in place; the audits did not test the quality of the systems and, most importantly, did not check whether they were being used or were effective”. In other words there existed a pre-occupation with system compliance at the expense system performance. Moreover the National Occupational Health and Safety Commission (2001) whilst recognising several benefits of safety audit tools observed that they can act as a barrier to the effectiveness of a SMS. The Commission listed several shortcomings with audit tools which include their “one size fits all” approach and their inability to assess the elements which are pre-conditions for an effective SMS such as senior management commitment and employee involvement in the planning, implementation and review of a SMS. The theme of audit frailty is further developed by Blewett and O’Keefe (2011) who identified several weaknesses related to the auditing of SMS which include paperwork for its own sake (so called tick box auditing), inherent problems associated with scoring an audit and lack of auditor independence and competence. DOI: 10.3303/CET1653053 Please cite this article as: Allford L., 2016, Internal auditing of process safety, Chemical Engineering Transactions, 53, 313-318 DOI: 10.3303/CET1653053 313 2. Introduction The objectives of the research were to explore the facts and opinion on the internal auditing of process safety from the perspective of process safety practitioners who play a role in the auditing of process safety. Specific objectives were to • Survey current policies, standards, practices and opinion associated with the internal audit of process safety • Examine the barriers to effective process safety auditing • Identify areas for future research in the field of process safety auditing The web survey was compiled with the use of commercial survey software. In the order of 300 personalised invitations selected from a database of process safety practitioners were distributed from 21-28 August 2014 and at the close of the survey on the 1 December 2014 the survey had attracted 89 respondents which represents a response rate of about 30%. As with many surveys of this nature not all respondents fully completed the survey although partially completed surveys were included in the overall results. The survey population comprised audit programme managers (17%), audit team leaders (23%) audit team members (34%) and auditees (10%) as well as other roles associated with auditing including technical specialist and management oversight roles. The majority of respondents (95%) saw themselves as working for multinational companies. 3. Scope The scope of the research was to be confined to that of internal auditing of process safety and the facts surrounding and opinions towards this subject as expressed predominantly by representatives of EPSC member companies who profess to have either current or recent experience of involvement in process safety auditing. Out of scope was the auditing of general workplace safety and external auditing of process safety as conducted by either customers or regulators and insurers and as described in Table 1. Table 1: Internal and external auditing Internal auditing External auditing Supplier auditing Third party auditing Sometimes called first party audit Sometimes called second party audit e.g. for regulatory, certification and insurance purposes 4. The Audit Process 4.1 Pre-Audit The planning and preparation of an audit is a necessary condition towards the satisfactory conduct of an audit. This section of the survey explored various aspects of preparing for the audit such as degree of interaction between the audit team and the the audited site and amongst the audit team itself. The results show in Figure 1 that about two thirds of audit teams hold a pre-audit meeting with the auditee and a similar number request that the auditee completes a self-assessment questionnaire. The survey did not explore the correlation between audit performance and self-assessment scores but anecdotal evidence presented at an EPSC meeting suggests that those sites which score themselves highly tend to perform less well during the audit than those whose assessment is less glowing. About three quarters of the respondents stated that they hold a meeting of the entire audit team prior to the audit visit. This meeting may be a virtual rather than a physical meeting due to team members having their home base on different sites and in different regions and therefore offering a degree of independence to the site under audit. A less common practice was to organise unannounced or short notice audits which no doubt would be less welcome than those where sites receive notice of some months or at least several weeks. The advantage of surprise audits is that they provide an opportunity to observe a site operating under its usual, normal working conditions. The drawbacks are that key staff may not be available or part of the plant may not be functioning at the time of the visit. 314 Figure 1: Pre-audit practices 4.2 Onsite Audit This section of the survey explored the practices associated with onsite activities such as gathering audit evidence. Gathering audit evidence ostensibly comprises three modes which are the visual such as observation and inspection, the verbal such as interview and the written such as records and documentation. From Figure 2 a sizeable majority of respondents stated that their business conducts inspection of safety critical equipment, observation of safety critical activity and examination of major hazard risk assessments. The survey posed further questions about the conduct of staff interviews be they group or one to one interviews. Each has its pros and cons with group interviews offering the competent auditor the opportunity for site interaction and one to one interviews the opportunity for candid disclosure of personal views. With respect to process safety each mode of gathering audit evidence has its strengths and weaknesses and an audit team may choose to weight its approach on an audit by audit basis. For example a team faced with a plant with an ageing profile may decide to spend the majority of its time on field inspection. For a plant where human error is a large contributor to major accident causation the team may elect to spend the majority of its time interviewing staff. In order to assess understanding of major accident hazards, the safeguards in place and emerging procedures. Figure 2: Onsite audit practices 4.3 Post-Audit The section covers a broad range of practices and activities after the onsite audit, some of which may lie outside the normal remit of the audit team. Post-audit activities can offer learning opportunities for the audit team and wider business. The survey posed several questions related to debriefing /evaluation of the onsite audit from the perspective of the audit team and auditee. Over 80% of respondents stated that they conduct debriefings with the entire audit team post-audit. An advantage with this approach is that collective feedback can result in several suggestions for improvement. In contrast about 30% of respondents reported members of 0% 20% 40% 60% 80% 100% a pre-audit meeting with auditee? an auditee self assessment questionnaire prior to onsite audit? meeting of entire audit team? unannounced /short notice audits? In the auditing of process safety does the audit team in preparation organise( N = 79) Yes No 0% 20% 40% 60% 80% 100% one to one interviews with site staff? group interviews with site staff? inspection of safety critical plant & equipment? observation of safety critical activities? examination of major hazard risk assessments? During the onsite auditing of process safety does the audit team organise (N = 77) Yes No 315 the audit being interviewed separately post-audit. An advantage of separate interviews is that they may well flush to the surface any concerns that an individual auditor may have with respect to the presence of any biases or lack of impartiality as exhibited by the audit team during the conduct of the audit. Audit teams may also opt to gather opinion on the audit from the point of view of the auditee which may seem appropriate given that the site may be the greatest benefactor of the audit. Figure 3: Post audit practices Several questions were posed regarding the sharing and publication of audit results. These served as a surrogate measure of employee engagement which is considered to be essential to successful auditing outcomes and as identified in the literature review. Feedback on audit results to site workers, some of whom would have had contact time with the audit team, would seem a sensible way to encourage employee engagement. About 70% of respondents said that that they shared audit results within the site and 50% respondents stated that they shared audit results across the business which can in some instances result in the complication of league tables of performance and publishing these throughout the company. About 57% of respondents declared that their business scored audits. Scoring audits offers the advantages of tracking audit performance over time for the same site and also benchmarking across sites. The downsides appear to be that scoring audits can diminish the audit to a bottom line score or several headline scores which can be misleading especially when taking into account changing or different risk profiles and the assessment of major and minor deficiencies. In a general SHE audit for instance the major hazard risks are in danger of being masked by scoring when their critical nature would be best elevated. 5. Auditing Barriers This section invited respondents to offer their personal opinion as to the biggest barriers to successful process safety auditing. Respondents were invited to score several suggested barriers to successful process safety auditing and a breakdown of those which attracted the highest scores are show in Figure 4. In the biggest possible barrier category, the most commonly assigned barriers were lack of senior management commitment, lack of auditor competence, the over-auditing of sites, that the audit is too shallow and that no budget is available to resolve audit findings. 0% 20% 40% 60% 80% 100% auditee evaluation of the audit? debriefing of the audit team altogether? debriefing of individual auditors separately? for audits to be awarded a score? for the audit results to be shared widely within the site only? for the audit results to be shared widely across the business? In the auditing of process safety does the business post-audit organise (N = 76) Yes No 316 Figure 4: Barriers to process safety auditing 6. Future Research The final section of the survey invited respondents to propose one question which they would have liked to have seen answered by their peers in a future survey. The proposed questions have been organised into the following topic areas. 1. Audit Organisation & Rationale 2. Auditor Competence & Motivation 3. Audit Intensity 4. Audit Content 5. Audit Results 6. Audit Benefit & Success Audit organisation concerns the strategic considerations relating to the organisation’s choice of audit as an assurance tool particularly at different levels of the business and how these audit programmes relate to each other. One interesting question is whether an audit for a multinational business adopts a common approach throughout its operations or tailors the audit to the risk profile of a particular hazardous operation. Auditor competence relates primarily to the formal qualifications for members of the audit team and the incentives for becoming an auditor in the first place. It is often overlooked that auditing places a load not only on the audited site but also on the auditor. Full time auditors can travel extensively within their territories and part time auditors return to their home workplaces and their “day jobs” after conducting an audit. The question is what rewards and incentives exist for a part time auditor to engage in taxing extracurricular activity such as auditing Audit intensity describes to the audit frequency and effort associated with the internal auditing of process safety. One interesting question is more related to lack of intensity and concerns the freedom of the audited site to postpone or cancel scheduled audits and the circumstances surrounding that decision. Audits can be disruptive but can also identify major accident vulnerabilities in such extenuating circumstances. Audit content comprises questions regarding the selection of audit topics including those that are identified though near miss analysis and the scientific basis for that choice stemming from an assessment of safety barrier quality. Audit results describe issues relate to the escalation process for close out of audit actions and the involvement of senior management in the audit follow up. The timely closure of agreed actions is vital to audit effectiveness. Nevertheless it is recognised that from time to time audit actions will not be addressed within the agreed timescales. The final topic area of audit benefit and success concerns questions about auditing cost benefits, definitions of auditing success and the correlation between internal audit scores and EHS performance. 0 20 40 60 80 100 120 140 1 2 3 4 5 What are the biggest barriers to successful process safety auditing? 1 = small barrier, 5 = big barrier (N = 74) Sites are over-audited. Lack of senior management commitment No budget to resolve audit findings Audit is too shallow/superficial Lack of auditor competence 317 7. Conclusion As far as audit implementation the majority of respondents reported in the pre-audit phase that they met with the auditee and convened a meeting with the entire audit team. There was widespread use of an auditee self assessment prior to the onsite audit. A less common practice was the organisation of unannounced or short noticed audits. For the onsite audit most respondents reported that they gathered audit evidence through the conventional modes of the visual, the verbal and the written. For interviews of staff most respondents conduct one to one and group interviews with site personnel which is in line with good auditing practice to use a variety of means to gather audit evidence. Post audit a sizeable majority of respondents reported that their business debriefs the entire audit team on the onsite activities. Fewer respondents but still in the majority stated that they conducted auditee evaluation and only a minority of respondents reported that they debriefed individual auditors. Only a modest majority of respondents declared that they scored audits, which reflects that a single audit score or series of scores can cause the auditee to be pre-occupied with the bottom line number at the expense of a broader understanding of how audit scores are derived. Disclosure and sharing of audit results offers an opportunity to engage the workforce of the site under audit in the auditing process. The majority of respondents reported that they publicise audit results within the audited site and less commonly that they publicise the audit results across the business. The biggest barriers for successful auditing was believed to be lack of senior management commitment, lack of auditor competence, sites are over audited, audit is too shallow and no budget to resolve findings. The first two barriers confirm many criticisms echoed in the literature by regulators and commentators alike. It is worth noting that even for the biggest auditing barriers many respondents still saw these as relatively minor impediments to successful auditing. Many survey respondents proposed future research questions in the area of process safety auditing which suggests that whilst those involved in process safety auditing do not see the exercise as offering a false sense of security that at the same time there is significant industry interest in improving the audit process. Several respondents elaborated their survey responses with comments which in the main indicated a sense of confidence in the effectiveness of internal audits although it is clear that an audit which at best is both a snapshot and sample offers no guarantee to avoidance of a major accident. References Baker Panel Report, 2007. Report of US Refineries Independent Safety review. http://news.bbc.co.uk/1/shared/bsp/hi/pdfs/16_01_07_bp_baker_report.pdf accessed 01.06.16 Blewett, V., O'Keeffe, V., 2011. Weighing the pig never made it heavier: Auditing OHS, social auditing as verification of process in Australia. Safety Science 49(7) European Process Safety Centre, 2001. SHE auditing practice in the process industries. EPSC. Available on Request Health and Safety Executive 2011. Buncefield: Why did it happen? HSE http://www.hse.gov.uk/comah/buncefield/buncefield-report.pdf accessed 01.06.16 National Occupational Health and Safety Commission, 2001. Occupational Health and Safety Management Systems - a review of their effectiveness in securing healthy and safe workplaces: Sydney NOHSC Sutton, I., 2009. Should Your Organization Fly Warning Flags? Chemical Engineering Progress, 105(12) 318