CET vol 99 DOI: 10.3303/CET2399032 Paper Received: 19 December 2022; Revised: 11 March 2023; Accepted: 21 April 2023 Please cite this article as: Gotti M., Carluccio L.M., 2023, Quantitative Risk Assessment: Best Practices and Limitations, Chemical Engineering Transactions, 99, 187-192 DOI:10.3303/CET2399032 CHEMICAL ENGINEERING TRANSACTIONS VOL. 99, 2023 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Sauro Pierucci, Flavio Manenti Copyright © 2023, AIDIC Servizi S.r.l. ISBN 978-88-95608-98-3; ISSN 2283-9216 Quantitative Risk Assessment: Best Practices and Limitations Mauro Gotti*, Leonardo M. Carluccio DEKRA Italia S.r.l., Process Safety Department, via Fratelli Gracchi, 27 – 20092 Cinisello Balsamo (MI) mauro.gotti@dekra.com Over the past decades, safety has been permeating process industry with the aim to ensure the protection of people involved in process plants and to prevent catastrophic events and industrial accidents that could be dangerous for the environment and the entire society. Nowadays, process industry requires specific and validated hazard assessment methods to understand the risks to which people are exposed. As a matter of fact, industrial accidents involving either installations or transportation of chemical compounds occur with relative high frequency and lead to many major accidents. QRA analysis (QRA is an acronym for Quantitative Risk Assessment) is a formal and systematic risk analysis approach for quantifying the risks associated with the operation of an engineering process. It can be used for the calculation of the economic and environmental impact that an industry could face after an incidental scenario has occurred, but its focus is on the loss of life; however, many pitfalls are hidden in this analysis, and we need to be aware of them and to avoid them. The paper aims to show the so-called “Seven Deadly Sins” and the “Five Commandments” of QRA, highlighting through real cases and examples from more than thirty years long experience how small mistakes in the input data can generate enormous mistakes. It will also show the golden rules to ensure that a QRA will give as an output useful and meaningful results. 1. Introduction By means of QRA (Quantitative Risk Assessment) several scenarios can be evaluated to understand if the associated risk can be acceptable or not, according to the chosen risk matrix (see Table 1). Indeed, the goal is to calculate the individual risk indices, also called IRPA (Individual Risk per Annum), that are combined with the frequency of presence of operators in each risk area. The resulting assessment is used to demonstrate that an asset meets acceptability criteria and that potential risks are as low as reasonably practicable (ALARP). For this reason, one of the most important things before conducing a QRA is to define the aim of the study. If you are not aware of the real scope, you may be wrong in setting up the correct criteria and the entire study could be affected by this initial mistake. Of course, this is not the only thing you have to worry about: in this kind of analysis many pitfalls are hidden. In the following sections, common errors are described, along with useful tips to avoid making such mistakes. Table 1: Risk Tolerability Criteria Risk Zone Risk Tolerability Criteria (events/year) Low Risk Area Risk < 1·10-6 Medium Risk Area 1·10-6 ≤ Risk < 1·10-4 Medium-high Risk Area 1·10-4 ≤ Risk < 1·10-3 High Risk Area Risk ≥ 1·10-3 2. The Seven Deadly Sins The “Seven Deadly Sins” are common mistakes that can affect the success of a structured and tricky study such as QRA (indeed, the adjective “deadly” is not used here by chance); they refer to the consequences that we could have in a plant (e.g., in a pharmaceutical plant or in a refinery) if the risk assessment and the incidental scenarios are not evaluated according to the best practices. 187 First, the list of such errors is shown below. Then, each one of them is discussed one by one: • Carrying out a risk assessment to attempt to justify a decision that has already been made; • Inappropriate use of data; • Inappropriate definition of a representative sample of events; • Inappropriate use of risk criteria; • Inappropriate use of statistics; • Using "reverse ALARP" arguments; • Not doing anything with the results of the assessment. 2.1 Carrying out a risk assessment to attempt to justify a decision that has already been made This can happen when a plant decides to build a new area. Instead of evaluating if this new construction may carry new risks for the people, they complete the project and only at the end they start asking themselves if this could affect the risk analysis of the old plant, increasing or not the overall risk (and if this risk is acceptable or not). Another typical situation is when, due to delay in scheduling, the risk assessment is developed when equipment or vessels have already been ordered or bought. This dodgy decision brings many drawbacks: first, the results of the QRA could be influenced by “external pressure” since a lot of money has been already invested. This will involve the people that will develop the study, and all the stakeholders, as everyone will try to pass the buck (and relative costs) to others, such as main client, EPC company, supplier of packages, and so on. Secondly, if the QRA results highlight the need of some changes, it could be much trickier to modify the new plant, once built, instead of in the design phase. In a similar way, it is never a good decision to carry out a Quantitative Risk Assessment for avoiding the implementation of an element of normal good practices. A QRA to show that the element is not worth should not be accomplished. To avoid this situation, it is possible to anticipate the risk assessment as early as possible; this can be achieved using different tools according to the stage (and progress) of the project, as described below: Preliminary Risk Assessment, or PRA and HAZID, or Hazard Identification and FERA, or Fire and Explosion Risk Assessment. Preliminary Risk Assessment (PRA) Preliminary risk assessment is developed in the “conceptual design” stage, which means that the main goals of the project, including “safe design philosophy”, have been defined, with approximate vessel, chemicals, and mass flow estimate, but with no detailed process Flow diagram or heat and material balance defined yet. It’s a high-level brainstorming, starting from an agreed checklist, covering the main safety issues, such as, but not limited to: • Substances involved in the project, and availability of all needed process safety information, such as flammability parameters, dust explosion parameters, decomposition temperature, runaway onset temperature (this point will be further analysed); • Storage of dangerous substances, quantities, means of storage, main safety measures foreseen (e.g., floating roof tanks or nitrogen blanketing for highly flammable substances); • Main unit operations (distillation, reaction, cracking, separation, etc.) and relevant operational conditions, such as high temperature, high pressure, runaway reaction risk, decomposition risk, etc.; • Transportation and handling of dangerous substances, such as tank truck, rail tanker, big bags loading and unloading; • Facility siting, i.e., position of equipment, considering potential interference risk from existing plant to new units and vice-versa, and potential impact on occupied building or sensitive targets, including facilities outside the plant (risk to public). The Preliminary Risk Assessment can include a semiquantitative risk estimate, using a risk matrix. A striking example: during a QRA, when equipment had already been bought, we highlighted that a single piece of equipment, a high-volume, high-pressure separator, would impact the nearby vessels and manned areas in such a strong way that any evacuation would have been impossible. Therefore, the project had to be suspended, and major engineering changes had to be adopted, with 6 months delay (and enormous additional cost). This criticality would have been easily identified in a well-done preliminary risk assessment. HAZID and FERA HAZID, or Hazard Identification, is a methodology that starts from a structured check-list, covering different subjects (such as natural events, climate, environmental and process safety issues, external factors, etc.). It is typically developed during Basic Design, that is, after quantified Process Flow Diagrams and Heat and Material Balance are available (but P&IDs are not ready yet). 188 HAZID is competed with a Fire and Explosion risk assessment, that is a simplified QRA that can anticipate, with precision limited to order of magnitude, the results of a detailed QRA. This could give important feedback to the project, allowing modification and inclusion of additional safety barriers when there is still the possibility to modify the unit and relevant pieces of equipment. 2.2 Inappropriate use of data In Process Safety, numbers and data are always important. In a QRA, using data in a wrong way or using wrong data could completely modify the results of the study. One brief example might clarify this concept: considering the wrong vapour pressure for a fluid could lead not to evaluate a dispersion scenario, losing the possibility to protect the personnel from it (or vice versa, a scenario that is not credible may be considered and money could be wasted for further unnecessary protection). The lack of adequate Process Safety Information is unfortunately quite common (a minimum list of required PSI is reported below): • Physical and chemical behaviour of substances and reaction mixture; • Dust explosion properties (MIE, MIT, LIT, Kst, Pmax, etc.); • P&IDs, C&E Matrix, H&M balance, etc. That information, as anticipated, can have a very important impact on the results of the risk assessment, leading to impactful mistakes. The example given below shows how bad data can lead to huge mistakes: in this real case, taken from a QRA used for decision making, the vapour pressure of formaldehyde was taken from MSDS without checking the validity of the data (Table 2). Table 2: Vapour pressure of Formaldehyde from different MSDS Physical property MSDS 1 ILO ICSC 37% SOLUTION Vapour Pressure [Pa] @ 20 °C 133 2,400 Using data from MSDS would lead to almost no toxic dispersion, and no risk nearby the release. Using data from ILO would lead to toxic dispersion, and to identify a dangerous area. Another striking example come from the permitting process of a plant: authorities asked to evaluate scenarios with a frequency as low as 1·10-12 events/year. If we compare this request to the known age of the universe, 13.787 109, such events would happen in a lag time roughly around one hundred times the universe history. Figure 1: Estimated age of the universe = 13.787 109 years 2.3 Inappropriate definition of a representative sample of events The errors that fall into this category are, for example, not considering all the scenarios that emerged from the HAZOP or not considering the scenarios deriving from domino effects. This could lead to an underestimation of the risk. 189 It is very common to find QRA (and Seveso Safety reports too) based only on statistical leakages and ruptures from pipes and vessels, not considering any process upset that could cause a release. The frequency of process deviations, according to the safeguards available or not available, could be many orders of magnitude greater that the frequency of generic pipe or flange leakage. In a specific case, the QRA of a Refinery was based on the assumption (wrong in principle) that the most dangerous part was LPG handling and storage. According to this principle, the following scenario were included: • Generic failures from vessels • Generic failures from pipes in the LPG station • Generic failures from pumps & compressors • Generic failures from arms/hoses • BLEVE Despite there is no doubt that a BLEVE is a very severe scenario, that approach is not covering other situations that could lead to impactful events, such as: • Operational failures: what about overfilling? • Domino effect: what about risk from the refinery? • Small leakages near plant boundaries • Leakages, ruptures, explosions in Refinery process area • Runaway reaction risk, that could have a much wider impact • Toxic dispersion: Hydrogen Fluoride, Ammonia, Hydrogen Sulphide, etc. 2.4 Inappropriate use of risk criteria Assessing a large group of people in terms of individual risk or targets may lead to a wrong risk assessment. Indeed, it is very different if we are analysing the scenario in terms of local risk (defined as the expected frequency of a given damage for a person permanently positioned in a specific point, with no possibility of being protected or evacuated), in terms of individual risk (defined as the expected frequency of a given damage for a person positioned in a specific point for a certain fraction of the total time, also considering its location with respect to the source of the damage and any protections present) or in terms of social risk (defined as the number of people who may be affected by a given damage, considering the number of people and the fraction of the total time that such people stay in one point, as well as their location with respect to the source of the damage and any protections present). Another inappropriate use of risk criteria may be the use of too permissive (or too stringent) ones that could impact on the final results of the study. In the following example, only individual risk was selected for decision making. But the impacted areas were very different, including sensitive elements with high occupancy. In this specific case individual risk cannot be taken as a reference, opting for Social Risk. Compared to the population exposed, the IR value of 8.93·10-6 for the residential area-1 cannot be considered acceptable, despite the lower frequency (Table 3). Table 3: Individual Risk vs impacted people Impacted Area Population Individual Risk Residential area-1 5,000 8.93·10-6 Public buildings 200 2.09·10-4 Residential area-2 100 1.89·10-5 2.5 Inappropriate use of statistics When a Probability of Failure on Demand is required (e.g., for a Fault Tree Analysis), the data of interest are often extracted from generic libraries (e.g., TNO Red Book, 2005), but that data could not take into account specific situation of our case (lack of maintenance, high corrosivity field, etc.); it is always a better option, if possible, to use specific data. Of course, specific data are not always available, in this case it is important to enquire whether the numbers that are going to be used are reliable or not. Table 4 shows how a wrong selection of data could lead to different results. Choosing the lower bound value instead of the upper bound would lead to a 2 order of magnitude difference in frequency of the event. 190 Table 4: Failure rate data (TNO Red Book, 2005) Element Lower bound (h-1) Median value (h-1) Upper bound (h-1) Pressure sensor, fails to operate 6.5·10-8 3.2·10-7 1.6·10-6 Safety valve, pilot operated, fails to open 1.0·10-8 1.3·10-7 1.8·10-6 Pipe D < 75 mm Leakage 2.1 10-11 2.1 10-10 2.1 10-9 2.6 Inappropriate use of results It shall be never forgotten that the QRA methodology, despites all precision efforts, is based on statistical data and assumptions and shall not be considered as the final decision-making tool. An example of how wrong use of results can lead to dangerous mistakes is shown in Figure 2. In this case, position and shape of a commercial building were set exactly on a risk level boundary coming from a QRA. With this example, it is clear that the boundary can by no means be understood as a physical limit, beyond which any area can be considered safe. Being an area open to public, an additional safety distance should have been kept in consideration when positioning the new building Figure 2: Example of wrong use of QRA for the design of a commercial building 2.7 Not doing anything with the results of the assessment In this paper many common mistakes that can be done during a QRA have been investigated, but the biggest mistake is doing nothing after the results obtained. Not only money and time have been wasted, but also a hazardous situations that we are aware of are being ignored, committing an error that transcends technique to fall back into ethics and morals. 3. Conclusions This paper serves as a window to understand the pitfalls which can be encountered during a Quantitative Risk Assessment. In conclusion, the “Five Commandments” that emerge from this study can be summed up as: • The scope of QRA, the appropriate approach and the level of detail shall be clearly defined; • All relevant actors shall be involved in a QRA; • The target of the risk shall be clearly identified; • It shall be known if Individual Risk or Societal Risk or both need to be considered; • QRA shall not be used to avoid applying engineering standards. It is a well-known fact that consequence modelling (and here there is the need to remember that QRA is based on consequence modelling) is not an exact science, there is still a 20 % to 40 % of uncertainty on the results. Moreover, results can be affected by several other factors (meteorological conditions, initial conditions). All of this points to the fact that you must keep in mind what you are doing and why. A safety factor should always be considered to avoid unwanted situations and strong caution shall be taken in decision making based on QRA. 191 Acronyms and abbreviations Table 5: Acronyms and Abbreviations Acronym Definition ALARP As Low As Reasonably Possible: the risk has been reduced to the feasible lowest level. This includes including cost/ benefit analysis BLEVE Boiling Liquid Expanding Vapour Explosion C&E Matrix Cause and Effects Matrix EPC Engineering, procurement and construction FERA Fire and Explosion Risk Assessment H&M balance Heat and Material Balance HAZID Hazard Identification HAZOP Hazard and Operability Risk Assessment ILO International Labour Organization ICSC International Chemical Safety Cards IR Individual Risk IRPA Individual Risk per Annum Kst Dust deflagration index, parameter used to quantify the severity of a dust explosion, by measuring the pressure increase rate LIT Layer ignition Temperature LPG Liquefied Petroleum Gas MIE Minimum Ignition Energy MIT Minimum Ignition Temperature MSDS Material Safety Data Sheet P&ID Piping and Instrumentation Diagram PFD Process Flow Diagram Pmax Maximum pressure that dust explosion can cause in a confined space PSI Process Safety Information QRA Quantitative Risk Assessment SR Societal Risk References Aven, T., Renn, O., 2009. On risk defined as an event where the outcome is uncertain. J. Risk Res. 12, 1e11. Cacuci, D.G., 2003. Sensitivity and Uncertainty Analysis. In: Theory, vol. 1. Chapman & Hall/CRC, Washington DC. CPR16, 1989. Methods for the Determination of Possible Damage (Green Book). The Netherland, The Hague. 2005, Methods For Determining and Processing Probabilities (TNO Red Book), The Netherland, The Hague ESA/Planck Collaboration (17 July 2018). "Measurements of the Hubble constant". sci.esa.int. European Space Agency. Flage, R., Aven, T., 2009. Expressing and communicating uncertainty in relation to quantitative risk analysis (QRA). Reliab. Risk Anal. Theory Appl. 2 (13), 9e18. Health and Safety Executive, 2012. Failure Rate and Event Data for Use within Risk Assessments. On-line report. http://www.hse.gov.uk/landuseplanning/failurerates. Hellas, M.S., Chaib, R. and Verzea, I. (2019). Artificial intelligence treating the problem of uncertainty in quantitative risk analysis (QRA). Journal of Engineering, Design and Technology, 18(1), pp.40–54. doi:https://doi.org/10.1108/jedt-03-2019-0057. Lees, F.P., 1996. Loss Prevention in the Process Industries. Elsevier, London. Milazzo, M., Vianello, c., Maschio, G., Uncertainties in QRA: Analysis of losses of containment from piping and implications on risk prevention and mitigation, Journal of Loss Prevention in the Process Industries 36 (2015) 98e107 Rijnmond Public Authority, 1982. Risk Analysis of Six Potentially Hazardous Industrial Objects in the Rijnmond Area, a Pilot Study. Reidel, Dordrecht. 192 99gotti.pdf Quantitative Risk Assessment: Best Practices and Limitations