untitled CCHHEEMMIICCAALL EENNGGIINNEEEERRIINNGG TTRRAANNSSAACCTTIIOONNSS VOL. 33, 2013 A publication of The Italian Association of Chemical Engineering Online at: www.aidic.it/cet Guest Editors: Enrico Zio, Piero Baraldi Copyright © 2013, AIDIC Servizi S.r.l., ISBN 978-88-95608-24-2; ISSN 1974-9791 System PHM Algorithm Maturation Jean-Rémi Massé*,a, Ouadie Hmadb,c, Pierre Beauseroyc, Edith Grall-Maësc, Agnès Mathevetb aSafran Snecma, rond point René Ravaud 77550, Moissy Cramayel, France bSafran Engineering Services, rond point René Ravaud 77550, Moissy Cramayel, France cUMR STMR – LM2S : ICD – Université de Technologie de Troyes, 12 rue Marie Curie CS 42060 10004, Troyes, France jean-remi.masse@snecma.fr The maturation of PHM functions is focused on two Key Performance Indicators (KPI): The NFF, No Fault Found ratio, i.e. probability of a fault detection to be unverified, and the Probability Of Detection POD The estimation of the second KPI can be done by counting the global abnormality threshold trespassing when each different kind of degradation is simulated. The first KPI is rather a requirement. It induces a constraint on the threshold position in terms of probability of threshold trespassing with no degradation. Typically, for a probability of fault occurrence of 10-7, a specified NFF ratio of 1 %, and an expected POD of 90 %, the order of magnitude of probability of threshold trespassing with no degradation should be 10-9. The estimation of such extreme level of probability needs some parametric adjustment of the distribution of the global abnormality score with no degradation. Two PHM functions are considered as case studies: Turbofan engine start capability (ESC) and turbofan engine lubrication oil consumption (EOC). In ESC the global abnormality score is a norm of a vector of specific abnormality scores. Some specific scores are devoted to starter air supply. In EOC, the global abnormality score is the consumption estimation. To reach acceptable POD at the specified NFF ratio three improvements are needed for ESC: – Adjust the abnormality decision threshold according to each candidate degradation using extreme value quantiles on the global abnormality score distribution – Average the global abnormality score on five consecutive starts – Learn the regression relations specifically on each engine. The first improvement is a novelty. It is successfully applied to both ESC and EOC functions. It is generic to all airborne system PHM functions based on abnormality scores. 1. Introduction A maturation process of PHM functions is followed. First, generic sub functions of system PHM are considered. This is illustrated on two use cases. Then, two Key Performance Indicators (KPI) are chosen according to the considered sub functions and to airline business models. The estimation of these KPI is defined. To reach acceptable levels of KPI on the use cases, some improvements of the functions are proposed. 2. Case studies 2.1 General PHM functions are usually represented as OSA-CBM architecture (MIMOSA, 1998). Table 1 typically represents such an architectural architecture applied to a system PHM (Lacaille, 2010). The specificities of a given PHM function are restricted to level #1 Data acquisition and level #2 Data manipulation through indicators and context parameters. The next levels are, in general, common and may DOI: 10.3303/CET1333048 Please cite this article as: Masse J.R., Hmad O., Beauseroy P., Grall-Maes E., Mathevet A., 2013, System phm algorithm maturation, Chemical Engineering Transactions, 33, 283-288 DOI: 10.3303/CET1333048 283 have à learning mode in addition to the basic PHM mode. Such learning modes are tagged in table 1 with an asterisk* or two**. Table 1: Typical system PHM OSA-CBM summary *Has a learning mode; **Is a learning mode #1 DATA ACQUISITION Acquire sensor and system data #2 DATA MANIPULATION Extract the indicators - Acquire the context parameters #3 STATE DETECTION Build the prediction model** - Score the prediction errors #4 HEALTH ASSESSMENT Learn reference patterns (syndromes)** - Cluster according to references - Isolate the potentially degraded LRU(s) or module(s) through Bayesian calculation - Isolate the potentially degraded LRU(s) or module(s) through fault isolation manual on failure condition precursors Score global abnormality* Adjust the abnormality decision thresholds** Detect abnormality #5 PROGNOSTIC ASSESSMENT Predict the probability of maximal degradation before failure within a given operational time #6 ADVISORY GENERATION Establish a global diagnosis and prognosis merging other health monitoring means. The PHM function section considered for maturation is part of level # 4 Health Assessment. It is tagged in table 1 with bold characters. The abnormality detection function considered here is based on global abnormality score threshold trespassing. On this general basis, two specific use cases are considered: – - Engine start capability, ESC (Ausloos et al. 2010 – - Engine oil consumption, EOC (Demaison et al. 2010). 2.2 Engine Start Capability function Engine start capability function, ESC, relies on a set of indicators (Figure 1) extracted during start sequence, sensitive to no start precursors. Figure 1: Engine start capability, ESC, indicators Some indicators are devoted to air supply degradations. Examples are the duration of phase 1 of the start, from starter air valve open command to ignition HP rotor speed, or, the average acceleration of HP rotor during phase 1. These indicators are sensitive to air starter valve slow opening. Such degradation is a precursor of valve stuck closed, which is a typical origin of no start. Some indicators are devoted to fuel metering degradations. Examples are phase 2 duration, from ignition to starter cut speed, or, Exhaust Gas Temperature slope during phase 2. Prediction error scores are centered and reduced residues between expected values of indicators and observed values of indicators. The expected values of indicators are estimations, through regression relations, using as inputs the other indicators and context parameters such as lubrication oil temperature at start. Referring to table 1, this is the basic PHM mode of “#3 – State detection - Score the prediction errors” The regression relations are learnt on start records with no degradations. The means and standard deviations of the residues needed for centering and reduction are learnt on the same records. Referring to table 1, this is the learning mode of “#3 – State detection - Build the prediction model”. The global abnormality score, ||Z||2, is the squared Mahalanobis norm of the vector, ε, of prediction error scores: ||Z||2 = ε T .ρ -1 . ε (1) Referring to Table 1, this is the basic PHM mode of “#4 Health assessment – Score global abnormality” 284 The correlation matrix, ρ, is also learnt on the same records with no degradations. This is the learning mode of “#4 Health assessment – Score global abnormality” 2.3 Engine Oil Consumption function Engine oil consumption function, EOC, relies on oil level extractions at taxi phase. The oil levels are captured at constant ground idle speed when the switch based level indication changes. A small correction of level is done according to temperature. Figure 2: Engine oil consumption, EOC, oil level captures The global abnormality score is the daily weekly or monthly consumption estimation on a daily increment. This relies on regressions on the oil levels versus flight time taking into account the oil fills. Referring to table 1, this is the basic PHM mode of “#4 Health assessment – Score global abnormality”. Unlike ESC, for EOC, this item has no learning mode. 3. P(No degradation|Detection) 3.1 Definition As seen previously, the PHM function section considered for maturation in § 2.1 needs to be assessed and quantified. This is addressed through two Key Performance Indicators, KPI. In commercial aeronautics, the major KPI for abnormality detection is an extension of the so called “No Fault Found” ratio, NFF. The original NFF ratio refers to failure detections which are false. The extended NFF ratio, considered in PHM, refers to degradation detections which are false. The degradations considered in PHM are failure precursors. The NFF ratio is defined as P(No degradation| Detection). The line maintenance wishes to avoid “No fault founds”. For instance, a false detection of fuel metering degradation may lead to hydro mechanical unit replacement. This is eight hours manpower. Therefore, NFF ratios should not exceed 5% at line maintenance stage. High NFF ratios would kill PHM. 3.2 Counterpart A second KPI CTQ CTB is the well known Probability Of Detection, POD. The POD is defined as P(Detection |Degradation). For line maintenance the POD should be as high as possible under the constraint of low NFF ratio. For operations management, the abnormality detection should occur as soon as possible. For operations, NFF ratio is not as critical as for line maintenance. The popular Probability of False Alarm, PFA, P(Detection |No degradation), is linked to the two KPI CTQ CTB by the following relation: (2) Figure 3: Diagram of PFA and POD for a decision based on threshold trespassing 285 With the type of decision considered, based on threshold trespassing, P(Detection| No degradation) is the probability of the global abnormality score with no degradation being higher than the abnormality decision threshold (Figure 3). For a typical P(Degradation) of 10-6 or 10-7 per decision, an expected NFF rate, P(No degradation| Detection), of 5 % and a POD, P(Detection| Degradation) of 90%, the PFA, P(Detection| No degradation), should be 5.10-8 or 5.10-9 (Formula 2). 3.3 Estimation The estimation of POD, P(Detection |Degradation), can be done by counting the global abnormality threshold trespassing when each different kind of degradation is simulated. The degradations are simulated rather than observed. The premise degradations typically occur with a probability of 10-6 or 10E-7 per engine flight. It would be necessary to cumulate more than 27.105 or 27 million flights to observe this event at least thirty times with a probability of 90 %. The simulations are based on transformations of the degradation indicators values with no degradation. Such transformations are characterized by the degradation considered and the degradation intensity. Strong intensity corresponds to ultimate degradation level just before failure. This concerns line maintenance. At this level P( No degradation | Detection) should be less than 5%. Weak or mean intensity correspond to initiation of the degradation. This concerns operations. At this level P(Detection|Degradation) should be favored even though P(No degradation|Detection) reaches up to 50%. In ESC, simulations of degradations related to starter air supply were learnt with a phase 1 simulator based on torques balance. Simulations of degradations related to fuel metering were learnt on start tests records including fuel metering biases. In EOC, the over consumptions are simulated by drifts in mean of the consumption estimations. The estimation of the NFF ratio may be done through the following formula: (3) (4) P(Degradation) may be known through FMEA or field experience. P(No degradation) = 1-P(Degradation) is close to 1. As seen previously, the order of magnitude of P(Detection| No degradation) should be typically 5.10-8 or 5.10-9 Figure 4: Observed and adjusted cumulative distribution function of ESC global abnormality score with no degradation. As seen previously, P(Detection| No degradation) is the probability of the global abnormality score with no degradation being higher than the decision threshold (Figure 3). The estimation of such extreme level of probability needs some parametric adjustment of the distribution of the global abnormality score with no degradation. This requires modeling correctly the distribution tail of the global abnormality score with no degradation. It appears that the adjusted Gamma and Normal distributions do not fit well the observed distribution of the global abnormality score. Conversely, according to figure 4, the multi parametric adjustment obtained with Parzen estimator (Silverman, 1991) fits well the observed distribution (Hmad et al., 2011). 286 4. Abnormality decision thresholds adjustment 4.1 Methodology The first improvement proposed to reach an acceptable level of P( No degradation |Detection) is to adjust the abnormality decision threshold on the global abnormality score with no degradation. As seen previously, P(Detection| No degradation) is the probability of the global abnormality score with no degradation being higher than the decision threshold. Conversely, if the expected value of P(Detection| No degradation) is known, the adjustment of decision threshold may take advantage of the accurate Parzen fit. As a first guess of P(Detection| No degradation), formula 2 may be used with a prior assumption of P(Detection| Degradation) being close to 100 %. In a second iteration with the prior threshold, a more realistic estimation may be done for P(Detection| Degradation) (Hmad et al., 2012). 4.2 Application to ESC This methodology is applied to ESC. A global abnormality score distribution is observed on starts with no degradations. Figure 5: Impact of the fit quality on decision threshold. Figure 5 shows the need to check the distribution fits. Figure 6 shows the initial performances of ESC with the Parzen threshold adjustment. Figure 6: Prior abnormality decision threshold and global abnormality score distributions with three starter air valve degradation intensities. Figure 7: Improvement of the performances with regression relations learnt on each specific engine and moving average of the global abnormality score 287 Only 20 % of the strong degradations are detected. This is not acceptable for line maintenance. None of the weak or mean degradations are detected. This is not acceptable for operations. The performances are improved with a moving average on the global abnormality score and the regression relations learnt specifically on each engine (Figure 7). This improves the accuracy of the indicator predictions. The performances become now acceptable for both line maintenance and operations. 4.3 Application to EOC The methodology of threshold adjustment is now applied to Engine Oil Consumption. Figure 8: Prior abnormality decision threshold and daily consumption distributions with two over consumption levels For this PHM function, almost all mean and all strong over consumptions are detected. 5. Conclusion The PHM sub function considered is abnormality detection based on threshold trespassing by a global abnormality score. For such function the No Fault Found ratio, NFF, P(No degradation| Detection) is relevant for line maintenance. The estimation of this performance indicator supposes to fit accurately the distribution of the global abnormality score with no degradation. To reach acceptable probabilities of detection at the specified NFF ratio three improvements are proposed for Engine Start Capability PHM function: - Abnormality decision threshold adjusted using extreme value quantiles on the global abnormality score distribution - Moving average of the global abnormality score - Regression relations learnt specifically on each engine. The first improvement is a novelty. It is successfully applied to both use cases considered. It is generic to all airborne system PHM functions based on abnormality scores. It is now being extended to other abnormality decision functions. References Ausloos A., Grall E., Beauseroy P. Masse J.R., 2009, Estimation of monitoring indicators using regression methods - Application to turbofan starting phase, ESREL conference, Chapter 28. Ausloos A., Mouton P., Massé J -R., Aurousseau C. A. Flandrois X., 2010, Method for monitoring the health status of devices that affect the starting capability of a jet engine. Patent # WO 2010/092080 A1. Demaison F., Massot G., Massé J –R., Flandrois X., Hmad O., Ricordeau J., 2010, Turbofan lubrication oil consumption follow up method. Patent # 1H105790 1093FR. Hmad O., Masse J.R., Grall E., Beauseroy P., Mathevet A., 2011, A comparison of distribution estimators used to determine a degradation decision threshold for very low first order error, ESREL conference. 346 – 352. Hmad O., Massé J -R., Grall-Maes E., Beauseroy P., Boulet X., 2012, Decision threshold adjustment method. Patent # FA 764429 FR 1254506. Lacaille J., 2010, Turbofan failures identification. Patent # FR2939924 A1. Lacaille J., 2010, "Turbofan monitoring data standardization. Patent # FR2939928 A1. MIMOSA, 1998. Open Systems Architecture for Condition-Based Maintenance, OSA-CBM v3.1 standard. Silverman, B. W,, 1986, Density Estimation for Statistics and Data Analysis. Chapman and Hall London, UK. 288