Microsoft Word - 1.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 77, 2019 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Genserik Reniers, Bruno Fabiano Copyright © 2019, AIDIC Servizi S.r.l. ISBN 978-88-95608-74-7; ISSN 2283-9216 CEQAT-DGHS Interlaboratory Test Programme for Chemical Safety - Need of Test Methods Validation Peter Lütha,*, Kirstin Frostb, Lutz Kurtha, Marcus Malowa, Heike Michael-Schulza, Martin Schmidta, Petra Schultea, Steffen Uhligb, Sabine Zakelc aBundesanstalt für Materialforschung und –prüfung (BAM), Unter den Eichen 87, 12205 Berlin, Germany bQuoData GmbH, Prellerstraße 14, 01309 Dresden, Germany cPhysikalisch-Technische Bundesanstalt (PTB), Bundesallee 100, 38116 Braunschweig, Germany peter.lueth@bam.de Various standardised test methods are used by laboratories to determine the hazardous properties and safety characteristics of chemical substances and mixtures, e.g. test methods published by the European Union (2008) and by the United Nations (2015). The test results are the basis (amongst others) for the correct classification and labelling of chemicals as hazardous according to GHS/CLP or as dangerous goods according to Transport Regulations. The REACH Regulation obliges all manufacturers and importers of chemicals to identify and manage risks associated with the substances they manufacture and market. Via the eChemPortal (www.echemportal.org) many data from the REACH registration dossiers are available. However, the quality and correctness of the information remains in the responsibility of the data submitter. Unfortunately, the quality of information on physicochemical properties is not always adequate, so the quality and adequacy of the data submitted shall be improved. This paper focusses on the quality of test results obtained by laboratories and the validation of the test methods used e.g. for classification of dangerous goods and indicates recent developments as well as steps to be taken for further improvement. In particular, the importance of interlaboratory tests and the need to validate test methods are presented in this publication. Interlaboratory tests play a decisive role in assessing the reliability of laboratory test results. Participation in interlaboratory tests is not only a crucial element of the quality assurance of laboratories; as such it is explicitly recommended in ISO/IEC 17025, ISO (2017). In addition, interlaboratory tests are also used to improve and validate test methods and can be used for the determination of the measurement uncertainty, Hässelbarth (2004), ISO (2004). This paper specially addresses interlaboratory tests for the validation of test methods performed by CEQAT- DGHS (Centre for quality assurance for testing of dangerous goods and hazardous substances, www.ceqat- dghs.bam.de) and outlines typical results and general conclusions and steps to be taken to guarantee that laboratory test results are fit for the purpose and of high quality. 1. Introduction An explosion in a chemical plant or a fire on a dangerous goods vessel - the reason for such accidents can be numerous. Prevention starts in the laboratory where chemicals are tested for their hazardous properties in order to be able to assess the risks during storage, transport and use. For this purpose, test methods have been developed and published, e.g. test methods by the European Union (2008) and by the United Nations (2015). Nowadays, they are applied globally. Safety experts, manufacturers, suppliers, importers, employers and consumers must be able to rely on the validity of safety-related test methods and on correct test results and assessments in the laboratory. Therefore, various internal and external quality assurance measures of the laboratory are required. For that purpose, interlaboratory tests play an important role. DOI: 10.3303/CET1977001 Paper Received: 16 January 2019; Revised: 4 May 2019; Accepted: 3 July 2019 Please cite this article as: Lueth P., Frost K., Kurth L., Malow M., Michael-Schulz H., Schmidt M., Schulte P., Uhlig S., Zakel S., 2019, CEQAT- DGHS Interlaboratory Test Programme for Chemical Safety - Need of Test Methods Validation, Chemical Engineering Transactions, 77, 1-6 DOI:10.3303/CET1977001 1 2. CEQAT-DGHS Interlaboratory test programme The two German Federal Authorities: Bundesanstalt für Materialforschung und -prüfung (BAM) and Physikalisch-Technische Bundesanstalt (PTB), based in Berlin and Braunschweig, have extensive experience in the field of testing and assessment of physical hazards of chemicals. Based on this experience and due to various changes in national and international chemicals legislation (e.g. GHS, REACH, CLP etc.) and questions on causes of accidents and related questions on the quality of test results, BAM founded the competence centre CEQAT-DGHS (Centre for quality assurance for testing of dangerous goods and hazardous substances, www.ceqat-dghs.bam.de) in 2007. As part of this competence centre, an interlaboratory test programme for test methods for the determination of physical hazards and of safety-relevant substance data has been set-up in 2007 and pursued since then. The aim is to obtain information on the reliability of test results obtained by laboratories and in particular on the quality / validity of the test method(s). Performing high-quality, meaningful interlaboratory tests for the purposes of method development and validation is labour intensive and time consuming and requires not only experience in the practical application of the test method but also special expertise in the design of the respective interlaboratory tests and statistical evaluation. In order to achieve best possible realization of such interlaboratory tests, BAM and PTB operate the CEQAT-DGHS interlaboratory test programme together with QuoData GmbH. QuoData GmbH is specialised in planning, performing and evaluation of interlaboratory tests. While BAM and PTB are responsible for the metrological part and the selection and production of the interlaboratory test samples, QuoData GmbH is responsible for the data management as well as for the statistical procedures and analyses by means of statistical software PROLab PlusTM and the QuoData’s online platform for proficiency testing and method validation studies. An overview of the test methods currently listed in the CEQAT-DGHS interlaboratory test programme is shown in Figure 1. This figure also indicates interlaboratory tests already performed by CEQAT-DGHS as well as the number of laboratories interested in these interlaboratory tests. Figure 1: Test methods currently listed in the CEQAT-DGHS interlaboratory test programme, number of laboratories with interest in participation in CEQAT-DGHS interlaboratory tests and interlaboratory tests performed by CEQAT-DGHS since 2007, RR = round robin test / interlaboratory test Since the founding of CEQAT-DGHS in 2007, the number of laboratories interested in the interlaboratory tests has steadily increased to a current total of about 90. The minimum number of participating laboratories required for meaningful interlaboratory tests (i.e. for statistical reasons) is now met in almost all test methods. Hence, interlaboratory tests could now be carried out for almost all test methods listed in Figure 1. If required, the CEQAT-DGHS interlaboratory test programme could be extended to further test methods within the scope of the work of CEQAT-DGHS, i.e. the determination of the physical hazards and of safety-relevant substance data. 2 2.1 Data check (inspection upon receipt of the data from the laboratory) A sound database is a prerequisite for a meaningful interlaboratory test evaluation and for a correct validation of test methods. All CEQAT-DGHS interlaboratory tests have shown that a comprehensive review of the test results sent by each laboratory is mandatory. The check of the data submitted by the laboratories includes the following items, Frost et al. (2016): • Completeness of the data, check e.g. for missing data, • Conformity - check for irregular deviations from the testing method and / or the interlaboratory test instructions, • Plausibility - check for obvious incorrectness of the submitted data, e.g. distorted data, • Consistency - check the correctness of the values in the submitted data input form, e.g. by comparing to raw data. The data check should be carried out immediately after receipt of the data from the laboratory and before starting the statistical analysis. The data check should be performed by different experts independently. If possible, faulty data should be corrected after consultation and in mutual agreement with the respective laboratory. As an example, a table showing typical deficiencies in the test data obtained from laboratories is given in Figure 2. Figure 2: Results of the data check in the course of the interlaboratory test 2016 for the validation of test method DIN EN 15188:2007, [red field/1] = relevant deviations from the test method and / or the interlaboratory test instruction, [blue field/0] = no deviation, [grey field/”xx”] = anonymised laboratory ID- number, Frost et al. (2016) A careful data check is mandatory for interlaboratory tests for methods validation and must not be neglected to minimise the impact of deficiencies in the submitted data on test conclusions, even though it is very labour intensive and time-consuming. 2.2 Typical results of the CEQAT-DGHS interlaboratory tests As shown in Figure 1, various interlaboratory tests for different test methods have been performed by BAM and PTB within the frame of CEQAT-DGHS. Some typical distributions of the laboratory results are shown in Figure 3. xx xx xx xx xx xx xx xx xx xx xx xx Planned test period is not met: 1st tested sample 9 0 1 1 1 0 1 1 0 1 1 1 1 Planned test period is not met: 2nd tested sample 5 0 0 0 0 1 0 1 0 1 1 1 0 Planned test period is not met: RS 1 1 0 0 0 0 0 0 0 0 0 0 1 0 Planned test period is not met: RS 2 5 1 0 0 0 1 0 0 1 0 0 1 1 Dif f erent laboratory assistants w ithin the tests 8 1 1 0 1 1 0 1 1 0 1 0 1 Bulk density of the sample not correct calculated 9 0 0 1 1 1 1 1 0 1 1 1 1 Bulk density of the sample does not comply w ith the requirements 9 1 1 0 1 1 0 1 1 0 1 1 1 Variation of bulk density > 2 % (related to mean value of all tests) 9 1 1 0 1 1 0 1 1 0 1 1 1 Variation of bulk density > 5 % (related to mean value of all tests) 5 1 1 0 0 1 0 0 0 0 0 1 1 Wrong distance of the sensor T1 and T2 to the sample basket 3 1 0 0 0 0 0 0 0 0 1 1 0 Dif f erence(s) betw een sensor T1 und T2 > 2 K 3 1 0 0 0 0 0 1 0 0 0 1 0 Dif f erence(s) betw een "go" and "no go" temperature > 2 K 11 1 1 1 1 1 0 1 1 1 1 1 1 Dif f erence(s) betw een "go" and "no go" temperature > 2.5 K 6 0 0 1 1 1 0 1 0 0 0 1 1 Dif f erence(s) betw een "go" and "no go" temperature > 3 K 2 0 0 0 1 0 0 0 0 0 0 0 1 Missing calibration (sensor or w hole measuring chain) 0 0 0 0 0 0 0 0 0 0 0 0 0 Missing calibration of the w hole temperature test chain 2 0 1 0 0 0 0 1 0 0 0 0 0 Missing calibration document 2 0 0 0 0 0 0 1 0 0 0 0 1 Sum 8 7 4 8 9 2 11 5 4 8 12 11 Laboratory Sum Param ete r 3 Figure 3: Typical distributions of results of CEQAT-DGHS interlaboratory tests (3 examples) - curves of the kernel density estimation of the results submitted by the different participating laboratories, Antoni et al. (2011), Lüth, Kurth (2013), Lüth et al. (2014), RR = round robin test / interlaboratory test Significant differences between the results of the participating laboratories were observed in all CEQAT-DGHS interlaboratory tests (see interlaboratory test reports: Brandes et al. (2017), Frost et al. (2016), Lüth et al. (2014), Lüth, Kurth (2013), Kunath et al. (2013), Kunath et al. (2011), Antoni et al. (2011), Antoni et al. (2010). The interlaboratory tests have shown that, for some test methods, the results of the participating laboratories varied to an intolerable degree, even if the laboratories had performed the tests in accordance with the requirements of the currently published test method. The different test results of the laboratories partly would have resulted in different classifications of the chemicals with regard to GHS/CLP and dangerous goods regulations. Evaluation of the different interlaboratory tests showed that the scattering of the test results has various reasons. The multi-peak, so-called multimodal distributions of the test results shown in Figure 3 clearly indicate that not only random errors of the laboratories (with measurement results that would follow a normal distribution corresponding to the Gaussian Bell Curve) but also systematic / methodological errors and deficiencies in the tests methods themselves contribute to these distributions. The methodological causes for the systematic errors are e.g. inaccurate descriptions of test methods allowing for too much room for interpretation and / or insufficient standardisation of the test method with too high a degree of freedom / flexibility in the performance of the test or in the selection of the test technique. For this reason, the interlaboratory test programme of CEQAT-DGHS currently focuses on interlaboratory tests for methods improvement and validation while proficiency tests (comparison of the performance between the laboratories on basis of already existing validated test methods) might be addressed at a later stage. 2.3 Measurement uncertainty of test methods and interlaboratory tests It is essential to know exactly how well a method is that is used to classify dangerous goods or hazardous substances or to determine safety-relevant parameters. A key criterion for the validation of the test method is the measurement uncertainty. One can choose simple methods with large error limits, but in any case, the measurement uncertainties should be known and communicated together with the test results. This would allow safety specialists to be able to assess test results correctly and make well-founded decisions, e.g. for adequate protective measures. The measurement uncertainty of test methods can be determined efficiently by the aid of interlaboratory tests for methods validation, Hässelbarth (2004), ISO (2004), and can be expressed as shown in Eq(1). ‘Laboratory result’ = ‘Measurement result’ ± U, (1) 4 where U denotes the expanded measurement uncertainty U = k * u with u = sR The factor k corresponds to the coverage factor k according to GUM (Joint Committee for Guides in Metrology (2008)) and the factor sR denotes the reproducibility standard deviation obtained in interlaboratory tests for methods validation. For example, in the interlaboratory test 2012-2013 for the validation of the test method EN 14522:2005, the measurement uncertainty of MINLTI (minimum value of the lowest temperatures of ignition of the 3 or 5 trials measured by Method S of EN 14522:2005, section 4.5.2.2 / IEC 60079-20-1, part 7) for Acetone und n-Heptane could be determined as given in Eq(2) and Eq(3), Lüth et al. (2014): • Test method measurement uncertainty of MINLTI for Acetone: ‘Laboratory’s result’ = ‘MINLTI’ ± 2 * 7,9 °C = ‘MINLTI’ ± 15,8 °C (k=2), (2) • Test method measurement uncertainty of MINLTI for n-Heptane: ‘Laboratory’s result’ = ‘MINLTI’ ± 2 * 2,5 °C = ‘MINLTI’ ± 5,0 °C (k=2). (3) When determining the measurement uncertainty of test methods by interlaboratory tests, only test results from laboratories meeting the requirements of the test method and the interlaboratory test instruction may be taken into account. This principle is illustrated in Figure 4 where test results shown in grey were not included in the calculation of the measurement uncertainty. Figure 4: Distribution of the laboratory results and assessment of the interlaboratory test 2012-2013 for validation of the test method EN 14522:2005 “Determination of the auto ignition temperature of gases and vapours” / IEC 60079-20-1, part 7 “Method of test for auto-ignition temperature” with reproducibility standard deviation sR and repeatability standard deviation sr, Lüth et al. (2014) It must be considered that the expanded measurement uncertainty U of the validated method calculated by equation Eq(1) may only be applied by laboratories fulfilling certain criteria, e.g. successful participation in suitable proficiency tests, Hässelbarth (2004), Lüth et al. (2014). 3. Conclusion and outlook In view of the to date results of the CEQAT-DGHS interlaboratory test programme, the following conclusions can be drawn: • A need for improvement is demonstrated for all examined test methods. Thus, interlaboratory tests shall initially aim at the improvement and validation of the test methods and not on proficiency tests. • To avoid any discrepancies in classification and labelling of chemicals validation of the test methods used should become state of the art and test results should be accompanied by the measurement uncertainty. PROLab Plus 5 • The laboratory management and the practical execution of the tests need to be improved in many laboratories. • The term "experience of the examiner" must be seen critically: A "long experience with many tests" is not necessarily a guarantee for correct results, Lüth, Kurth (2013), Lüth et al. (2014). Therefore, continued efforts are necessary to further develop interlaboratory test programmes such as CEQAT-DGHS. Currently, the CEQAT-DGHS interlaboratory test programme is operated by BAM in collaboration with PTB and QuoData GmbH. Method validation by interlaboratory tests require considerable efforts with regard to personnel/capacities and the resources available at the competence centre CEQAT- DGHS therefore limit the number of interlaboratory tests that can be offered to approximately one per year. Interested laboratories can get detailed information and register for participation in interlaboratory tests at the CEQAT-DGHS website (www.ceqat-dghs.bam.de). References Antoni S., Kunath K., Lüth P., Schlage R., Simon K., Uhlig S., Wildner W., Zimmermann C., 2010, Evaluation of the interlaboratory test on the method UN test O.1 “Test for oxidizing solids” with sodium perborate monohydrate 2005 / 06, Final report, BAM, Berlin, Germany, ISBN 978-3-9814281-2-4, BAM accessed 26.06.2018. Antoni S., Kunath K., Lüth P., Simon K., Uhlig S., 2011, Evaluation of the interlaboratory test on the method UN O.2 / EC A.21 “Test for oxidizing liquids” 2009 – 2010, Final report, BAM, Berlin, Germany, ISBN 978- 3-9814634-0-8, BAM accessed 26.06.2018. Brandes E., Colson B., Frost K., Lüth P., Simon K., Stolz T., Uhlig S., 2017, Evaluation of the interlaboratory test 2015 – 2016 on the method UN Test L.2 “Sustained combustibility test” / EN ISO 9038:2013 “Determination of sustained combustibility of liquids", Final report, BAM, Berlin, Germany, ISBN 978-3- 9818270-3-3, BAM accessed 26.06.2018. European Union, 2008, Commission Regulation (EC) No 440/2008 of 30 May 2008 laying down test methods pursuant to Regulation (EC) No 1907/2006 of the European Parliament and of the Council on the Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH), OJ L 142, 31.5.2008, p.1. Frost K., Lüth P., Schmidt M., Simon K., Uhlig S., 2016, Evaluation of the interlaboratory test 2015-2016 on the method DIN EN 15188:2007 “Determination of the spontaneous ignition behaviour of dust accumulations”, Final report, BAM, Berlin, Germany, ISBN 978-3-9818270-0-2, BAM accessed 26.06.2018. Hässelbarth W., 2004, BAM-Leitfaden zur Ermittlung von Messunsicherheiten bei quantitativen Prüfergebnissen, Forschungsbericht 266, BAM, Berlin, Germany, ISBN 3-86509-212-8. ISO, 2004, ISO/TS 21748:2004 Guidance for the use of repeatability, reproducibility and trueness estimates in measurement uncertainty estimation, Geneva, Switzerland. ISO, 2017, ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories, Geneva, Switzerland. Joint Committee for Guides in Metrology, 2008, Evaluation of measurement data — Guide to the expression of uncertainty in measurement (GUM 1995 with minor corrections), BIPM accessed 26.06.2018. Kunath K., Lüth P., Schmidt M., Simon K., Uhlig S., 2013, Evaluation of the interlaboratory test 2010-2011 on the method DIN EN 15188:2007 “Determination of the spontaneous ignition behaviour of dust accumulations”, Final report, BAM, Berlin, Germany, ISBN 978-3-9815748-4-5, BAM accessed 26.06.2018. Kunath K., Lüth P., Uhlig S., 2011, Interlaboratory test on the method UN test N.5 / EC A.12 “Substances which, in contact with water, emit flammable gases” 2007, Short report, BAM, Berlin, Germany, ISBN 978- 3-9814634-1-5, BAM accessed 26.06.2018. Lüth P., Brandes E., Stolz T., 2014, Interlaboratory test 2012-2013 on the method EN 14522:2005 "Determination of the auto ignition temperature of gases and vapours" / IEC 60079-20-1, part 7 "Method of test for auto-ignition temperature", Final report, BAM, Berlin, Germany, ISBN 978-3-9816380-0-4, BAM accessed 26.06.2018. Lüth P., Kurth L., 2013, Ringversuch mit dem Fallhammer gemäß Abs. 1.6.2 Mechanische Empfindlichkeit (Schlag) der Methode A.14 Explosionsgefahr 2011, Kurzbericht, BAM, Berlin, Germany, ISBN 978-3- 9815748-6-9, BAM accessed 26.06.2018. United Nations, 2015, Recommendations on the Transport of Dangerous Goods, Manual of Tests and Criteria, Sixth revised edition, New York and Geneva. 6