Developments of interlaboratory comparisons on pressure measurements in the Philippines ACTA IMEKO ISSN: 2221-870X June 2023, Volume 12, Number 2, 1 - 6 ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 1 Developments of interlaboratory comparisons on pressure measurements in the Philippines Mary Ness I. Salazar1,2 1 National Metrology Laboratory of the Philippines – Industrial Technology Development Institute - DOST, Philippines 2 Measurement Science, KRISS School - University of Science and Technology, Daejeon, Republic of Korea Section: RESEARCH PAPER Keywords: pressure measurement; intercomparison; proficiency testing; interlaboratory comparison Citation: Mary Ness I. Salazar, Developments of interlaboratory comparisons on pressure measurements in the Philippines, Acta IMEKO, vol. 12, no. 2, article 36, June 2023, identifier: IMEKO-ACTA-12 (2023)-02-36 Section Editor: Leonardo Iannucci, Politecnico di Torino, Italy Received January 13, 2023; In final form May 24, 2023; Published June 2023 Copyright: This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This work was supported by the Laboratory Proficiency Evaluation Program of the NML-ITDI, DOST. Corresponding author: Mary Ness I. Salazar, e-mail: marynesssalazar@yahoo.com 1. INTRODUCTION The National Metrology Laboratory (NML) of the Industrial Technology Development Institute (ITDI) under the Department of Science and Technology (DOST) conducted interlaboratory comparisons in the field of hydraulic pressure measurement among the local calibration laboratories in the Philippines. The provision of interlaboratory comparison, otherwise known as proficiency testing (PT), is a program to strengthen the NML relationship with the said laboratories in establishing scientific metrology in the country. This PT program aims to [1], [2]: (a) determine the technical capabilities and performance of the laboratories; (b) assess the reliability of their measurement results and validate their calibration measurement capabilities; (c) disseminate a harmonized and validated calibration procedure; (d) demonstrate metrological equivalence to the NML and most importantly, (e) provide access to interlaboratory comparisons for their compliance to ISO/IEC 17025 [[3]] requirements. The Pressure Standards Section (PSS) of the NML acted as the program coordinator and reference laboratory, accredited under the terms of ISO/IEC 17025:2005 (2017 at present). The PSS was responsible for providing the artefact, the reference value and its measurement uncertainty, the monitoring of the program as a whole, and preparation of written reports for the two intercomparisons being compared; one was conducted in 2010 while the other was in 2016 The following will be the outline of this paper. In Section 2, the comparison process will be discussed followed by the measurement results of the participants. Discussion of these results commences in Section 4, and the evaluation of both the PTs is compared in Section 5. Finally, the concluding section summarizes the major accomplishment and development based on these experiences. ABSTRACT The National Metrology Laboratory of the Philippines, Industrial Technology Development Institute under the Department of Science and Technology (ITDI-DOST) offered first of its kind interlaboratory comparison in pressure measurement in the country following a demand survey identifying the gaps in proving the competency of local calibration laboratories. This paper presents the development in implementing the two interlaboratory comparisons in the Philippines with six years interval using the same pressure artefact. With its many objectives, the program also aimed to provide these laboratories access to proficiency testing (PT) to comply with the ISO/IEC 17025 requirements. Comparatively, while both are considered successfully implemented, the improved awareness and commitment to quality of the participants and the enhanced competency of the pilot laboratory in implementing such activity are a few of the enumerated factors instigating the increase in the number of participants as well as those obtaining satisfactory results in the latter PT. The interlaboratory comparison schemes offered aimed at sustaining the demands of metrology stakeholders and continuously develop this service to support further progress in the calibration and measurement capabilities of local laboratories in the country. mailto:marynesssalazar@yahoo.com ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 2 2. COMPARISON PROCESS The two intercomparisons followed an almost similar process; differences are, however, emphasized in this paper, specifically the contributing factors that affected the results of the two schemes. It is herein referred to the first PT scheme as 2010 and the second PT scheme as 2016 throughout the discussion. The interlaboratory comparison program was designed as a cycle where the NML, as a reference or pilot laboratory, calibrates the artefact at the program's beginning, middle, and end. One main difference between 2010 and 2016 is the conduct of a preparatory workshop before the start of the PT program. This workshop proved essential in getting to know each laboratory's capabilities, standards, and procedures they follow in calibrating pressure gauges. Also, in this workshop, an agreement between the participants and the NML was reached, fulfilling the objective of disseminating a harmonized and validated calibration procedure and the identification of limitations of the participants and the PT program in general. 2.1. Participants Participants in the two PTs are local calibration laboratories with NML as the reference lab. In 2010, five (5) participants were all private laboratories in Metro Manila, the capital of the Philippines. In 2016, the 16 participants comprised private and government laboratories, including some DOST regional metrology laboratories from different provinces in the Philippines. 2.2. Artefact The artefact (Figure 1and Table 1) used in 2010 and 2016 is a Bourdon-tube-type pressure gauge. It was noted that in 2010, two artefacts of different ranges were measured by the participants, but in this paper, comparisons will be based on only one artefact used on both PTs, and the description is as follows: The artefact was subjected to initial and subsequent characterization before the PTs and maintained a regular interval of calibration and intermediate checks when not used as an artefact. 2.3. Calibration method The participants were asked to calibrate the artefact through direct comparison to their standard. In 2010, each participant used the typical calibration procedure in their laboratory, which means their laboratory-developed method. Meanwhile, in 2016, after the earlier mentioned preparatory workshop, it was agreed among the participants to follow an international guideline, the DKD-R 6-1 Calibration of Pressure Gauges [4]. which not only guided the calibration procedure but the computation of measurement uncertainty as well. The NML uses the said guideline. 2.4. Measurement scheme The NML chose a measurement scheme to monitor the artefact's metrological quality throughout the PT process. In 2010, the artefact was calibrated before and after a trip to a participant, as shown in Figure 2. It was hand-carried to and from each participating lab by its representative. In 2016 however, since there were more participants than in 2010 and some were outside Metro Manila, the NML calibrated the artefact before and after a group of participants, usually 2 to 3 labs, strategically chosen based on location so the sending back and forth to the NML of the artefact are made most efficiently. Figure 3 shows this scheme. Figure 1. The artefact. Table 1. Technical specification of the artefact. Manufacturer Ashcroft Serial Number / Identification S2-W-006 Measuring range 25 000 kPa Scale division 100 kPa Accuracy 0.25 % Medium Liquid Figure 2. PT 2010 measurement scheme. Figure 3. PT 2016 measurement scheme. NML Lab B NML Lab C NML Lab D NML Lab E start NML end Lab A Lab4 Lab5 Lab6 NML Lab7 Lab8 NML Lab9 Lab10 Lab11 Lab12 NML Lab13 Lab14 Lab15 NML Lab16 NML End NML Start Lab1 Lab2 Lab3 NML ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 3 2.5. Report of the participants In 2010, the participants were only asked to submit the filled- out NML-provided measurement datasheet and the calibration certificate they usually issue to their customers. Other information, such as the uncertainty budget, was only known when required by the NML. This exchange of information could have been more efficient, consequently causing a delay in data evaluation and publishing of the final report. Hence, the NML changed this practice in 2016. The participants should submit documents such as the measurement datasheets, a copy of the calibration certificate of their standard proving valid traceability, their usual calibration report, and the uncertainty budget. This change improved the data evaluation process for the NML, making it easier and faster since information transparency was present from the beginning. 2.6. Reference values In both PTs, the reference values used in evaluating the normalized error (𝐸n ) for each participant were based on the values nearest the participant’s reported results, either before or after the calibration of the reference laboratory. In 2016, this result was given to participants in an Interim Report, showing only the specific participant’s results compared to NML’s. This interim report was beneficial for participants having to prove their competence to technical peers during their assessment while the intercomparison is not yet completed. It should be noted, however, that in the final report, the reference values reflected are the weighted average of all the measurement results of the NML. 3. MEASUREMENT RESULTS The measurement results of participating laboratories are evaluated using the earlier mentioned normalized error or the 𝐸n ratio [5], calculated using the equation: 𝐸n = 𝑥lab − 𝑥ref √𝑈lab 2 + 𝑈ref 2 , (1) where 𝑥lab is the measured value of the participating laboratory, 𝑥ref is the reference value, 𝑈lab and 𝑈ref are the expanded uncertainty (k = 2) of the participant’s measured value and the reference value, respectively. The reference value in this equation is the deviation of the artefact reading from the NML’s applied pressure at the nominal calibration points. Similarly, the measured value of the participating laboratory is the deviation of their reported value to the nominal calibration points. This practice ensures the uniformity of values to be compared. Moreover, the expanded uncertainties were reported with a coverage factor of k = 2, indicating a confidence level of approximately 95 %. Figure 4 shows the performance of participants in the two PTs. More participants joined in 2016 with 88 % (14 out of 16) satisfactory performance compared to 40 % (2 out of 5) in 2010. Two participants joined both PTs and one participant performed better in 2016 while the other still failed. In 2010, four out of the five participants were able to calibrate the artefact in its full measuring range, while one participant did not submit a result in the last two highest points. Meanwhile, in 2016, all the participating laboratories were able to calibrate the artefact as a whole. Table 2 and Table 3 show the 𝐸𝑛 value of the participants in 2010 and 2016 respectively. Figure 4. Participants’ performance on two PTs. Table 2. Summary of 2010 participants’ En values relative to the reference values. Table 3. Summary of 2016 participants’ En values relative to the nominal pressure values 0 5 10 15 20 2010 2016 N o . o f P a rt ic ip a n ts Year satisfactory unsatisfactory ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 4 4. DISCUSSION OF RESULTS The laboratory’s satisfactory performance was determined when |𝐸n| ≤ 1 in all the prescribed calibration points. In 2010, only 95 % of the total calibration points with |𝐸𝑛 | ≤ 1 were required to be considered satisfactory laboratory performance, but this was later corrected to 100 % of the calibration points in 2016. Some participants interpreted that the 95 % confidence level in the uncertainty budget estimate may also be applied in the inter-laboratory comparison result, thus, assuming that they don’t need to perform well in all the measurement points since there is a 5 % margin of error. The NML had to explain the rationalization that the 5 % margin of error cannot be tolerated in the measurement procedure since the basic requirement of the PT was to calibrate the artefact as a whole and any doubt in their procedure must well be accounted for in their uncertainty budget and not on their measurement value. Also, a 95 % satisfactory performance will not be possible in the calibration points prescribed since any one point was already 10 % of the artefact’s ten calibration points. It is, however, emphasized that the laboratory’s performance is considered satisfactory only through the 𝐸n score. Accordingly, so as not to abuse the said acceptance criteria, it was later suggested that a limiting value for uncertainty is to be set, and most of the participants followed it in 2016. For illustration purposes, Figure 5 and Figure 6 show the participant’s deviation from the reference value with its corresponding uncertainties in the non-zero minimum and the maximum calibration points, respectively. The 2010 participants were represented by blue markers and labeled alphabetically (LAB A to LAB E), while participants in 2016 were of green markers and were labeled with numbers (LAB1 to LAB16). Comparing the participants’ performance in the two PTs, all the participants performed satisfactorily in the minimum calibration point in 2010 as opposed to those in 2016, with 1 participant whose value already lies beyond the limit if not with its uncertainty. Differences in the computed uncertainty values depended mainly on the standard they used, primarily a digital pressure gauge; only a few used a pressure calibrator and a deadweight tester. Moreover, in 2010, while the NML prescribed a guideline for measurement uncertainty calculations, most participating laboratories estimated the expanded uncertainties using their laboratory procedure and technique with the notion that declaring a low uncertainty means better performance. Figure 5. Participant’s deviation from the reference values and its corresponding uncertainties at 2500 kPa. Figure 6. Participant’s deviation from the reference values and its corresponding uncertainties at 25 000 kPa. -150 -100 -50 0 50 100 150 200 D e v ia ti o n , k P a 2 500 kPa Measurement point -350 -300 -250 -200 -150 -100 -50 0 50 100 150 D e v ia ti o n , k P a 25 000 kPa Measurement point ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 5 Contrastingly, in 2016, all the participating laboratories followed the agreed guideline on procedure and uncertainty estimates, with the lower limit as the resolution of the artefact. This agreement ensured that measurement uncertainties were not over or under-estimated. A sample calculation of the recommended minimum components of measurement uncertainty of the pilot laboratory is shown in Table 4. The minimum components of measurement uncertainty were enumerated as the factors coming from the standard, the height difference between the standard and the artefact or the so-called hydrostatic pressure effect, the artefacts’ resolution, zero deviation, hysteresis, and repeatability. The expanded uncertainty, as earlier mentioned, was evaluated with a 95 % level of confidence with the coverage factor, k = 2. The uncertainty from the standard was taken from the most recent calibration certificate of the artefact before the start of the interlaboratory comparison. The uncertainty due to the hydrostatic pressure effect was evaluated but was found negligible due to minimizing, if not eliminating, the height difference of the reference pressure points of the standard and the artefact during calibration. Similarly, the uncertainty due to zero deviation was also found negligible due to the behaviour of the artefact. The major contributor to uncertainty in the reference value was the resolution of the artefact, which was agreed to be the limiting factor for uncertainty evaluation for all the participants, including the pilot laboratory. This decision accommodated the limited accuracy capability of some participants, which caters mainly to industrial calibrations. The uncertainty due to hysteresis and repeatability are the maximum values among the measurement performed by the NML. The NML used these factors in both the 2010 and 2016 PT. 5. EVALUATION OF COMPARISONS Comparing the two PTs as a whole, defined factors affecting the performance of participating laboratories are summarized in Figure 7. The earlier mentioned preparatory workshop, not done in 2010 but conducted in 2016, proved to be one key factor that led to the increased satisfactory performance of participants. In 2016, four laboratories could not attend this workshop; however, two laboratories asked for details after the event and strictly followed the agreed procedure, 1 lab with a satisfactory result but did not follow the agreed uncertainty limit, and one did not perform satisfactorily. In both PTs, the main objective of the laboratories' participation was to fulfil the ISO/IEC 17025 requirement for PT participation. However, the urgency of this requirement was only partially realized by the 2010 participants since the accreditation to ISO/IEC 17025 was reasonably new in the country during that time. Moreover, most 2010 participants have inexperienced or untrained personnel and need to familiarize themselves with the calibration method used by the NML. Contrastingly, in 2016, most laboratories had trained personnel and had proven competencies in scopes other than the pressure field. Their submitted results showed that knowledge of estimating uncertainty budgets also significantly improved through training. The latter PT also showed an enhanced selection of standard and upgraded facilities by the laboratories. It is most probable that the 2010 PT was considered as a test run by the participating laboratories, with two labs participating satisfactorily on both, 1 lab that improved in 2016, 1 that still unsatisfactorily performed, and one that did not continue to be a calibration laboratory, this is also the participant that did not complete the measurement due to inappropriate standard. The 2016 participants, on the other hand, are mostly maintaining their Table 4. The NML’s calculation of the recommended minimum measurement uncertainty components. Calibration Item Test Gauge Measurement Range 25 000 kPa Scale Division 100 kPa Resolution 50 kPa Uncertainty of standard used 4.1 kPa Coverage factor (k) 2 Components of Uncertainty Nominal Pressure Uncertainty from the Standard Uncertainty due to height difference Uncertainty due to resolution Uncertainty due to zero deviation Uncertainty due to hysteresis Uncertainty due to repeatability Expanded uncertainty kPa kPa kPa kPa kPa kPa kPa kPa 0 2.6 0.0 28.9 0.0 0.0 0.0 58 2500 2.6 0.0 28.9 0.0 0.2 0.1 58 5000 2.6 0.0 28.9 0.0 0.2 0.3 58 7500 2.6 0.0 28.9 0.0 0.1 0.1 58 10000 2.6 0.0 28.9 0.0 0.3 0.8 58 12500 2.6 0.0 28.9 0.0 0.1 0.1 58 15000 2.6 0.0 28.9 0.0 0.2 0.1 58 17500 2.6 0.0 28.9 0.0 0.1 0.1 58 20000 2.6 0.0 28.9 0.0 0.0 0.2 58 22500 2.6 0.0 28.9 0.0 0.1 0.0 58 25000 2.6 0.0 28.9 0.0 0.0 0.1 58 ACTA IMEKO | www.imeko.org June 2023 | Volume 12 | Number 2 | 6 ISO/IEC 17025 accreditation already or are in the process of acquiring their certification in the pressure scope, supported by this intercomparison. In both PTs, the NML recommended that all the laboratories with unsatisfactory performance review their calibration method and uncertainty budget analysis, investigate sources of error leading to unsatisfactory results, and initiate corrective actions. The NML, on the other hand, as the reference laboratory, continuously improves as a PT provider learning from experience, from the handling of the artefact to the analysis of data that is most appropriate to all participants. Consequently, the NML extended its PT offering regularly, with different pressure ranges and other fields of measurement; also, the conduct of the concluding workshop was planned for the succeeding PTs. Furthermore, coordination with the local accreditation body as the channel to know the demands for PT in the country for NML's plan on PT provision, and in return, the laboratories are made aware of the NML PT offerings. Availability of artefact is still the most significant limitation of the PT provision but was hoping to be resolved to cope-up with ongoing demands on PT. Currently, the laboratories' commitment to quality, supported by courses and training which are not only technical but also in the quality management systems, are contributors to the laboratories' handling of intercomparison prominent to satisfactory performance. The participants have become more aware of good laboratory practices and are encouraged to continuously improve through refresher and new Metrology awareness courses. 6. SUMMARY AND CONCLUSION The intercomparisons, 2010 and 2016, are two independent PTs and are generally considered successful in terms of results, coordination, and the experience gained by the participants and the reference laboratory. Measurement results revealed the calibration and measurement capabilities of each participating laboratory. Based on the requirements of ISO/IEC 17043:2010, the performances are mostly satisfactory, in terms of the 𝐸n values, especially in the 2016 PT. This also indicates that the measurement practices of these participants significantly improved and are aligned and complying with an internationally validated method. The PT schemes offered by the NML will be of continuous improvement to support further progress of the local calibration laboratories in the Philippines. ACKNOWLEDGEMENT Funding support for DOST and the Laboratory Proficiency Evaluation Program (LPEP) of the NML, ITDI-DOST is gratefully acknowledged. The LPEP was initially funded by the Philippines Council for Industry, Energy and Emerging Technology for Research and Development (PCIEERD) making it possible to acquire the artefact used in these intercomparisons. REFERENCES [1] Maryness I. Salazar, Final Report for Pressure Test Gauge Calibration Interlaboratory Comparison 07-2010-PTILC-0042, NML, ITDI-DOST Philippines, 2011. Unpublished results. [2] Maryness I. Salazar, Interlaboratory Comparison on Hydraulic Pressure Standards 01-2016-PTILC(PRES)-0002m NML, ITDI- DOST, Philippines, 2016. Unpublished results. [3] International Organization for Standardization (2017) ISO/IEC 17025:2017 General requirements for the competence of testing and calibration laboratories. ISO Geneva, Switzerland. [4] PTB Deutscher Kalibrierdienst, Guideline DKD-R 6-1 Calibration of Pressure Gauges. DKD, Edition 03/2014. [5] International Organization for Standardization (2010) ISO/IEC 17043:2010 Conformity assessment – General requirements for proficiency testing. ISO, Geneva, Switzerland. Figure 7. Defined factors affecting participants’ performance. 0 2 4 6 8 10 12 14 16 Preparatory workshop attendance Quality management system Training of personnel Following international guideline Use of appropriate standard N o . o f P a rt ic ip a n ts Year 2010 2016