Interlaboratory comparison results of vibration transducers between TUBITAK UME and Roketsan ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 401 ACTA IMEKO ISSN: 2221-870X December 2020, Volume 9, Number 5, 401 - 406 INTERLABORATORY COMPARISON RESULTS OF VIBRATION TRANSDUCERS BETWEEN TÜBİTAK UME AND ROKETSAN S. Ön Aktan1, E. Bilgiç2, İ.Ahmet Yüksel, K.Berk Sönmez, T. Kutay Veziroğlu, T. Torun 1 Department of Calibration Laboratory, Roketsan Missiles Industries Inc., Ankara, Turkey, son@roketsan.com.tr 2 TÜBİTAK Ulusal Metroloji Enstitüsü (UME), Gebze, Kocaeli, Turkey, eyup.bilgic@tubitak.gov.tr Abstract: This paper presents an interlaboratory comparison on vibration metrology field which can be used as a powerful method of checking the validity of results and measurement capabilities according to ISO 17025 [1]. In this standard it is advised to participate in an interlaboratory comparison or a proficiency test in order to prove measurement capabilities of calibration providers. In this study it is aimed to statistically evaluate the measurements results in the scope of sinusoidal acceleration between TÜBİTAK UME (National Metrology Institute of Turkey) and Roketsan as per related International Standards. After statistical evaluation, for unsatisfactory results, root cause analyses and corrections to improve measurement quality are presented and conceptually explained. Keywords: vibration transducer; vibration comparison; vibration metrology; vibration uncertainty 1. INTRODUCTION Human began to manufacture machines, and especially motors were used to strengthen them, engineers encountered vibration isolation and reduction techniques [2]. Contrary to this, vibration can be generated intentionally for testing purposes to understand functional and physical response and resistibility of any system in vibration environments. For above protective or testing purposes, acceleration sensors are used to measure acceleration, vibration and shock values, which are one of the most important components used in navigation systems of missiles, aircrafts, ships and submarines. As a result of significant developments on these industries such as automotive, defence, aviation and space, the need for accurate measurement has increased and over the years, there have been several improvements in vibration measurement methods [3], [4], [5], [6] with many innovations. Accelerometers can be used in varied applications and the most commonly used ones in the market are piezoelectric and capacitive type accelerometers. Piezoelectric accelerometers have a more widespread usage due to their advantages such as large measuring frequency range, no need for power supply, reliability, robust design, and long- term stability. A typical response curve of a piezoelectric accelerometer is given in Figure 1 [7]. Figure 1. Response-Curve of an Accelerometer The limits of the usable range are both mechanical and electrical including frequency (f), acceleration (a), velocity (v), and displacement (d); and also force of a vibration generating system. The displacement amplitude for a given acceleration is inversely proportional to the frequency 𝑑 = 𝑣 2𝜋𝑓 = 𝑎 (2𝜋𝑓)2 . (1) While displacement measurements require attention at low frequencies, it is necessary to pay attention to the acceleration level at high frequencies [8], [9]. When selecting the vibration transducer for a specific application, it is essential to pay attention to the parameters such as number of axes, measurement range, overload or damage limits, mass, sensitivity, impedance and frequency range. For reliable usage of accelerometers, a calibration plan shall be scheduled periodically by producing http://www.imeko.org/ mailto:son@roketsan.com.tr mailto:eyup.bilgic@tubitak.gov.tr ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 402 right test results to assure the process and provide metrological traceability. Accurate equipment, metrological traceability, trained personnel, well defined methods, documentation, uncertainty evaluation, internal verifications can play an important role to provide accurate test results however it is also necessary to prove that the laboratory can actually produce results externally by going through comparison tests [10]. Interlaboratory comparison (ILC) is the regulation, implementation and evaluation of tests or measurements of two or more laboratories on the same or a similar substance according to predetermined conditions [11]. Interlaboratory comparison tests are planned according to ISO 17043 [11] and the performances, 𝑬𝒏 values or zeta scores ζ of the participating laboratories are evaluated according to ISO 13528 [12]. 2. BACK-TO-BACK CALIBRATION METHOD AND APPLICATION The purpose of vibration transducer comparison is to compare the sensitivity of accelerometer by using as a secondary level (back to back method) ISO 16063:21 Vibration Calibration by a Reference Standard [13]. Calibration of an accelerometer is to determine the sensitivity values at various frequencies. The reference (double ended transducer) and device under test (DUT) are firmly coupled on a shaker so that both are exposed to the same mechanical motion. Back-to-back calibration requires a shaker (vibration exciter), power amplifier, signal generator and FFT frequency meter. Currently, automated systems are also available in the market with advantages of easy operation, user friendly and short calibration time requirement. Basic configuration of Roketsan’s vibration transducer calibration system is illustrated in Figure 2 below: Figure 2. Roketsan Vibration Transducer Calibration System As shown in Table 1, the system used is an automatic vibration transducer calibration system that operates between 10 Hz to 5000 Hz and also, the accuracy values are illustrated. Brüel&Kjaer 8305 S is used as a reference accelerometer. Table 1. Specifications Performance Features Accuracy (10 to 2000) Hz : 0.7 % ( >2 to 5 ) kHz : 1.1 % Acceleration, max 110 m/s2 Max. Transducer Weight 60 g Force 45 N Max. Displacement 8 mm Environmental conditions of Roketsan vibration laboratory are (23 ± 3) °C for temperature and maximum 75 % rh for relative humidity. According to ISO 16063:21 the frequency and acceleration values are given below: 1. Frequencies (Hz): Frequencies are selected from one-third-octave frequency series. In case exact frequency values are required, they are calculated for the 1/3 octave bands [14] with the formula below 𝑓 = 𝑓𝑟 ∙ 10 ( 𝑛 10 ) 𝑓r = 1000 Hz (2) where n= -20, -19, …, 7 for 10 Hz to 5 kHz. 2. Acceleration (m/s2): 1, 2, 5, 10 or their multiple of tens. 100 m/s2 is recommended. The main principle of back-to-back calibration is direct comparison of indicated sensitivity values between reference transducer and DUT transducer. The applied vibration to each transducer is identical and if the sensitivity of the reference transducer is known, the sensitivity of the DUT can be obtained by using the following equation: 𝑆𝐷𝑈𝑇 = 𝑆𝑅𝐸𝐹 · 𝑉𝐷𝑈𝑇 𝑉𝑅𝐸𝐹 (3) 𝑆𝐷𝑈𝑇 : Sensitivity of Device Under Test 𝑆𝑅𝐸𝐹 : Sensitivity of Reference Accelerometer 𝑉𝐷𝑈𝑇 : Electrical Output of Device Under Test 𝑉𝑅𝐸𝐹 : Electrical Output of Reference Accelerometer Even though above approach is suitable for a single frequency value, it may take excessive time to perform this operation at all frequency values. Hence, dual channel FFT analysis is used to monitor fast frequency response functions in amplitude and phase angle in shorter time period. 3. UNCERTAINTY APPROACH As it can be observed over the uncertainty budget given in Table 2, one of the maximum uncertainty contribution comes from reference transducer set. Furthermore, voltage ratio measurement affects the measurement results. Influence on voltage ratio http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 403 from temperature variation, gravitational acceleration, distortion, transverse acceleration, non-linearity effects shall be added to the uncertainty budget. The following contributions are taken into account while calculating the measurement uncertainty budget in the calibration of accelerometers. Table 2. Uncertainty Budget Quantity 𝑿𝒊 Definition Standard Uncertainty 𝒖(𝒙𝒊) Probability Distribution Sensitivity Coefficient 𝒄𝒊 Uncertainty Contribution 𝒖𝒊(𝒚) u(SREF) The calibration uncertainty of reference transducer set 0.5 Normal 1 0.250 u(SREF,S) The uncertainty due to drift of reference transducer set and amplifier 0.08 Rectangular 1 0.046 u(SA,Kal) The calibration uncertainty of conditioning amplifier 0.09 Rectangular -1 0.045 u(VR) The uncertainty from voltage ratio 0.08 Rectangular 1 0.046 u(VR,T) Temperature influence on voltage ratio measurement 0.2 Rectangular 1 0.173 u(VR,S) Voltage ratio measurement from maximum difference in reference level 0.2 Rectangular 1 0.115 u(VR,N) Voltage ratio measurement from mounting parameters 0.3 Rectangular 1 0.173 u(VR,d) Voltage ratio measurement from acceleration distortion 0.0024 Rectangular 1 0.001 u(VR,v) Voltage ratio measurement from transverse acceleration 1.2 Special 1 0.283 u(VR,e) Voltage ratio measurement from base strain 0.05 Rectangular 1 0.029 u(VR,r) Voltage ratio measurement from relative motion 0.05 Rectangular 1 0.029 u(VR,L) Voltage ratio measurement from non-linearity of transducer 0.03 Rectangular 1 0.017 u(VR,I) Voltage ratio measurement from non-linearity of amplifiers 0.03 Rectangular 1 0.017 u(VR,G) Voltage ratio measurement from gravity 0.03 Rectangular 1 0.017 u(VR,B) Voltage ratio measurement from magnetic field effect of the vibration exciter 0.03 Rectangular 1 0.017 u(VR,E) Voltage ratio measurement from other environmental effects 0.03 Rectangular 1 0.017 u(VR,R) Voltage ratio measurement from residual effects 0.03 Rectangular 1 0.017 u(VR,RE) Repeatability 0.17 Normal 1 0.098 Combined Uncertainty of Measurement 𝑢𝑡 0.48 Expanded Uncertainty of Measurement U, k = 2 0.97 http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 404 Reference transducer and if exists, its conditioner should be calibrated as a set by a primary level laboratory. The measurement uncertainty 𝑢(𝑆𝑅𝐸𝐹) is presented in the calibration certificate of the reference transducer shall be added to uncertainty budget as divided by reliability coefficient (95% confidence level, k = 2). 𝑢(𝑉𝑅,𝑣 ) is the uncertainty contribution due to transverse accelerations. Transverse vibration 𝑎𝑇 is maximum 10 % for vibration exciter. Transverse sensitivity for reference transducer 𝑆𝑣,𝑅𝐸𝐹 is maximum 2 % and the device under test 𝑆𝑣,𝐷𝑈𝑇 is maximum 5 %. Using the formula below, the uncertainty can be evaluated as 1.2 %. 𝜎 = √(𝑆𝑣,𝐷𝑈𝑇 2 + 𝑆𝑣,𝑅𝐸𝐹 2 ) 𝑎𝑇 2 (4) Repeatability ( 𝑢(𝑉𝑅,𝑅𝐸𝑃 ) ) is an experimental standard deviation of the arithmetic mean to the uncertainty. It is an inevitable contribution for an uncertainty budget. The model function is given below. 𝑆𝐷𝑈𝑇 = 𝑆𝑅𝐸𝐹 ∙ 𝑆𝐴1 𝑆𝐴2 ∙ 𝑉𝐷𝑈𝑇 𝑉𝑅𝐸𝐹 ∙ 𝐼1 ∙ 𝐼2 … ∙ 𝐼𝑀 (5) 𝐼𝑖 = 1 − 𝑒2,𝑖 1 − 𝑒1,𝑖 (6) 𝑒𝑖 ; Indicates the i th error contribution The uncertainty budget for vibration transducer is in Table 2 for 10 Hz to 1000 Hz. Combined uncertainty (𝑢𝑡 ) and the expanded uncertainty (𝑈) (k=2, 95% confidence level) can be calculated with following formulas (7) and (8) according to EA-4/02 [15]: 𝑢𝑡 = √𝑢 2(𝑆𝑅𝐸𝐹 ) + 𝑢 2(𝑆𝑅𝐸𝐹,𝑆) + (𝑢𝑉𝑅 ) 2+.. (7) 𝑈 = 2 ∙ 𝑢𝑡 (8) 4. COMPARISON RESULTS The technical protocol [16] specifies in detail, the aim of the comparison, the transfer standard used, time schedule, the measurement conditions and other subjects. The frequency range covered by the requirements stated in the technical protocol has been carried out with TÜBİTAK UME and Roketsan. The model of transfer standard used is B&K 4371. The pilot laboratory is TÜBİTAK UME, which is primary laboratory in Turkey. Since Roketsan performs related measurements with lower uncertainty than any other secondary level calibration providers, an interlaboratory comparison with TÜBİTAK UME as primary level laboratory was necessary to understand Roketsan’s reliability of accuracy level. Figure 3 presents the calibration results of sensitivity obtained for a transfer standard. The results can be observed in Table 3. Figure 3. Measurement Results from TÜBİTAK UME and Roketsan Table 3. Measurement Results from TÜBİTAK UME and Roketsan Frequency UME; Reference Value Roketsan 𝑬𝒏 (Hz) (pC/(m/s2) (pC/(m/s2) 10 1.0046 1.0063 0.13 12.5 1.0042 1.0110 0.51 16 1.0043 1.0087 0.33 20 1.0029 1.0080 0.38 25 1.0016 1.0063 0.35 31.5 1.0007 1.0070 0.47 40 0.9989 1.0060 0.54 50 0.9981 1.0060 0.60 63 0.9956 1.0040 0.63 80 0.9935 1.0010 0.57 100 0.9919 1.0010 0.69 125 0.9905 0.9993 0.67 160 0.9893 1.0010 0.89 200 0.9872 0.9957 0.65 250 0.9847 1.001 1.24 315 0.9818 0.9923 0.80 400 0.9813 0.9918 0.80 500 0.9801 0.9905 0.80 630 0.9800 0.9883 0.64 800 0.9776 0.9873 0.75 1000 0.9771 0.9854 0.53 1250 0.9746 0.9834 0.53 1600 0.9759 0.9826 0.40 2000 0.9757 0.9802 0.27 2500 0.9729 0.9814 0.51 3150 0.9762 0.9849 0.52 4000 0.9800 0.9842 0.25 5000 0.9800 0.9814 0.08 0,95 0,96 0,97 0,98 0,99 1 1,01 1,02 1 0 1 6 2 5 4 0 6 3 1 0 0 1 6 0 2 5 0 4 0 0 6 3 0 1 0 0 0 1 6 0 0 2 5 0 0 4 0 0 0 S e n si ti v it y ( p C /( m /s 2 ) Frequency (Hz) Roketsan UME http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 405 The uncertainty values for laboratories are given below in Table 4: Table 4. Uncertainty Values UME; Reference Laboratory Frequency (Hz) f ≤ 1250 1250 < f ≤ 5000 Uncertainty 0.9% 1.1% Roketsan Frequency (Hz) f ≤ 1000 1000 < f ≤ 5000 Uncertainty 0.97% 1.3% The performances of the measurements can be evaluated as described in ISO 13528 Standard with the following equation (9). The most common statistical approach to understand capability of a laboratory is calculating 𝐸𝑛 values and shall be below or equal to one. 𝐸𝑛 = 𝑋ROK − 𝑋UME √𝑈ROK 2 + 𝑈UME 2 (9) 𝑋ROK: The mean value of Roketsan 𝑋UME : The mean value of reference laboratory (UME) 𝑈UME: The measurement uncertainty of reference laboratory (UME) 𝑈ROK: The measurement uncertainty of Roketsan Calculated 𝐸𝑛value at 250 Hz is 1.24 and the rest of the other frequencies are satisfactory |𝐸𝑛 | < 1 in Table 3. After receiving unsatisfactory result only for 250 Hz frequency, root cause analysis or some corrective actions will be carried out to improve measurement system of Roketsan Inc. 5. CORRECTIVE ACTION Nonconformity in ISO 9001 is defined as the failure to meet one or more requirements [17]. AS9131 “Nonconformance Data Definition and Documentation” classify the nonconformity process codes (Shipping and Transportation, Manufacturing, Document Preparation), cause codes (Machine, Management, People, Material, Method, Environment, Measurement), corrective action codes (Machine, Management, People, Material, Method, Environment, Measurement) [18]. Root cause analysis is the process of identifying casual factors using a structured approach with techniques designed to provide a focus for identifying and resolving problems [19]. It is essential to determine the root cause and create a corrective action plan in order to eliminate the causes of nonconformity before occurring again or in another field. Principles of continuous improvement and monitoring of efficiency are important for the continuity of management systems. When comparison results are not satisfactory, a non-conformance record shall be issued and action process shown in Figure 4 shall be started in order to find a solution to keep system reliable. A method such as Pareto Chart, 5 WHYs, Fishbone Diagram, Scatter Plot Diagram, Failure Mode and Effects Analysis (FMEA) for determining root cause of problem should be applied to gain appropriate vision for detecting and removing problem. Figure 4. Nonconformance Process Among all error source possibilities, sensitivity value of the reference transducer set had top priority to check since calibration status was close to calibration due date. After forwarding this equipment to primary laboratory for re-calibration, although 1 year has passed between two calibrations, it has been observed that previous sensitivity value at 250 Hz had been changed from 0.1312 pC/(m/s2), to 0.1307 pC/(m/s2). When considering last two calibration certificates difference, higher measurements result change have been obtained and the reference value has shifted unlike assumption for drift may occurred during 1 year. Above condition was considered as main reason for the detected nonconformity. Verification of the vibration system has a vital role for getting accurate measurements, the reference accelerometer used in calibration (working standard) is connected back-to-back with the reference accelerometer. Subsequent verifications compare the first results to the new results and accept the results whether it deviates less than 0.8 %. The controller checks that the standard deviation of the measurements is less than 0.2 %. When the Fishbone diagram is applied, the root causes are seen in Figure 5. After an extensive training for all operators, Gage R&R application indicators showed competency of appraisers are satisfactory. After this, temperature gradient of measurement room was examined and it has seen that there is no need for action on temperature subject. Regarding mechanical effects, the torque value was adjusted to 2 N m as desired precisely and it is Definition of Nonconformance Determining Root Cause Identification of Corrective Action Effectiveness of Corrective Action http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 406 confirmed that requirement of standard have been met. Figure 5. Fishbone Diagram As a result of system review as detailed above, verification and calibration issue had been estimated as only root cause which gave rise to unsatisfactory 𝐸𝑛 value at 250 Hz frequency measurement with the new calibration results, we have confirmed that there is a drift in the value of the reference. As a result of evaluation, it is decided to perform detailed investigation on root cause of drift on reference sensor. Since the reason is not fully understood, it is decided to reorganize interlaboratory measurement to receive satisfactory 𝐸𝑛 values. 6. SUMMARY Further aspect could be considered to understand the unsuccessful results at 250 Hz. Further study may cover participation in a new comparison test and in case of another insufficient result, decreasing the calibration period, increasing the measurement uncertainty due to drift of reference transducer set can be taken as further actions. The results produced by the laboratory become valid with comparison tests as well as the method of measurement, competency of appraisers, and calculated measurement uncertainty, suitability of the equipment used, calibration and traceability. Since ISO 17025 wants also a risk and opportunity based approach, proficiency testing can be used as a training and risk tool. 7. REFERENCES [1] ISO 17025:2017 General Requirements for the Competence of Testing and Calibration Laboratories [2] Measuring Vibration, Brüel&Kajer, www.bk.com [3] X.Bai, “Absolute Calibration Device of the Vibration Sensor”, The Journal of Engineering, 2018 [4] V.Mohanan, B.K. Roy, V.T. Chitnis, “Calibration of Accelerometer by Using Optic Fiber Vibration sensor”, Applied Acoustics, 28, pp. 95-103, 1989 [5] R. R. Bouche, “Calibration of Vibration and Shock Measuring Transducer”, The Shock and Vibration Information Center, 1979 [6] K.Havewasam, H.H.E. Jayaweera, C.L. Ranatunga, T.R. Ariaratne, “Development and Evaluation of a Calibration Procedure for a 2D Accelerometer as a Tilt and Vibration Sensor”, Proceedings of the Technical Sessions, 25, pp. 53-62, 2009 [7] C. Vogler, “Calibration of Accelerometer Vibration Sensitivity by Reference, College of Engineering, 2015 [8] W. Ohm, L.Wu, P. Henes, G. Wonk , “Generation of Low-Frequency Vibration Using a Cantilever Beam for Calibration of Accelerometers”, Journal of Sound and Vibration 289, pp. 192–209, 2006 [9] N. Garg, M.I. Schiefer, “Low Frequency Accelerometer Calibration Using an Optical Encoder Sensor”, Measurement, vol. 111, pp. 226-233, 2017 [10] K. B. Sönmez, T. O. Kılınç, İ.A.Yüksel, S.Ö.Aktan, “Inter-laboratory Comparison on the Calibration of Measurements Photometric and Radiometric Sensors”, International Congress of Metrology, 2019 [11] ISO/IEC 17043:2010, “Conformity Assessment General Requirements for Profiency Testing” [12] ISO 13528:2015, “Statistical Methods for Use In Profiency Testing By Interlaboratory Comparisons” [13] ISO 16063:21, “Vibration Calibration by a Reference Standard” [14] ISO 266, “Acoustics Preferred Frequencies” [15] EA-4/02, “Evaluation of the Uncertainty of Measurement in Calibration” [16] UME-G2TI-2018-01, “Technical Protocol of the Interlaboratory Comparison on Acceleration”, 2018 [17] ISO 9001:2015, “Quality Management Systems” [18] AS9131:2012, “Nonconformance Data Definition and Documentation” [19] M.A.M.Doggett, “A Statistical Comparison of Three Root Cause Analysis Tools”, Journal of Industrial Technology, vol.20, no:2,2004 Inexperienced Personnel Temperature Change during Calibration Power, other utility variations Calibration Drift of Reference Value Verification Torque Value Mounting Personnel Measurement Environment Setup 250 Hz En >1 http://www.imeko.org/ http://www.bk.com/