Upsala J Med Sci 99: 277-306, 1994 6. The Nordic Protein Project Per Hyltoft Petersenl, Ole Blaabjergl, Kerttu Irjala2, Art0 Ic6n3, Kristian Bj0r04. 1 . Department of Clinical Chemistry, Odense University Hospital, DK-5000 Odense C, Denmark. 2. Department of Clinical Chemistry, T u r k u University Hospital, SF-20520 Turku, Finland. 3. Department of Clinical Chemistry, Helsinki University Hospital, SF-00290 Helsinki, Finland. 4. Department of Clinical Chemistry, Rikshospitalet, N-0027 Oslo I, Norway. 6.1 Aim of the Project When the Nordic Protein Project was decided by the Nordic Control Committee in 1986 and supported by NORDKEM in 1987, the purpose of the project was to 1. Improve analytical quality of specific plasma protein measurements in the Nordic countries, and Develop a more general model for analytical quality management, which could be applied to other laboratory analytes as well. 2 The basis for the project was the relationship between calibration and control, where the idea was that external control was nothing but a registration of the current situation, as long as calibration was dependent on a variety of commercial calibrators with no - or questionable - documentation of standardization. The first aim was, therefore t o introduce a common and reliable calibrator (cf. chapter 5.3 and 5.4) available to all participating laboratories - and then assess their analytical performance with control samples of the same high quality as the calibrator. In other words: ’you cannot control analytical quality into your method’ - the quality depends on calibration, analytical specificity etc. as described below. For proteins (and other components to be measured by immunological methods) the identity between the measurand in calibrator, patient samples and controls is essential. This ideal, however is not possible for plasma proteins, which are present in a variety of forms, so the aim was ’to measure the average composition of proteins from healthy adults correctly’. This concept, looks rather modest, but even at this low level it can be very difficult, as seen from chapter 5.4 and chapter 7, and later in this chapter. 277 Specificity and interference are dependent on the analytical principle and on the reagents used in the measurement procedure and the trueness of a measured value is also dependent of these factors. Interference is defined as modification of the signal t o be measured, and a variety of blood substances are interfering with the protein molecules and, thereby, modifying the signal - but for healthy individuals this effect is assumed t o be comparable in all samples and therefore, negligible for the investigations in this project. The non-specificity due to turbidity in samples, however, is a serious problem for many protein methods, as the reaction to be measured is the increase in turbidity based on the formation of immunocomplexes, whether these are measured photometrically (turbidimetric) or fluorimetrically (nephelometric). This problem with turbid samples is the major specificity problem and, in consequence, the aim was to design a control system, which could separate bias due to unspecificity from bias due t o the calibrator, and pointing t o the methods which are sensible to turbihty, and need clarification of samples before measurements. Further, the aim was to assess the analytical performance in the individual laboratories challenging their calibration function as well as the performance in general. ELEMENTS OF ANALYTICAL QUALITY 1 (-) ANALYTICAL (-) (-) SPECIFICATIONS / \ During the project it became clear that a definition of good and poor quality was needed. The most relevant concept would have been to estimate analytical quality specifications from the clinical outcome of the results. This approach, however, could 278 not be applied as there are no clear recommendations for interpretation of data from measurements of plasma proteins in the Nordic countries. Interpretations were generally based on the reference intervals and, consequently, a new aim was to define analytical quality specifications according to the concept of sharing common reference intervals. These quality specifications were evaluated a s a general principle a s described in chapter 4. Production of common reference intervals could not be directly integrated into the project, but by co-operation with the Danish and the Finnish protein groups the common reference intervals were established (cf. chapter 7). COMMON REFERENCE INTERVALS ESTABLISHMENT REFERENCE VALUES %cEu 1 SPECIFICATIONS / \ [-) [-) Based on the extensions of reference intervals and analytical quality specifications the aim developed t o analytical qua& management in the sense that all elements were integrated entirely, where clinical useful reference intervals were backed up by a system of analytical quality improvement in order to establish the base for the common reference intervals and the control system was designed to test whether each laboratory could qualify for the use according to the analytical quality specifications derived for this purpose, i.e. t o share the common reference intervals established. A number of problems which are related to genetical variants and pathological changes are not taken into consideration in the project. The most relevant are: 279 1. The influence of genetic phenotypes on the quantifications e.g. Haptoglobin and ccl-Antitrypsin 2. Varying composition of immunoglobulins e.g. oligoclonal and monoclonal immunoglobulins 3. Binding of components to proteins e.g. Bilirubin to Albumin and Haemoglobin to Haptoglobin 4. Extreme concentrations e.g. Detectability for Haptoglobin and extreme values of immunoglobulins 280 6.2 Analytical Quality Specifications The analytical quality specifications were achieved according to the concept of sharing common reference intervals, and were based on an average of published reference intervals, assuming log-Gaussian distributions. This assumption was confirmed by the actual distributions obtained from the reference values as described in chapter 7. The actual curves for maximum allowable combined analytical bias and imprecision are displayed in Fig. 6.2.1. 2 BIAS % l 4 1 12 10 a 6 4 2 0 MAXIMUM ALLOWABLE COMBtNED BIAS AND IMPRECISION Transfer r in 0 5 10 15 20 25 30 IMPRECISION (Cv) % Fig. 6.2.1. Combined analytical quality specifications as used in the beginning of the project. From Hyltoft Petersen et al. (3) with permission. It is interesting to see that the acceptable variations and errors are very wide for S - Haptoglobin and the two immunoglobulins IgA and IgM, with acceptable imprecision about 25 % (with negligible bias). The acceptable values for bias are about half of this value (when imprecision is negligible). Most of the proteins have acceptable imprecisions (with negligible bias) of about 10 % and corresponding bias of 4 to 5 %. The most demanding criteria are for S-Albumin with maximum allowable values of 4 and 2 % for imprecision and bias, respectively. These criteria are hardly fulfilled by any laboratory, and this protein illustrates that the variation of homeostatic set-points is narrow. 28 I In the beginning of the project the two analytical specifications for bias and imprecision were used separately allowing the maximum of both simultaneously, but at the end where the new specifications were introduced based on the estimated reference intervals (cf. chapter 7) the combined criteria were taken into consideration. These criteria are shown in Fig. 6.2.2. BIAS % ALLOWABLE COMBINED BIAS AND IMPRECISION FOR SPECIFIC PLASMA PROTEINS 14 1 10 l 2 l - - 12 - Transthyretin (Prealbumin) al -Trypsin Inhibitor Transthyretin (Prealbumin) al -Trypsin Inhibitor 0 5 10 15 20 25 30 C V % Fig. 6.2.2. Combined analytical quality specifications as used at the end of the project, based on the estimated reference intervals (chapter 7). The new analytical quality specifications are very close to the first ones. This was not expected as the separation of the reference intervals into subgroups due to the biological differences according to age, sex and the use of estrogens, might have resulted in more demanding specifications (chapter 7). 282 6.3 Creation of Analytical Quality The calibrator. As mentioned under aim of the project, a very important element of the project was t o supply all participants with the same high quality calibrator for daily use in order t o create that element of quality in t h e project. In the first survey, however, t h e laboratories should use their own calibrator for comparison, and the calibrator was sent together with t h e control samples in the first mail. The calibrator is described in chapter 5.3 and t h e traceability to IFCC/CAP/BCR 470 is documented in chapter 5.4 (A new production of calibrator is available from The Danish Society of Clinical Chemistry). The calibration function. Calibration functions are related to analytical principles and to equipments, and each calibration function should fit the points of dilutions of t h e calibrator in an x-y plot or transformations of data. This problem has not been investigated in the project, but beside instructions from producers of equipment and reagents, DAKO has developed a number of recommendations for analytical performance at different instruments, which are well documented. BCR CRM 470 SPECIFICATIONS ------- I CREATION OF ANALXTICAL B U M PERFORMANCE CONTROL OF ANALmrcAL I Q U f i W 283 Analytical procedure, When buying an equipment for measurements of plasma proteins the analytical principle is already decided. Some instruments are designed for the purpose whereas, others can be used for kinetic as well as for endpoint measurements. The two dominating analytical principles are turbihmetry and nephelometry, whereas, the gel-methods are too time consuming for routine work. The key reagents for protein measurements are the antisera, where the commercial available products for the main plasma proteins all are of a quality, without cross reacting immunoglobulins. The individual laboratory is without influence on equipment and antibodies (except from the first choice of buying), whereas, it is responsible for the implementation and the current performance. These, include instructions education and training of technicians as well as maintenance and an internal control system for stability and imprecision. 284 6.4 Principles for Control As outlined above the analytical quality of the measurements are determined by external factors and internal factors. The external factors, equipment, calibrator and reagents, are outside the laboratory's influence. The internal factors are implementation and performance, and here the laboratory is responsible for the process. Another approach is to divide the factors into permanent and variable factors. This, makes the concept of control easier to grasp. The permanent factors are the external: equipment (analytical principle), type of calibrator (traceability), and the internal: implementation. The variable factors are the external: batches of calibrator and reagents, and the internal: daily performance. Internal control. An average internal control system can only disclose variations and errors in the variable factors, and these will show up as systematic changes (change in concentration level) and changes in random variation (imprecision). (With special designs other problems, like detection limit, changes in specificity can be controlled internally? but this is not done in general). In consequence? the internal control systems are capable of monitoring stability and imprecision only. VARIABLE FACTORS INTERNAL CONTROL CONTROL ( - G i i = ) - ( Y ) \ / 1 CONTROL OF QUALITY / \ CONTROL IMPRJXISION VARIATION 285 External control systems. External control systems can only describe the analytical quality at the time of control, and only to a certain level. So, they will see t h e sum of a mixture of permanent and variable factors - if it is not designed for separating these factors. In general, however, an external control system should describe the permanent situation. PERMANENT FACTORS EXTERNAL CONTROL CONTROL [=) 0 CALIBRATION \ / ~ / \ SPECIFICITY 1-b CONTROL 0 SPECIFICm Principle for the control system. In order t o separate the permanent factors from the variable the control is performed with three assessments over two years with the same control materials, and with repeated measurements over four days in duplicates. Thereby, the internal factors can be described separately and the estimate of actual bias can be performed with a rather narrow confidence interval. By repeating the assessment three times the permanent bias can be estimated - as long as the laboratory has not changed the method. The permanent factors are calibration and specificity and they are challenged by a set of controls where the main specificity problem, turbidity, is approached by a set of two control materials, which are identical except from the lipid fraction (responsible for t h e turbidity) and three clear pools with relative concentrations of all proteins t o be measured of 0.5, 1.0 and 1.5 as described below. 286 6.4.1 Principles for C o n t r o l Materials Control materials designed for assessing the calibration under optimal conditions should be like an ideal sample without any interfering substances, as denaturation of the measurand, unspecific reactions o r interference will invalidate the calibration and result in an estimated bias, due to conditions which have nothing to do with the calibration proper. These other factors should be dealt with separately by samples which are identical, except from the presence of the interfering substance in one of them. In the project the following four control pools were used (1). (CLEAR) I CONTROL T CONTROL A ( = C O N T R O L T) (CLEAR) I (TURBID) CONTROL c (= 0 S " C O N T R O L A ) ( C L E A R ) The clear controls A, B, and C are designed for control of calibration at three levels. Control A is a pool from healthy individuals and should have a concentration value near the middle of the reference interval (or slightly below, due to the log-Gaussian distribution) and Control C is constructed to give half the concentration of Control A for the proteins under consideration, while Control B has 1.5 times the concentration of Control A. Control T is identical to Control A, but is turbid with the lipoproteins present Production of the Controls. A serum pool collected from 100 volunteers was produced after test of all individuals for antibodies against HIV and Hepatitis. From each 0.5 litre blood was drawn and after coagulation and centrifugation the individual sera were frozen at -80 "C and stored at this temperature until all the portions were thawed and pooled to be used as Control T. This pool contains all lipoproteins. A fraction of the Control T-pool is ultracentrifuged and restored to three different control pools. The procedure for all of these pools is initially a 'normal' centrifugation at 4 "C in order to get rid of cryo-precipitates. Then a portion is weighed and 287 ultracentrifuged for 48 hours at 180.000*g. There are three layers: The lipid layer which is sucked of, The infranatant, which consist of all the low molecular weight: constituents of plasma, but without the plasma proteins under consideration, and the lowest layer with all the plasma proteins in a high concentration. ControZA After removal of the lipid layer the weight is reconstituted with infranatant from an other ultracentrifugation. ControZB. After removal of the lipid layer, part of the infranatant is removed until the volume is 2/3 of the original (by weighing, keeping the specific gravity in mind). Control C. After removal of the lipid layer, infranatant (from the production of Control B) is added to a volume of 2 times the original (by weighing, keeping the specific gravity in mind). ( 100%) A ( 150%) Vol. adj. ___t 180.000 g Lipid free 40 h supernatant (100%) (50%) CONTROL T CONTROL CONTROL C All control pools are dispensed into cryotubes (1 mL) and frozen at - 80 "C, and kept at that temperature until shipping packed with dry ice to each laboratory. The target values were assigned by measurements in four laboratories by Mancini technique. (The assigned values were used as the original target values of the calibrator, and they were important for the evaluation until the IFCC/CAP/BCR 470 values were transferred, but the old values have no interest now). 288 6.4.2 Principles f o r Evaluations of D a t a The general evaluations of data were rather traditional with construction of histogrammes of the data from each participant, but with indication of target value and the accept limits for bias accordng t o the analytical quality specifications shown. For the turbid Control T, the results were further divided into turbidmetric and nephelometric analytical principles in order to disclose the robustness of the analytical principles to this challenge. Two examples, one for the clear Control A and one for the turbid Control T, both for S-Transferrin, are shown in Fig. 6.4.2.1 and Fig. 6.4.2.2, respectively. Further, the results for Control T are divided according t o nephelometric and turbidimetric analytical principles and illustrated in Fig. 6.4.2.3. It is seen that the turbidimetric methods are more robust. TARGET VALUE Local Calibrator 1 2 00 2 50 3 00 350 gll S-TRANSFERRIN CLEAR CONTROL (CON A) TARGET VALUE 2 04 232 3 w 3 % OiL Fig. 6.4.2.1. Histogrammes of S-Transferrin showing the results for the clear Control A. Upper: Results when laboratories used their own calibrator. Lower: Result when The Nordic Calibrator was used. From Blaabjerg et al. (2) with permission. The individual eualuations were designed in order to separate the elements of analytical variations and errors according t o calibration function, specificity as well as within- and between-run variation. The results were compared to the analytical quality specifications and described graphically in order to support the local trouble- shooting process. 289 TARGET VALUE I Local Calibrator Fig. 6.4.2.2. 2 00 2 50 3 00 350 g l l S-TRANSFERRIN TURBID CONTROL (CON T) TARGET VALUE 2w 2 50 3 DO 350 g l L Histogrammes of S-Transferrin showing the results for the turbid Control T. Upper: Results when laboratories used their own calibrator. Lower: Result when The Nordic Calibrator was used. From Blaabjerg et al. (2) with permission. Nordic Calibrator S-TRANSFERRIN TURBID CONTROL (CON T) TARGET VALUE a ENOPOINT K I M T I C NEPHELOMETRIC TURBlOlMETRlC + GEL TECHNIOUES t : TARGET VALUE w - 3 DO 0 TURBOlMTRlC ROCKET 7 350 g l t Fig. 6.4.2.3. Histogrammes of S-Transferrin showing the results for the turbid Control T. Same data as i n Fig. 6.4.2.2, but separated according to nephelometric and turbidimetric analytical principles. From .Blaabjerg et al. (2) with permission. 290 Combined evaluation of calibration function and imprecision For each survey, each laboratory, and each protein the results from Controls A, B, and C were arranged in a matrix and treated according to a two-way analysis of variance with replicates. First the data were 'normalized' by multiplying results from Control C with a factor 2 (which should bring them to the level of Control A) and results from Control B were multiplied by a factor 2/3 (also t o bring them to the level of Control A). This was done according to the relative dlutions of of the three controls - so, if the calibration function was correct the between-control variance component should be negligible. The data were treated as a two-way analysis of variance with crossclassification according to controls and days. As mentioned above the between-control component described the proportionality according to the calibration function (as an average). The between-day-component described the between-run variation (reflecting both variations in performance and variations in the average level of calibration), and the replicate-component described the within-run variation. Finally, the interaction-component described fluctuations in the calibration function, i.e. the variation in proportionality of results. This design and evaluations allowed validation of the analytical performance in each laboratory within a short period of time (one week) at three occasions (over two years), and should give a firm basis for the individual laboratory t o disclose the weak links of the analytical procedure, and the informations needed for trouble-shooting. Evaluation of effect of turbidity For each day the difference between means of Control T and Control A were calculated and the mean difference from four days was calculate with 90 % confidence intervals. This was an estimate of the effect of turbidity. Graphical presentation of data The results for the clear controls were displayed as a difference plot with the measured results minus target values as ordinate and target values as abscissa. The 90 % confidence interval was shown for each point and, further, the accept limits (for sharing common reference intervals) were illustrated as straight lines. An example of S-Albumin is shown in Fig. 6,4.2.4. 20-955059 29 1 S- Albumin (BCG-method) DIFFERENCE (measured - target) (g/L) 6- 4, 2- 0, -2 - accept limit Control C Control A accept limit i ........... ................... I a Control C Control A 1 -2 accept limit -10 -8j It Control B 0 20 40 60 8 0 Target c o nc e n t r a t i on ( g / L ) Fig. 6.4.2.4. Difference-plot for S-Albumin from a laboratory using a BCG-dye binding method. Ordinate: difference between measured value and target value. Abscissa: Target value. The points for Control C, Control A, and Control B are given with 90 % confidence intervals. Further, the dotted area illustrates the acceptance area for sharing common reference intervals. From Hyltoft Petersen et al. (3) with permission. This example with S-Albumin illustrates a calibration function which should have been curved, but was fitted by a linear function (one-point calibration). A further illustration of the combined effects of the laboratory's own calibrator and the turbidity is performed in a double difference plot for the two differences (measured Control A minus target value) and (measured Control T minus measured Control A). The ordinate is the same as for the difference plot in Fig. 6.4.2.4 (but now the only point shown is the Control A point with 90 % confidence interval) and the abscissa is the 'turbidity-difference (also shown with 90 % confidence interval). The acceptance limits are shown for both differences. The combined results show up as a cross, which for acceptance should be located within the square formed by the two acceptance areas. 292 In the example (Fig. 6.4.2.5) for S-Haptoglobin the 'cross' is located within the square, and the calibration as well as the turbidity correction are acceptable. + I \ f \ Accept Limits '4- 0 Difference (Control T - Control A) TURBIDITY Fig. 6.4.2.5. Double d i fference-plot for S-hap toglobin. Ordinate: Difference between measured Control A and the target value. Abscissa: Difference between measured Control T and measured control A. The acceptance limits for sharing common reference intervals are indicated. The data were collected and typed by Jens Rahbeck Nargaard in Odense and the computations and printouts were performed by a computer group in Uppsala, chaired by Torsten Aronsson. For each survey each laboratory received a printout for each of the proteins with the computations and plots, together with an evaluation as judged from us, Fig, 6.4.2.6 A and B on the next pages is an example of such a printout. S-Orosomucoid is taken as an example from an indwidual report (Fig. 6.4.2.6 A and B). On the first page (A), the calculations are presented. Point 1 shows results with the local calibrator and for calibration with The Norhc Calibrator. Mean values and to,,+SEM for estimation of the 90 % confidence interval, then within- and between-run coefficients of variation (CVw and CVB) calculated according to a simple nested design (repeated balanced subsampling), further the target value is shown. In point 2, the effect of turbidity in Control T (compared to Control A) is given, and in point 3, the variations in the three clear controls over four runs are evaluated accordmg to a two- way analysis of variance with crossclassification according to controls and runs, with 293 NKK's protein projekt Ororomucoid NORDIC PROTEIN PROJECT Lab.: 2 9 5 2 EVALUATION O? TRE LAB DATA 111111111111111..111111-111 1) INDIVIDUAL POOLS AND EFCECTS OF RECALIBRATION ........................................................................... values directly CAL CON C CON A CON B CON T Hean value 0 . 5 2 8 0 . 2 6 0 0 . 5 0 5 0 . 7 5 9 0 . 5 3 0 to. l*SEM 0.016 0.005 0.014 0.023 0.011 Var. within run cvwt 0.9 1 . 9 1 . 4 1 . 4 0.9 Var. between run CVb I 2.4 0 . 8 2 . 1 2 . 3 1 . 6 Recalibrated values Hean value --- 0.409 0 . 7 9 5 1.194 0 . 8 3 4 to. 1*SEM --- 0.007 0.011 0.014 0.012 Var. between run cvb I --- 0.0 ' 0 . 7 0.0 1.0 ___--_--------- ------- ------- ------- ---____ __----- ------------------- 0.830 0.380 0.760 1 . 1 5 0 0 . 7 6 0 2 ) EVALUATION 01 BIAS (uring control A and control TI TOTAL BIAS BIAS ?OR C L W R CONTROL INTERFERENCE ( CON T - TAXGET) (CON A - TARGIST) (CON T - CON A ) ---------------- ---------------- --------------- Direct Recal. Diract Recal. Direct Mean value -0.23 0.07 - 0 . 2 5 0 . 0 3 0 . 0 3 tO.l*SW 0.01 0.01 0.01 0.01 0.00 3) EVALUATION OI LINEARITY (uring 2 W O N C, 1*CON A and 2/3*CON B) Evaluation Sourcer o f variation 6 N I Replicatar 0.008 1.6 within run variation Between run 0.009 1.8 Betvaen run variation Between poolr 0 . 0 0 8 1.6 Conrtant unlinearity Interaction 0.000 0.0 varying unlinearity ------ ------ ------ ------ ------ ........................................................................... _____------_----------------- .................... ------- ------- EVALUATION O? NJALYTICAL PERIORFUNCE 1111111111-111111111-1----11111-1-1- Good Reproducibility within run ................ between run Linearity constant (between p o o l r ) . varying (interaction)....... Calibrator present ................... - Nordic calibrator............ Roburtnerr in relation to turbidity ............. ................ $ .... Accept Unaccept Commentr: Fig. 6.4.2.6.A. Example ofprint out from a survey. First page. For further explanations, see text. 294 A c c u ra c y o n 3 le v e ls M E A S U R E D - T A R G E T 0 . 11 L a b . : 2 9 5 2 C o m p o n e n t: O ro s o m u c o id C O R R E C A E Y ,- T A R G E T - 0 . ::i -0. 9 B ia s fo r c le a r c n r o l (C o n A - T a rg e t) 8 .\I - 0 . -0 . -:; -0. + 4 , In te r fe r e n c e ? & n T - C o n A ) subsampling. Finally, a valuation of the different factors is performed, with possibilities for writing comments, which could focus on the problems and advise for trouble-shooting o r improvements. On page 2 of the report (Fig. 6.4.2.6.B) two hfference-plots - one for the local calibrator and one for The Nordic Calibrator - are presented for the three controls with 90 % confidence intervals. The zero-line indicates no bias and the two sloping lines frame the acceptance area. In the example. the bias is reduced by use of The Nordic Calibrator, but the results are not perfect, as all controls show values above the target, and the differences are rather constant, indicating 'a constant bias in performance' which e.g. could be a problem with a blanc correction or wrong calibration function. Below, the double bias-plot - for bias due to calibration (own calibrator) for Control A and for bias due to poor correction for turbidity - is shown, together with the acceptance area. The 'cross' - with 90 % confidence intervals - repeats the informations above in the report showing acceptable correction for turbidity, but analytical bias due to the local calibrator. After the three surveys the results were cumulated for each laboratory and the final conclusion was given. Fig. 6.4.2.7 illustrates such a conclusion for S-Orosomucoid (same laboratory as illustrated in Fig. 6.4.2.6, run no 2). The report summarizes the 'Turbidity robustness' from the three rounds. Further, the results for each clear control in each round are shown in a plot combining bias (with sign) and imprecision, and with acceptance-area (hatched area) according t o the analytical quality specifications for using the common reference interval. Only values outside the acceptance-area are shown, and in the example, it is clear that t h e laboratory had serious problems in the first round (A1 with a CV of 20 %, B1 just outside the limits, and C1 with a bias of t30 %), but the situation has improved in the two next rounds (only C2 with a bias of + 7.5 %). Communication with the participants Some of the evaluations done in this project are a little untraditional and it was therefore of importance to keep the participants informed about the project and the types of evaluations. This information was maintained by papers with a short explanations designated Blue Folders. An example of a front page and a page with information is shown in Fig. 6.4.2.8. 296 The Nordic Protein Project March 1994. Laboratory No. 2452 Final Evaluation in the Nordic Protein Project PROTEIN: S-Orosomucoid (S-a,-Acid Glycoprotein) Control T Turbidity robustness Controls: CONtnOL m I - 1 S T O N T R C L 4 1 Acceptable y e s = 1st round x - Pndround ST - 3rdround - I I C L E A R 1 I lCLEARi CONTROL T CONTROL A I - CONTPOL T I ITURBID) Controls A, B, and C in rounds 1,2, and 3 Calibration & Imprecision +a% NO of control values 4 f outside acceptance l i z CONTROL C 1 - 0 s ~ C O N T R O L A I ICLEAR) Three controls in 3 runs Only points outside t h e acceptance figure are indicated IMPRECISION $/5 (CV %) -10 -9 -8 -7 -6 - 5 -4 -3 -2 - 1 0 1 2 3 4 5 6 7 8 9 10 BIAS' Overall Evaluation of the Three Surveys Regarding the Use of the Common Reference Intervals for this Protein: By use of The Nordic Calibrator or other calibrator traceable to BCR CRM 470 - you can use the Common Reference Intervals for t h e native population ................................................................... [ 1 you can possibly use the Common Reference Intervals .... you need improvement of your method ................................... - DQ hy& [ ] ~ + o u m d , , n . U l - Kind regards T h e Nordic Protein Project Fig. 6.4.2.7. Example of a cumulated report to a laboratory after the three surveys. For further explanations, see text. 297 N W 0 0 % $ ' R P R O JE C T S O N R E F E R E N C E I N T E R V A L S A F in ni sh p ro je ct o n re fe re nc e in te rv al s f or an dS -I gM ln cl ud in gr ef er en ce in te rv al s f or vi du a1 s) an da ge de pe nd en t i nt er va ls fo r c hi ld re n vi du al s) h as b ee n pe rf or m ed .* A D an is hp ro je ct o n re fe re nc e in te rv al s f or th e ni ne se ru m pr ot ei ns is ta ki ng p la ce p ri m o 19 90 . H er e ap pr ox . 8 00 a du lt s ar e p la nn ed to b e in ve st ig at ed . R es ul ts fr om th e tw o m at er ia ls w ill b e co m pa re d an d th e po si bi lit ie sf or sh ar in gs om e of t he re fe re nc e in te rv al s w ith - in g re at er g eo gr ap hi ca l a re as w ill b e in ve st ig at ed . F O R C O M M O N I R E F E R E N C E IN T E R V A L S K Ir jn h et 01 . R ef er en ce ra ng es fo r ir nr nu nq lo hu tin s I @ . I@ , an d Ig M in se ru m in n di rk s n nd c hi ld re n q e d I1 2 to ll ye n rs . (f ob ep u bl iv h ed ) T H E N O R D IC P R O T E IN P R O JE C T I G O A LS I A N A L Y T IC A L Q U A L IT Y S TA N D A R D IZ A TI O N C O N TR O L ST A R T O F P R O JE C T Y ou w ill re ce iv e t he fi rs t s et of s am pl es in th e pr oj ec t ( ca li - br at or a nd c on tr ol s A , B , C , a nd T , f ou r a m po ul s o f e ac h) in Ja nu ar y. T he sa m pl es w ill b e se nd fr oz en o n dr y ic e. Pl ea se al so fi nd e nc lo se d - th e i ns tr uc tio ns fo r p er fo rm an ce an d tim e s ch ed ul e - a ne w fo rm ul a f or re gi st ra tio n o f m et ho ds . In th is fo ld er th e ba ck gr ou nd fo r e st im at io n o f t he g oa ls fo r an al yt ic al q ua li ty is gi ve n. 6.5 Results 6.5.1 General Evaluations For the general evaluations the main object was to assess the three analytical principles (turbidmetric, nephelometric, and gel methodshn relation to calibration and influence of turbidity of samples. Fig. 6.5.1.1. Histogrammes o f S-IgM showing the results for the clear Control A. Upper: Results when laboratories used their own calibrator. Lower: Result when The Nordic Calibrator was used. From Blaabjerg et al. (2) with permission. The best overview is given by a histogramme for each control sample, and in Fig. 6.5.1.1 are shown the results for the clear Control A for the use of local calibrator and for The Nordic Calibrator by measurements of S-IgM. These histogrammes illustrate the general result, that under optimal conditions it is possible to obtain acceptable results from all analytical principles by use of a reliable calibrator. Practically all results are within the acceptance limits for the use of common reference intervals. This was expected, but the evaluations of the analytical principles under optimal condtions, here the clear Control A, are necessary for the further evaluations of influence of turbidity in patient samples on the robustness of the principles. 299 TARGET VALUE : +L Local Calibrator n o 05 I " 1 5 2 0 2 5 q r l S-IgM TURBO CONTROL ICON TI TARGET VALUE ;I Nordic Calibrator Fig. 6.5.1.2. Histogrammes of S-IgM showing the results for the turbid Control T. Upper: Results when laboratories used their own calibrator. Lower: Result when The Nordic Calibrator was used. From Blaabjerg et al. (2) with permission. Nordic Calibrator S IgM TURBID CONTnOL (CON T I TARGET VALUE NEPHELOMETRIC 0 TURBlDlMETRlC +GEL TECHNIWES TARGEl VALUE Fig. 6.5.1.3. Histogrammes of S-IgM showing the results for the turbid Control T. Same data as in Fig. 6.5.1.2, but separated according to nephelometric and turbidimetric analytical principles. From Blaabjerg et al. (2) with permission. 300 When the analytical principles are evaluated under 'ideal' conditions it is easier to make conclusions from the result obtained with the turbid Control T, as the only additional factor is the turbidity of the sample. Thus, Fig. 6.5.1.2 illustrates the same informations as in Fig. 6.5.1.1, but now the results obtained with local calibrator are complicated by the combined effect of calibrator and turbidity, whereas, the results with The Nordic Calibrator demonstrate a nearly pure effect of turbidity, as we know that the the calibrator alone would give acceptable results. The combined use of well defined control materials makes the interpretation clear. Table 6.5.1.1 Quality of calibration and influence of turbidity TURBTDIMETRY Acceptable laboratories2 in percentage of all using the same analytical principle Protein Prealbumin Albumin Orosomucoid al-Antitrypsin Haptoglobin Transferrin IgA IgG IgM ~ ~ Calibrator Local Nordic 14 % 86 % 29 % 86 % 25 % 69 % 44 % 100 % 8 % 5a % 6 % a7 % 60 % 97 % 60 % 86 % 60 5% 94 9% ~~ Turbidity1 - Number of Laboratories 43 % 50 % 57 % 56 % 100 % 81 % 91 % 97 % 69 % 7 12 14 16 27 31 35 35 35 ~ ~~ ~- 1.: Measurements directly on turbid samples. 2.: Acceptance criteria according to Fig. 6.2.2. The effects of turbidity on the different analytical principles are further illustrated in Fig. 6.5.1.3, which shows that the gel and turbidimetric methods are rather insensitive to the turbidity. The nephelometric principles are illustrated for both kinetic and endpoint methods, and both distributions are broadened towards higher values indicating the problem. 30 1 S-Transferrin concentrations in plasma are higher than S-IgM concentrations and in most analytical principles they are analysed in higher dilutions. Accordingly the effect of turbidity was expected t o be less distinct, but as seen from figures 6.4.2.1 to 3, there are still effects for nephelometric endpoint principles. Table 6.5.1.2 Quality of calibration and influence of turbidity NEPHELOMETRY Acceptable laboratories2 in percentage of all using the same analytical principle ~ ~~ Protein Prealbumin Albumin Orosomucoid al-Antitrypsin Haptoglobin Transferrin IgA IgG IgM ~ ~~ Calibrator Local Nordic 26 % 70 % 10 % 50 % 56 % 69 % 10 % 52 % 58 % 100 % 6 % 71 % 44 % 100 % 28 % 96 % 44 % 96 % Turbidity1 50 % 80 % 19 % 62 % 92 % 47 % 52 % 96 O/o 4 % Number of Laboratories 10 10 16 21 24 1 7 25 25 25 ~ ~~~ ~ 1 .: Measurements directly on turbid samples. 2.: Acceptance criteria according to Fig. 6.2.2. In Table 6.5.1.1 t o 3 the data are summarized for all nine plasma proteins according to the three main analytical principles, turbidimetric, nephelometric and gel methods. It is seen from the tables - as expected - that a common calibrator can improve the trueness of all analytical principles t o a high degree. Further, the problems with turbid samples are demonstrated for S-IgM using the nephelometric principles. Two proteins, however, demonstrate special problems: S-Albumin has a narrow reference interval and the quality specifications are accordingly demanding, which is reflected in a low percentage of acceptable 3 02 laboratories for all methods. Further, many of the dye-binding methods have problems with calibration functions as the calibration curve is fitted by a straight line, resulting in bias for all concentration values, except from one point. Table 6.5.1.3 Quality of calibration and influence of turbidity GEL METHODS Acceptable laboratories2 in percentage of all using the same analytical principle Protein Calibrator Turbidity1 Local Nordic Number of Laboratories Prealbumin Albumin Orosomucoid al-Antitrypsin Haptoglobin Transferrin IgA I& IgM 25 % 0 % 25 % 27 % 43 % 50 % 50 % 100 % 100 % 100 % 0 % 50 o/o 45 % 100 5% 100 % 100 % 100 % 100 % 100 % 100 5% 63 % 45 % 100 % 50 % 100 % 100 % 100 % 4 1 8 11 7 2 2 2 1 1.: Measurements directly on turbid samples 2.: Acceptance criteria according to Fig. 6.2.2. S-al-Antitrypsin disclose another type of problem which is related to the protein structure in the calibrator, as evaluated in chapter 5.5, but the storing conditions in the different laboratories may not have been optimal, so a random (i.e. not possible to investigate now) effect of minor denaturation could occur. This effect, however, could also origin from difficulties with measuring the protein by turbilmetric and nephelometric principles. 303 6.5.2 Individual Evaluations Evaluation of individual laboratories is - by definition - individual, and it is more difficult to give an overview of these, as there are only few general aspects which are not reflected in the general evaluations. The most relevant summing up is therefore to count how many laboratories fulfilled the analytical quality specifications over the three surveys as evaluated by the revised quality specifications (Fig. 6.2.2) and evaluated according t o the combined criteria for analytical bias and imprecision as illustrated in Fig. 6.4.2.7. Two types of evaluations are performed. The first is the average of control results inside the figure of combined criteria in percentage, and the second is the percentage of laboratories with 66 % or more acceptable values. Table 6.5.2 Evaluation of analytical quality according to the quality specifications for sharing common reference intervals by two criteria. 1. Average of the percentage of control results with acceptable values 2. Percentage of laboratories with 66 % or more acceptable values Protein Prealbumin Albumin Orosomucoid al-Antitrypsin Haptoglobin Transferrin IgA IgG IgM Criteria 1 50 % 33 % 66 % 26 % 97 % 63 % 95 % 80 % 90 % Criteria 2 Number of Laboratories 45 % 16 % 73 % 10 % 97 % 51 % 98 % 85 % 95 % 20 51 41 48 61 57 66 66 66 The percentage of laboratories which can share common reference intervals is close to 100 % for S-Haptoglobin, S-IgA, and S-IgM, mainly due t o rather loose criteria for sharing common reference intervals for these three proteins (cf. section 6.2), whereas, 304 the percentages for S-IgG and S-Orosomucoid are reasonable high. S-Prealbumin and S-Transferrin are close to 50 %, but the percentages for the two proteins S-Albumin and S-al-Antitrypsin are so low, that it may be questionable whether it is possible t o use common reference intervals. The first is due t o the very demanding criteria, and the other may be related to instability of the calibrator or controls as discussed above. 305 6.6 Conclusions The two main problems in protein measurements are calibration and turbid samples. The calibration problems can be solved for all analytical principles by use of The Nordic Calibrator or some other reliable calibrator with concentration values traceable to the reference preparation for plasma proteins, BCR 470, allowing the majority of laboratories to use the common reference intervals (with some reservations for S-Albumin and S-al-Antitrypsin). Nephelometric analytical principle have problems with turbid samples, but this can be solved by delipidation of turbid samples, preferable by high speed centrifugation, e.g. with a bench-top ultracentrifuge. Acknowledgements We want to thank all the people and institutions who have supported the project and all the laboratories which have participated. We are especially greatful to The Nordic Committee for Quality Control, which initiated the project, to NORDKEM, LAB- QUALITY and The Danish Society of Clinical Chemistry for stimulation and economical support. Further, we want to thank Torsten Aronsson, Per Gidfors and Jens Rahbek Nprrgaard for the very important data-handling of all the results, as well a s Spren Blirup-Jensen and Per Just Svendsen for transfer of concentration values to the calibrator. A special thank t o Karin Jorgensen and Ane Richter for skilful laboratory work and handling of data. References 1. Blaabjerg 0, Blom M, Gry H , Hyltoft Petersen P, Uldall A. Appropriate Sera for Assays and Calibration of Specific Proteins Scand J Clin Lnb Inuest 1992;52 suppl. 212:13-5. Blaabjerg 0, Irjala K, Hyltoft Petersen P, 1ct.n A, Bj0ro K. Effect of Calibrator and Specimen Turbidity. Moodz 1992;2:109-11. Hyltoft Petersen P, Blaabjerg 0, Irjala K, I c h A, Bjpro K. A Model for Quality Achievement - The NORDKEM Protein Project. Scand J Clin Lab Inuest 1992;52 suppl. 212:lO-2. 2. 3. 306