Upsala J Med Sci 95: 265-273, 1990 Some Methodological Aspects on Quality Assurance in Laboratory Medicine Torgny Groth Unit f o r Biomedical Systems Analysis, Uppsala University, Uppsala, Sweden KNOWLEDGE-BASED SYSTEMS AND QUALITY ASSURANCE During the last decade database and knowledge-based system technology have been used increasingly for documentation and processing of medical knowledge related to decision making in laboratory medicine. Such systems will hopefully lead to more cost-effective diag- nostic and therapeutic procedures. It is also obvious that such formal and non-formal descriptions of various aspects of medical decision making will increase the possibilities and facilitate the intricate work to assess quality requirements on laboratory data in a clinical context. As pointed out before (3 , 4 , 7) the discussion on optimal quality of laboratory tests should be broadened to include not only analytical and pre-analytical quality components, but also the equally or more important aspects of e.g. (i) selection and combination of tests; (ii) frequency of specimen collection; (iii) the influence of incorrect conceptual models for in- terpretation of test results; and (iv) mode of communication and presentation of results. Some examples have previously been presented in detail in refs. (3,4,7) and will not be repeated here. Other more recent applica- tions developed within the Nordic R&D Project on Knowledge-Based Systems in Medicine may be found in refs. (11,13,14,17,20). It could be expected that these new techniques for data -, and knowledge-processing and communication will help the clinical laboratory profession in setting standards, guiding training, disseminating information, and establishing collaboration between the laboratory and the clinic (9). 265 PRACTICAL USE OF ANALYTICAL QUALITY SPECIFICATIONS Considering the diversity of approaches advocated for assessing analytical quality specifications, and the circumstance that any quality specification of laboratory results will only be temporary (1) , the "right thing to do" would be to set up a first provisional IfNordic List" as a starting point to a coordinated learning and teaching process on how to apply such quality specifications in practice. Analytical quality specifications expressed as "allowable analytical errors" (AAE) at specified concentration levels or ranges should then be applied for: 1. Method evaluation studies in connection with establishment of an analytical method for routine use (15). The assessment of "stable analytical performancell should lead to an estimation of imprecision and inaccuracy (bias) over the whole measurement ran- ge, and be evaluated in comparision with limits set by the allow- able analytical error. It is worth emphasizing that, according to the basic principles of metrology, the analytical bias (remaining after all measures have been taken to reduce it) should be corrected for in the routine operation of the method. This could be performed with use of a correction function, estimated by regression analysis of "conventional true values" versus corresponding measured values. 2. Internal Analytical Quality Assurance of the routine method to achieve analytical stability within the limits of allowable errors at one or more selected concentration levels. Statistical procedures could be designed to assure the specified quality (10,16), and be optimized in terms of "test yield" (as a measure of "internal laboratory costs") and "defect rate" (as a rough measure of "clinical costsIt). The term internal analytical quality assurance (IAQA) is preferred to the conventional term IQC to denote this new approach. Interactive "IAQA Design Programs" are available on PCs based on Monte Carlo simulation of specified measurement and stat- istical control procedures under realistic analytical pertur- bations, and given analytical quality goals (5,12). 266 Figure 1 shows typical output results from such a program. The optimal number of controls (N) for a given situation could be determined from the plot of test yield and defect rate as func- tions of N, and the condition that the false accept rate (defect rate) should be below a certain limit, e.g. 0.1%. In the example shown this means that 3 controls should be used, giving a test yield of 89%. Predicted outcome of analytical runs is summarized in a decision matrix, from which predictive values for reject and accept signals are calculated for the quality assurance system. * + Ficrure 1. Typical output results from the IAQA Design Program as applied to an analytical method with excellent process stability (1% frequency of critically sized systematic shifts). (a) Plot of test-yield and defect rate versus number of controls (N); 267 ( b ) CHOL e2 OKT.-~O AAE=180.0000 F= 1% SYSTEMATIC; 0 % RANDOM CONTROL RULES: SHEWHART 1 (3.00*S) RULE WITH CONTROL L I M I T S CALCULATED FROM S ( T ) FOR WITH N= 3 SHEWHART 1(3.00*S) RULE PREDICTED OUTCOME O F ANALYTICAL RUNS: ACCEPTABLE ACCEPTABLE ? -__-___-- _______^_--_---_--__--------- ACCEPTED 96.3% .O% REJECTED 1 2.7% .O% NOT ACCEPTABLE .1% .9% P R E D I C T I V E VALUE O F REJECT= . 2 5 P R E D I C T I V E VALUE OF ACCEPT=1.00 S E N S I T I V I T Y = .90 S P E Z I F I C I T Y = . 9 7 TEST YIELD: .90 DEFECT RATE: -1% Ficrure 1. Typical output results from the IAQA Design Program as applied to an analytical method with excellent process stability (1% frequency of critically sized systematic shifts). Decision matrix showing the predicted outcome of analytical runs for a Shewhart 1:3s rule with N=3. (b) It has been suggested that the critically sized systematic error ( SE,) and the critically sized increase in inherent random error ( RE,) (to be detected by the control systems to assure the specified analytical quality) should be calculated from formulas where known analytical bias is subtracted from the allowable analytical error, i.e. SE,=(AAE-bias)/s-1.65 RE,=(AAE-bias)/l.96 s It should be noted that this may lead to unreasonable and unnecessary high demands on process capability (very small critically sized errors and even negative values for bias AAE- 1.65s). 268 - - I +- AAE 1- AAE - C" C Fisure 2. Calculation of process capability, i.e. critically sized systematic error, A S E c , in one case where known bias has not been corrected for (a), and another case where bias is cor- rected for in the routine operation of the analytical method (b) , C*=conventional true value; C=local stable level; s=imprecision of the method. 269 The way out of this dilemma (Fig 2.) is to follow the metrolo- gical principle of correcting measured values for known bias, and design the statistical procedure to detect critically sized errors calculated from A SE,=AAE/s-l. 65 A REC=AAE/1.96's Such a procedure will then assure analytical stability within the AAE limits. In the lack of generally accepted ''conventionally true values",C*, analytical stability within allowable deviations from Illocal stable levelsIl,C, should be the goal. Another related problem of internal analytical quality as- surance to be considered (especially in high production laborato- ries) is the of different analyzers which are used to measure the same components. For more details see ref (10). 3 . External Analytical Quality Assessment & Assurance. Such external programs, administered by a coordinating organi- zation, should be complementary to the IAQA procedures (operating at one or two selected concentration levels) by providing an ass- essment of analytical performance (estimation of analytical bias) over a wider concentration range. As described and discussed in more detail elsewhere (6,10,16) this could be performed by (i) distribution of a sufficient number of Ifaccuracy assessment stan- dards" ( 'Iexternal calibrators") to each participating laboratory, and by (ii) analysis of measured values versus agreed upon "con- ventional true values" with use of e.g. linear regression. The slope (k) and intercept (1) of the regression line could be used to characterize each laboratory in a strict quality assessment/proficiency testing procedure on the basis of allowable analytical error specifications mapped into allowable regions in the k-1 plane. An "EQA&A Simulation Program" has been used to study various strategies for decreasing interlaboratory variation (6), and for optimizing the statistical design. A new interactive version of this program has been developed (Groth and Falk, to be publis- hed). The input data describe the analytical performance of a set of participating laboratories, and the proposed design of the quality assessment & assurance program. The performance of 270 various statistical procedures for assessment of accuracy and testing the agreement with the quality goals may be investigated as well as the possible effect of various correction procedures. The regression function could be used in the individual labo- ratory for routine correction of significant bias, and/or when called for in connection with communication of laboratory results in health care and scientific work. There are certain limitations of the applicability of this approach due to different chemical specificities of the methods used today. Some characterization and standardization of methodologies based on their chemical specificity will certainly be necessary in order to take full advantage of the type of external quality assurance programs discussed here. TRANSMISSION AND TRANSFERABILITY OF LABORATORY TEST RESULTS Communication of laboratory results is still a major problem both in health care and medical research. In the effort by the commission of EC to improve European health care in a coordinated way, standardization and application of information and communi- cation technologies are two important large scale activities. The EUCLIDES project, one of several projects within the CEC AIM Exploratory Project on Advanced Informatics in Medicine, has worked out a proposal for a European standard for transmission of clinical laboratory data, i. e. "patient related data, control- sera related data, test related data, and laboratory related supervisory data" (2) . In order to take full advantage of the emerging technology for transmission of laboratory results between health care units and clinical laboratories via networks (or portable medical "smart- cards") it is necessary to solve the transferability problem by developing the appropriate quality assurance procedures (10). Knowledge-based systems also have great potentials in this area by providing "expert diagnostics and advice" to individual laboratories and health care units concerning analytical problems and how to improve. 27 1 REFERENCES 1. 2 . 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. Barnett RN. Quality control in the clinical laboratory. An individual perspective. Lab Med, 1989; June: 385-387. De Moor GJE. EUCLIDES: A European clinical laboratory data exchange standard. In Proceedings of the AIM-EUROFORUM Conference, Sevilla, 1990. Groth T. Methods and approaches to determine lloptimalll- quality for clinical chemical data. Scand J Clin Invest 1980:40 (Supp1.155): 47-63. Groth T. Biodynamic models as they relate to test selec- tion. In Clinics in Laboratory Medicine, eds. ES Benson, D Connelly and E Burke, vol. 214,701-715, Marcel Dekker Inc., New York and Basel, 1982. Groth T I Falk H and Westgard JO. A quality control simula- tor for design and evaluation of internal quality control procedures. Scand J Clin Lab Invest 1984; 44 (supp1.172): 195-201. Groth T. Strategies for decreasing interlaboratory varia- tion. Studies by computer simulation. Scand J Clin Lab Invest 1984; 44 (suppl. 171): 331-347. Groth T. The role of formal biodynamic models in laboratory medicine. Scand J Clin Lab Invest, 1984; 44 (suppl. 171): 175-192. Groth T. Analytical quality, test quality and quality costs. Upsala J Med Sci 1986; 91: 205-208. Groth T. Database management and knowledge-based systems in clinical laboratory medicine. In Kerkhof, Van Diejen-Visser (eds.) Laboratory Data and Patient Care. Plenum Press, New York 1988; 101-108. Groth T. Internal and external quality assurance. Scand J Clin Lab Invest 1989; 49 (suppl 193): 87-94. Groth T. Quality assurance by knowledge based techniques. In Automation in Blood Transfusion, eds. Smit-Sibinga, Das & Hogman, Kluwer Academic Publishers, Boston, 1989; 83-90. Groth T. and Westgard JO. A second generation simulation program for optimizing quality control design. Clin Chem 1990; 36: 1003. Groth T, Hakman M, Hedlund A, Zaar B. KBSIM/FLUID THERAPY: 272 14. 15. 16. 17. 18. 19. 20. A system for optimized design of fluid resuscitation in trauma. Comp Meth Progr Biom, 1991; 34: in press. Van Waeg G and Groth T. Allopurinol kinetics in humans as a means to assess liver function: design of a loading test. Am J Physiol 1989; 257: R237-R245. Westgard JO. Precision and accuracy: Concepts and assess- ment by method evaluation testing. CRC Critical Reviews in Clinical Laboratory Sciences. 1981; 283-330. Westgard JO, Groth T and de Verdier C-H, Principles for developing improved quality control procedures. Scand J Clin Lab Invest 1984; 44 (suppl. 172): 19-41. Wiener F, Groth T and Nilsson U, A knowledge-based system for automatic interpretation of an analytical profile of complement factors. J Clin Lab Anal 1989; 3: 287-295. Wiener F, Groth T and Mortimer D, Hallquist I and Rane A. A knowledge-based information system for monitoring drug levels. Comp Meth Prog Biom, 1989; 29: 115-128. Wiener F, C-H de Verdier and Groth T. The use of knowledge- based information systems for interpreting specialized clinical chemistry analyses-experience from erythrocyte enzymes and metabolites. Scand J Clin Lab Invest, 1990; 50: Wiener F, Hedlund A and Groth T. A knowledge-based in- formation system for advice in the crisis management of the burn patient. Burn Care Rehab, 1990; 11: 201 247- 259. Correspondence: Torgny Groth, Associate professor, Unit for Biomedical Systems Analysis, Uppsala University, Box 2103, S-750 02 Uppsala, Sweden 18 -908573 273