Accuracy – review of the concept and proposal for a revised definition ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 414 ACTA IMEKO ISSN: 2221-870X December 2020, Volume 9, Number 5, 414 - 418 ACCURACY – REVIEW OF THE CONCEPT AND PROPOSAL FOR A REVISED DEFINITION C. Müller-Schöll1 1 Mettler-Toledo International Inc., Greifensee, Switzerland, christian.mueller-schoell@mt.com Abstract: Accuracy as a concept is widely used in metrol- ogy. It has undergone a historical development and is defined and used differently in different docu- ments, communities and usage situations. Present existing definitions are sometimes not clear, not un- ambiguous and sometimes even not useful. This pa- per explains the difficulties regarding present defi- nitions (section 3 and 4), throws a spotlight on pre- sent day use of the concept (section 5), clarifies the question if accuracy is a qualitative concept (sec- tion 6) and finally presents conceptual ideas and a proposal for a new wording of a sustainable defini- tion for accuracy (section 7). This paper is intended to support the highly estimated work of JCGM WG2. Keywords: Accuracy; Terminology; Metrology 1. INTRODUCTION, MOTIVATION Language is subject to ongoing change, due to its usage and due to its users. The term “accuracy” in the context of metrology has undergone changes in its definition, in its use and in its understanding and application in the past. There is still today uncer- tainty about its proper use and its proper meaning, and there is more than one normative document de- fining accuracy, using different words. It might be time to reflect on the language and it might be time for a clarification resulting in a clear and (for the field of metrology) universally applicable definition. It turns out that two distinguishable concepts are in use: One is taken from the International Vocabu- lary of Metrological Terms and Concepts [1] hence- forth referred to as "the VIM", the other one is more prevalent in standard documents published by the International Organization for Standardization (ISO)1. 2. REVIEW OF THE VIM DEFINITION2 2.1. Text Review The definition of accuracy according to [1] reads: 1 The VIM is not considered an "ISO document" here, although it is also published as "ISO Guide 99" under an ISO name. "2.13 (3.5) measurement accuracy; accuracy of measurement; accu- racy closeness of agreement between a measured quantity value and a true quantity value of a measurand NOTE 1 The concept ‘measurement accuracy’ is not a quantity and is not given a numerical quantity value. A meas- urement is said to be more accurate when it offers a smaller measurement error. NOTE 2 The term “measurement accuracy” should not be used for measurement trueness and the term “measurement precision” should not be used for ‘measurement accuracy’, which, however, is related to both these concepts. NOTE 3 ‘Measurement accuracy’ is sometimes understood as closeness of agreement between measured quantity values that are being attributed to the measurand." Additionally, an "annotation" can be found in the Internet-version of the VIM, reading: "ANNOTATION (informative) [9 June 2016] Historically, the term "measurement accuracy" has been used in related but slightly different ways. Sometimes a single measured value is considered to be accurate (as in the VIM3 definition), when the measurement error is assumed to be small. In other cases, a set of measured values is considered to be accurate when both the measurement trueness and the measurement precision are as- sumed to be good. Sometimes a measuring instrument or meas- uring system is considered to be accurate, in the sense that it provides accurate indications. Care must therefore be taken in explaining in which sense the term "measurement accuracy" is being used. In no case is there an established methodology for assigning a numerical value to measurement accuracy." 2.2. Text analysis The VIM definition uses the concept “closeness of agreement”, however, the term “closeness” is not defined and therefore is subject to interpretation. The definition speaks of the closeness of agree- ment of “a” measured quantity value, so it applies to a single measured value. NOTE 2 mentions a "relation" between accu- racy and precision and between accuracy and true- ness, but leaves open what kind of relation that is. Although accuracy being "not a quantity" [1], a comparative statement like “more accurate" is pos- sible according to NOTE 1. It is left open how two things can be judged “more” and “less”, when being not a quantity. 2 In the following paragraphs we investigate some defini- tions of the term accuracy as examples. Other definitions might exist. http://www.imeko.org/ mailto:christian.mueller-schoell@mt.com ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 415 A relation between "measurement accuracy" and “measurement error” is mentioned in NOTE 1. “Measurement error” in turn is given a relation to “quantity value” in its VIM-definition (2.16), mean- ing that both “measurement error” and “quantity value” are of the same kind. “Quantity value” in turn is used as the first fraction/component of a “meas- urement result” (VIM definition 2.9) where meas- urement result consists of a first fraction/component "value" and a second fraction/component "uncer- tainty associated to it". Consequently, measurement error is clearly a one-dimensional property (it is a one-dimensional quantity value) and not a two di- mensional vector like "measurement result" (with its two fractions/dimensions/components "value" and "uncertainty"). The "Annotation" (2016) that can be found in the Internet-version of the VIM clearly points at ambi- guities of interpretations of the definition, calling them "historical". But in the text of the annotation no hint is given that either of the two interpretations given there is considered outdated, so they still both exist. The two interpretations a) either relate "a" measured value to its error (note "a" value, not "values"), so "smaller error = more accurate" b) or relate "a set of measured values" to both its trueness and its precision (where both the trueness and the precision need to be "good" for the set to be considered "accurate"). But this is not consistent in itself, because even a single value can be attributed both a trueness and also a precision (!) if the distribution is known from which this single value is drawn (type A uncertainty of a single measured value). So the relation of "a" measured value to its trueness and its precision is missing here. The NPL has published interpretations referenc- ing to the VIM-definition where accuracy is inde- pendent from precision [2] and where a set of meas- urements (!) is said to have "high accuracy" while having "low precision" at the same time. The con- sistency with case b) mentioned above appears un- clear or not being given. 2.3. Conclusion regarding VIM With regards to the VIM definition, it is remains unclear how accuracy is defined and how accuracy can be expressed. It is said to be "not a quantity", but VIM does clearly not state it be "qualitative". Accuracy can be judged "more accurate [than]" but accuracy cannot be assigned a value. It is unclear how this goes together. Accuracy is stated to relate to "error" (smaller error = more accurate), however an additional property like dispersion or variance or uncertainty is not an explicitly mentioned part of this definition (only in the notes). On the other hand, accuracy is said to be "related to both" trueness and precision (NOTE 2), where "precision" (VIM 2.15) is clearly not related to error, but to dispersion. This is a contradiction in the VIM in the view of the au- thor. Since the idea of accuracy being applied to more than one value and the idea of dispersion of values are not visible in the VIM definition text (not con- sidering the non-normative Annotation 2), the com- monly used dartboard model with dispersing hits cannot be considered an appropriate illustration for the definition of accuracy according to the VIM. Accuracy and precision are on the same hierar- chical level in this concept (they are like apples and pears). Within this concept it is possible that a result can be attributed e.g. “accurate but not precise” which can be found in publications (e.g. [2]). 3. REVIEW OF THE "ISO DEFINITION" 3.1. Text review ISO (ISO 5725-1:1994) The title of the standard ISO 5725-1 [3] is: “Ac- curacy (trueness and precision) of measurement methods and results". This standard gives definitional phrases in two different locations: One rather explanatory (yet def- initional in character) in section 0.1, and a definition in section 3.6. (in chapter 3 named "Definitions"): Section 0.1 of ISO 5725-1 uses two terms "true- ness" and "precision" to describe the accuracy of a measurement method. "Trueness'' refers to the "closeness of agreement between the arithmetic mean of a large number of test results and the true or accepted reference value". "Precision" refers to the "closeness of agreement between test results". Additionally, we find a so-declared "definition" in the same standard: 3.6 accuracy: The closeness of agreement be- tween a test result and the accepted reference value. NOTE 2 The term accuracy, when applied to a set of test results, involves a combination of random components and a common systematic error or bias component [emphasis by the author]. 3.2. Text Analysis – ISO 5725-1:1994 Section 0.1 of ISO 5725-1:1994 According to section 0.1 of [3], it is both “true- ness” and “precision” that are used to form accuracy. The text of 0.1 refers to accuracy as a property of a measurement method (and not of a single result and not of a set of results). This is a clear, yet under- standable difference to the VIM concept since the focus of [3] is methods. Both trueness and precision are explained using the words “a large number of” results or at least “re- sults” (in plural form) in [3]. According to this, ac- curacy, as explained here, cannot be applied to a sin- gle measurement result, since the language does not cover a single result (again in contradiction to VIM). http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 416 Precision is not the same as uncertainty. So, ac- curacy – according to this concept – is not related to what is usually perceived as being the quality of a "measurement result" (VIM 2.9) which consists (VIM 2.9 NOTE 2) of "a measured quantity value and a measurement uncertainty" [emphasis by au- thor] and is thus a two-dimensional statement. The two dimensions of "measurement result" in- troduced in this definition are: • The first dimension, either the measured quantity value or the measurement error (with reference to a reference value) calculated from it and, • a second dimension characterizing the dispersion which is the uncertainty associated with the first component. The wording of 0.1 of [3] is in contradiction with the title of the standard, which announces both the accuracy of "methods and results". The accuracy of results is neither explained nor even mentioned here. Section 3.6 of ISO 5725-1:1994 In section 3.6 of the same standard, a definition is given for accuracy. This definition is the same as the VIM definition which refers to “a” test result (singular form) and to a “test result”. Methods are not mentioned in the definition. However, a note (NOTE 2) has been added, broadening the scope to “a set of … results”. This circumstance in the note is inconsistent with the definition. The note is not in alignment with the definition and a definition that can be applied to the accuracy of a method is miss- ing in the whole standard despite its title. Additionally, the restriction "when applied to a set of results" is misleading (or maybe even wrong), since also a single value can be assigned a disper- sion originating from the population this single value was drawn from (as explained above). According to 3.6 of [3], accuracy embeds both trueness and precision. This defines a hierarchy: Trueness and precision are on the same level and accuracy combines both, defining a parent hierar- chical level. (Accuracy is like fruits and trueness and precision are like apples and pears.) A two-dimensional dartboard model is fre- quently used to visualize the concept depicting true- ness and precision with a set of hits (by means of centering of the mean value and spread of the hits). 3.3. Conclusion regarding ISO 5725-1:1994 It seems that in the definition of accuracy given in [3], the authors intended to take both, a "deviation component" and a "spread component" into account. However, instead of measurement uncertainty, the precision was chosen to describe what was missing when accuracy "at one time" (quoted from section 0.6) contained only what we today call trueness. Nevertheless, the different statements within the standard are hard to match (if not impossible to match). Note 2 broadens the scope of the definition to "a set of results" which is not consistent with the mere text of the definition. 3.4. Text review ISO IWA 15 ISO IWA 15 [4] reproduces the VIM definition for accuracy. However, in a specific definition of "uncertainty" (section 3.1.3), we find the following phrase in a note: "Uncertainty is inversely related to accuracy, and is a quantity value." This is one of the rare occasions that in the defining literature uncer- tainty comes explicitly into play in a relation to the concept of accuracy. Nevertheless, it is neither de- tailed, how this "relation" looks like nor how a re- lation would lead from a non-quantitative accuracy to a quantitative uncertainty nor if and how "in- versely" is to be understood mathematically. Chapter 5.1 of [4] reads: "Accuracy may be im- proved by improving precision and trueness." This hints again at the concept of [3] where accuracy is (whatever kind of) a combination trueness and pre- cision and accuracy is on a higher hierarchical level than trueness and precision (please note that at this point, no connection to "uncertainty" is made). However, immediately afterwards, we read "Ac- curacy, precision and trueness are conceptual terms. Quantitative expressions of these concepts are given in terms of uncertainty, random error and sys- tematic error, respectively." This is clarifying accu- racy: Accuracy is said to be quantified by uncer- tainty, which in turn is said to be combined of sys- tematic and random errors. The latter of course is wrong since uncertainty according to the concept of the GUM [5] is not just a combination of random and systematic "errors". (In order to clearly separate these conceptual areas, the GUM has introduced "type A" and "type B" uncertainties and does not support the wording quoted from [4] above.) Later in [4], accuracy is even expressed as a quantity value (e.g. in Table 2, column name: "typi- cal accuracy", column content is values). This is in clear contradiction to the definition given in the same document. And also later in the standard (sec- tion B.6.1.4) "accuracy" and "precision" are men- tioned on the same hierarchical level, which again is an internal contradiction in the same document and in section B.7.1 systematic error is equalized to accuracy which also contradicts internally in the document. 3.5. Conclusion regarding ISO IWA 15 Also [4] does not clarify the matter. It adds ideas of uncertainty to the concept of accuracy, however, also appears not fully internally consistent. 4. CURRENT USE OF THE TERM ACCURACY A recent high-level metrology publication adds an interesting view to the discussion: It is the paper titled "Evaluation of the accuracy, consistency, and http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 417 stability of measurements of the Planck constant…" by A Possolo, S Schlamminger et al. [6]. In section 3 of the paper (section named "Accuracy"!), the au- thors detail on the "accuracy requirements" of the CCM regarding the re-definition of the kilogram. In fact, the requirements made by the CCM are not given this specific name ("accuracy requirements") in the original CCM document [7]. In the publica- tion, these requirements and the values that are com- pared to them are solely expressed in terms of un- certainties. (It is obvious that "errors" are no subject in this field of work.) The only accuracy measure (quantification) the authors take into consideration (and relate to the title of the publication) is uncer- tainty. This is an encouraging indicator that according to present recognized metrologists, uncertainty must be at least part of the concept of accuracy. 5. QUALITATIVE OR QUANTITATIVE? NOTE 1 to the VIM definition of accuracy states "The concept ‘measurement accuracy’ is not a quantity and is not given a numerical quantity value.". This statement (being "not a quantity") ob- viously tempts a number of authors to make state- ments of accuracy being "qualitative", e.g. [2] and [8]. However, this is not what is said in the referred VIM text and is probably also not the intention of the wording of the VIM. There is a fine yet signifi- cant difference between being "not a quantity" and being "qualitative": Accuracy is said to be "not a quantity" [1] and not being given "a quantity value". This, according to the understanding of the author, wants to express that – in in simple words – there is no globally ac- cepted number scale and no unit to express accuracy to make accuracy a metrologically traceable or met- rologically comparable property (= a quantity). This NOTE statement should probably separate the con- cept of accuracy from concepts like "quantity" or "uncertainty", where there is a common, global un- derstanding how to quantify these. However, there is clearly no statement and no hint anywhere in the VIM definition that accuracy be qualitative! Common usage of "accuracy" requests compar- ative statements like "more" or "less" accurate which indicates the necessity to give accuracy "some kind of number" (a value), at least in a given situation or for a given purpose. This gives room for a user to apply a purpose ori- ented algorithm to get and assign an indicative ac- curacy number which allows comparative state- ments in a given situation. 6. SYNTHESIS, PROPOSAL FOR A SOLUTION It appears that the concept "accuracy" as we find it today in the VIM and in ISO documents is a left- over of an ongoing, not yet completed, development. It is stated that "at one time" ([3], 0.6) the concept of accuracy, being perceived as one-dimensional (only related to measurement error), was amended by the concept of precision as additional infor- mation to take account of the possible dispersion of values as a second dimension. Probably, the concept of uncertainty was not yet fully established at that time. However today, according to the VIM, it is ex- actly measurement uncertainty that is "characteriz- ing the dispersion of the quantity values being at- tributed to the measurand" ([1] 2.26), which is ex- actly what should be used if a "dispersion dimen- sion" is to be considered in the concept of accuracy in addition to a "trueness dimension". In addition, it is historically obvious and inten- tional, that there is not one metrologically traceable and metrologically comparable ([1] 2.46) way of quantifying accuracy. Yet, it is necessary in practice that accuracy can be quantified in order to compare or rank. These quantifications may be done using an algorithm (e.g. a mathematical equation) which may follow a purpose given by the specific situation. Therefore, we propose the introduction of a new definition for “accuracy” which should take into account: • The modern understanding of a measurement re- sult, consisting of a "measured quantity value and a measurement uncertainty" (VIM 2.9, NOTE 2), • backwards compatibility still allowing to com- bine accuracy from a combination of trueness and precision, • the possibility of applying the concept of accu- racy to a single value and to a set of values and to a method and to a procedure, • clarifying that "not a quantity" does not mean "being qualitative", • the fact that there may be (various possible) ways of assigning values to accuracy for making com- parative statements. This could be realized with the following word- ing which is submitted to further discussion and consideration: http://www.imeko.org/ ACTA IMEKO | www.imeko.org December 2020 | Volume 9 | Number 5 | 418 Accuracy (measurement accuracy, accuracy of measurement) Term to describe the closeness of agreement between one or several measurement results and a true quantity value. Accuracy consists of a combination of a trueness property and a dispersion property. Any algorithm to combine these two to yield a quantification, may follow its intended purpose. Quantifications of accuracy which origin from the same algorithm may offer comparability (statements like "more accurate" or "less accurate" are then possible). Note 1: The trueness property is preferably the meas- urement error (VIM 2.16) and the dispersion property is preferably the measurement uncertainty (VIM 2.26). Note 2: Accuracy according to this definition can be applied, if the necessary information is available, to a set of measurement results, to a single measurement result and to a measurement method and to a measurement pro- cedure. Note 3: Possible algorithms can be summations of e.g. measurement error and measurement uncertainty in quadrature or considering the absolute of measurement error and the quadrature of measurement uncertainty etc. Also a two-dimensional dartboard model can used to illustrate the concept, see Figure 1: Figure 1: Accuracy dart board model; x-axis: improving dispersion, y-axis: improving trueness The x-axis is labelled "improving dispersion", the y-axis is labelled "improving trueness". It is left to the user whether "dispersion" is substituted by "precision" or by "uncertainty". The definition en- compasses both concepts, which is no problem due to the fact that according to the definition proposed above, accuracy does neither deliver mathemati- cally unambiguous nor metrologically traceable or comparable figures. It is admitted that the above proposed definition is rather lengthy and it would be desirable that defi- nitions were brief and clear without "Notes". How- ever, given the history of this concept and the ambi- guities explained in this paper, it appears to be nec- essary to use more words to connect the new word- ing to its history and to clarify misconceptions. 7. SUMMARY In this paper, we have analysed current use of the concept "accuracy". As means of example, we in- vestigated three normative documents in which ac- curacy is defined. We have detected various incom- patibilities, even within the same document but also in the community using this concept. We tried to un- derstand the historical development that might have affected the understanding of this concept and con- clude with a synthesis proposal for a new definition that encompasses historical as well as modern con- cepts and thus offers full backwards-compatibility. 8. REFERENCES [1] JCGM 200:2012: International vocabulary of me- trology – Basic and general concepts and associated terms (VIM) 3rd edition 2008 version with minor corrections (including "VIM Definitions with In- formative Annotations", last updated 29 April 2017), accessed 06/2020 [2] NPL: The difference between ACCURACY & PRECISION https://www.npl.co.uk/skills-learn- ing/outreach/school-posters/npl-schools-poster-_- accuracy-precision-v7-hr-nc.pdf (accessed 06/2020) [3] International Organization for Standardization ISO 5725-1:1994 Accuracy (trueness and precision) of measurement methods and results, Geneva. Swit- zerland [4] International Organization for Standardization ISO IWA 15:2015 Specification and method for the de- termination of performance of automated liquid handling systems, Geneva. Switzerland [5] JCGM 100:2008. Evaluation of measurement data – Guide to the expression of uncertainty in meas- urement (GUM 1995 with minor corrections) [6] A Possolo, S Schlamminger et al. Evaluation of the accuracy, consistency, and stability of measure- ments of the Planck constant… Metrologia 55 29 [7] Recommendation of the Consultative Committee for Mass and related Quantities submitted to the In- ternational Committee for Weights and Measures; Recommendation G 1 (2013) On a new definition of the kilogram https://www.bipm.org/cc/CCM/Al- lowed/14/31a_Recommenda- tion_CCM_G1(2013).pdf (accessed 06/2020) [8] NIST Engineering Statistics Handbook https://www.itl.nist.gov/div898/hand- book/mpc/section1/mpc113.htm (accessed 01/2020) http://www.imeko.org/ https://www.npl.co.uk/skills-learning/outreach/school-posters/npl-schools-poster-_-accuracy-precision-v7-hr-nc.pdf https://www.npl.co.uk/skills-learning/outreach/school-posters/npl-schools-poster-_-accuracy-precision-v7-hr-nc.pdf https://www.npl.co.uk/skills-learning/outreach/school-posters/npl-schools-poster-_-accuracy-precision-v7-hr-nc.pdf https://www.bipm.org/cc/CCM/Allowed/14/31a_Recommendation_CCM_G1(2013).pdf https://www.bipm.org/cc/CCM/Allowed/14/31a_Recommendation_CCM_G1(2013).pdf https://www.bipm.org/cc/CCM/Allowed/14/31a_Recommendation_CCM_G1(2013).pdf https://www.itl.nist.gov/div898/handbook/mpc/section1/mpc113.htm https://www.itl.nist.gov/div898/handbook/mpc/section1/mpc113.htm