Cultura e Scienza del Colore - Color Culture and Science 59 Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 ISSN 2384-9568 Is it possible to improve the weighting function for lightness in the CIEDE2000 color-difference formula? ABSTRACT We have compared the performance of the CIEDE2000 color-difference formula (∆E00) with three CIEDE2000-modified formulas: 1) ∆E00_M1, which incorporates a new V-shaped function proposed at the University of Leeds (UK) with a minimum at the specific lightness of the background; 2) ∆E00_M2, a formula where the original SL function in ∆E00 was replaced by SL=1, as proposed by the CIE94 color-difference formula; 3) ∆E00_M3, a formula developed by us, with the same structure than ∆E00, but avoiding its original SL function by replacing the lightness differences in CIELAB by a new lightness definition based on Whittle formula. Our comparison used the STRESS index and thirteen visual datasets (CIE 217:2016), including filtered subsets to test the symmetry of the SL function proposed by ∆E00. None of the three mentioned CIEDE2000-modified formulas performed statistically significantly better than the original ∆E00 formula for any of the mentioned datasets or subsets, with only one exception (∆E00_M2 formula, Witt dataset). Therefore, the replacement of the SL function in ∆E00 by SL=1 is not recommended. ∆E00_M1 and ∆E00_M3 improved ∆E00 for most datasets, but such improvements were not statistically significant. Results for color pairs with average L* values below and above 50 were not statistically significant different for neither ∆E00, ∆E00_M2 and ∆E00_M3 formulas. It is interesting to note that for eight of the thirteen visual datasets there were no color pairs with average L* values below 25, which claims for future studies using darker color pairs. KEYWORDS Color difference, Color-difference formula, CIE94, CIEDE2000, Whittle formula CITATION: Melgosa M., Cui G., Oleari C., Pardo P.J., Huang M., Li C., Luo M. R. (2018) ‘Is it possible to improve the weighting function for lightness in the CIEDE2000 color-difference formula?’, Cultura e Scienza del Colore - Color Culture and Science Journal, 10, pp. 59-65, DOI: 10.23738/ccsj.i102018.07 Received 16 July 2018; Revised 13 August 2018; Accepted 07 September 2018 Manuel Melgosa is a full professor in optics and current representative of Spain at CIE Divisions 1 and 8. He was Secretary, Vice-president and President of the Color Committee of Spain (SEDOPTICA). Member of the editorial board of the journals Color Research and Application (USA), Cultura e Scienza del Colore (Italy), Atti della Fondazione ‘Giorgio Ronchi’ (Italy), and Luces (Spain). Guihua Cui, full professor in colour science, received his B.S. and M.S. degrees in optical engineering from Beijing Institute of Technology (China), and Ph.D. degree in colour science from University of Derby (UK) in 2000. He was fully involved in the development of the CIEDE2000 color-difference formula. Claudio Oleari passed away on January 23rd 2018. Emeritus professor at Università degli Studi di Parma, emeritus member of the Italian Society of Optics and Photonics (SIOF), founder and emeritus member of Gruppo del Colore. Editor and co-author of the book “Misurare il Colore” (Hoepli, 2015), and author of the book “Standard Colorimetry” (John Wiley & Sons, 2016). Pedro J. Pardo earned his Ph.D. in physics at University of Extremadura (Spain) in 2004, and currently works as an associate professor at the same University. He served as member of CIE TC 1-90 “Colour Fidelity Index” and co-authored CIE 224:2017. Min Huang earned her Ph.D. degree in 2008 from National Lab of Color Science and Engineering at Beijing Institute of Technology (China). She has undertaken some projects mainly focused on color difference evaluation and colorimetric observer categories and their applications. Changjun Li is a full professor at the Department of Computer Science, University of Science and Technology Liaoning (China). He received his B.Sc. (1979), M.Sc. (1982), and Ph.D. (1989) in computational mathematics from Peking University (China), Chinese Academy of Science, and Loughborough University (UK), respectively. Ming Ronnier Luo is a CIE Vice-President, a full professor at the College of Optical Science and Engineering, Zhejiang University (China), a part-time professor at the Taiwan University of Science and Technology, and a visiting professor at the University of Leeds (UK). Recipient of AIC Judd Award 2017 for his important work in the field of color and imaging science. 1Manuel Melgosa mmelgosa@ugr.es [corresponding author] 2Guihua Cui guihua.cui@foxmail.com 3Claudio Oleari claudio.oleari@fis.unipr.it 4Pedro J. Pardo pjpardo@unex.es 5Min Huang huangmin@bigc.edu.cn 6Changjun Li cjliustl@sina.com 7,8Ming Ronnier Luo m.r.luo@leeds.ac.uk 1 University of Granada, Department of Optics, Faculty of Sciences, 18071 Granada, Spain 2 University of Wenzhou, Department of Information & Communication Engineering, School of Physics and Electronic Information Engineering, Wenzhou 325035, China 3 Università degli Studi di Parma, Department of Physics, I-43100 Parma, Italy 4 University of Extremadura, Department of Physics, Faculty of Sciences, 06071 Badajoz, Spain 5Beijing Institute of Graphic Communication, Beijing 102600, China 6University of Science and Technology Liaoning, School of Electronics and Information Engineering, Anshan 114051, China 7Zhejiang University, State Key Laboratory of Modern Optical Instrumentation, Hangzhou, China 8University of Leeds, Leeds LS2 9JT, United Kingdom 60 R. Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Melgosa M., Cui G., Oleari C., Pardo P.J., Huang M., Li C. and Luo M. R. ISSN 2384-9568 DOI: 10.23738/ccsj.i102018.07 1. INTRODUCTION Amongst the five corrections to CIELAB proposed by the current CIE-ISO recommended color- difference formula CIEDE2000 (ISO/CIE, 2014; CIE, 2001; Luo, Cui and Rigg, 2001), it can be said that the weighting function for lightness (symbol SL), also called lightness tolerance, has been the most controversial one in recent literature (Melgosa et al., 2017). The CIEDE2000 color-difference formula (symbol ∆E00), proposed a V-shaped symmetrical function SL with a minimum at L *=50 (the assumed lightness of the background), accounting for the so-called ‘crispening effect’. In the current paper, from 13 experimental datasets (7420 color pairs) previously employed by the CIE Technical Committee 1-55 (CIE, 2016), we have used the Standardized Residual Sum of Squares (STRESS) index (García et al., 2007) to test the performances of the SL function proposed by CIEDE2000, as well as three CIEDE2000- modified color-difference formulas. Low STRESS values, always in the range 0-100, indicate better performance of a color-difference formula (i.e. better predictions of average visually-perceived color differences reported by real observers with normal color vision). The CIEDE2000 color-difference formula (ISO/ CIE, 2014; CIE, 2001; Luo, Cui and Rigg, 2001) is given by: , (1) where the three parametric factors will be assumed in this paper as kL=kC=kH=1.0 (i.e. the so-called ‘reference conditions’), and the weighting function for lightness, SL, is given by: . (2) It can be noted that the CIEDE2000 color- difference formula in Eq. 1 is not an Euclidean distance, because of the chroma-hue interaction term controled by the RT factor, which is often designated in the literature as the rotation term. The following three CIEDE2000-modified color- difference formulas will be considered in this paper: 1) The ∆E00_M1 color-difference formula, analogous to ∆E00 in Eq. 1, but using the next SL´ function, proposed by researchers at the University of Leeds (UK) (Ho, 2006): , (3) where Lb is the specific CIELAB lightness of the background for each visual dataset. 2) The ∆E00_M2 color-difference formula, also analogous to ∆E00, but replacing the SL function in Eq. 2 by SL = 1, as proposed by the CIELAB and CIE94 (CIE, 2004) color-difference formulas. In a recent paper, R.S. Berns has recommended to introduce this modification in CIEDE2000 (Berns, 2016). 3) Finally, ∆E00_M3 is a color-difference formula recently developed by us (Melgosa et al., 2017), with the same structure than ∆E00, but avoiding its original SL function. Specifically, the ∆E00_M3 color-difference formula proposes the replacement of the ratio ∆L*⁄SL in Eq. 1 by ∆Lw , where Lw is the next new definition of lightness, based on Whittle formula (Whittle, 1992): (4) (5) where Y is the luminance factor in the range 0-100; i.e. relative colorimetry (CIE, 2004). For each one of the 13 visual datasets mentioned before, in addition to the complete datasets we have also considered different subsets: Color pairs with almost only lightness differences (specifically, color pairs with│∆L*⁄∆E*ab│>0.9), where ∆E * ab is the color-difference in CIELAB units), and color pairs with average L* values below and above 50, the assumed lightness of the background in ∆E00 (see Eq. 2). Table 1 shows the percentage of color pairs in five different subsets, for each of the 13 visual datasets considered here. Excluding the two ‘region specific’ datasets with only blue (Lee et al., 2011) and black (Shamey et al., 2014) color pairs, Table 1 shows that the percentage of color pairs below and above L*=50 is balanced well enough (averages of 42.2% vs. 57.8%). However, it is noticeable that the percentage of color pairs with L*≤25 is considerably low: More specifically, Table 1 shows that there were 61 Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Is it possible to improve the weighting function for lightness in the CIEDE2000 color-difference formula? ISSN 2384-9568 no color pairs with L*≤25 for 9 of the 13 tested datasets, and the percentage in this range was 100% for the NCSU-Black (Shamey et al., 2014) dataset. 2. RESULTS 2.1. COMPLETE DATASETS Table 2 shows STRESS values for each of the 13 visual datasets and different color-difference formulas. Specifically, Table 2 shows the STRESS results (García et al., 2007; CIE, 2016) for the original ∆E00 color-difference formula in column 2, three modifications of this formula previously described (∆E00_M1, ∆E00_M2 and ∆E00_M3) in columns 3-5, and the results found by the original ∆E00 formula modified by a power function with exponent 0.70 (column 6), which produced particularly good results in a previous work (Huang et al., 2015). The last two columns in Table 2 are the critical F-values to be considered at a 95% confidence level in order to test the statistical significance of the differences between two given color-difference formulas (García et al., 2007; CIE, 2016). Cells filled with blue/yellow color in columns 3-6 of Table 2 mean that the formula indicated in the header row performs better/worse than the original ∆E00 formula. In addition, numbers in bold/italic fonts in columns 3-6 of Table 2 indicate that there were statistically significant/ non-significant differences (95% confidence level) between the formula in the header row and the original ∆E00 formula. Table 1 - Percentages of color pairs with mainly lightness differences (column 2) and average CIELAB lightness (L*) in four different intervals (columns 3-6), for each one of the 13 visual datasets considered in this paper. From Table 2 it can be noted that for the three CIEDE2000-modified formulas (columns 3-5) there is only one case with statistically significant better performance than the original ∆E00 formula: The case of the ∆E00_M2 color- difference formula in Witt’s dataset (Witt, 1999), with a cell filled with blue color and a STRESS value of 27.4 in bold font. This situation is in contrast with the highly satisfactory results achieved by using a power function with exponent 0.70 in ∆E00 (Huang et al., 2015), which produced statistically significant improvements upon the original ∆E00 formula for 9 of the 13 datasets, as indicated by the cells filled in blue color with numbers in bold font in column 6 of Table 2. Currently, it can be said that the use of power functions is the most effective way to improve the performance of advanced color-difference formulas (Huang et al., 2015). 2.2 COLOR PAIRS WITH MAINLY LIGHTNESS DIFFERENCES Table 3 shows analogous results to those in previous Table 2, but filtering each one of the original datasets to consider only the color pairs with │∆L*⁄∆E*ab│>0.9 (i.e. color pairs with almost only lightness differences). Because we are interested on studying the weighting function for lightness in ∆E00, results shown in Tables 2 and 3 allow us to check whether the performances of ∆E00 and the remaining proposed color-difference formulas are or not different for the complete datasets (Table 2) and their corresponding filtered subsets with color pairs exhibiting almost only lightness Table 2 - STRESS values for 13 visual datasets (column 1) and five color-difference formulas (columns 2-6, see text). Cells filled with blue/ yellow colors indicate better/worse performance than the one achieved by using the ∆E00 color-difference formula, while numbers in bold/italic font mean significant/non-significant differences with respect to ∆E00, from specific FC and 1⁄FC critical values shown in last two columns assuming a significance level of 95%. 62 R. Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Melgosa M., Cui G., Oleari C., Pardo P.J., Huang M., Li C. and Luo M. R. ISSN 2384-9568 DOI: 10.23738/ccsj.i102018.07 differences (Table 3). From Table 3 it can be noted that none of the three CIEDE2000-modified color-difference formulas (∆E00_M1, ∆E00_M2 and ∆E00_M3) reported statistically significant better results than ∆E00 for any of the 13 datasets (i.e. there are no cells filled in blue color and numbers in bold font in columns 3-5 in Table 3). Comparing Tables 2 and 3, it can be stated that the performance of all tested color-difference formulas (except ∆E00_M3) with respect to the original color-difference formula ∆E00 is slightly worse for color pairs with almost only lightness differences (Table 3) than for complete datasets (Table 2). From Tables 2 and 3 it can be also added that color-difference formulas ∆E00_M1 and ∆E00_M3 (but not ∆E00_M2) improved ∆E00 for most datasets, but such improvements never (except for 1 dataset) were statistically significant. Tables 2 and 3 show that for a majority of datasets the ∆E00_M2 formula was significantly or non-significantly worse than the ∆E00 color-difference formula, which indicates that, beside the recommendation made by Berns (Berns, 2016), for current experimental datasets the replacement of the SL function in ∆E00 by SL=1 is not advisable. Table 3 - Idem to Table 2, but for color pairs with mainly lightness differences: │∆L*⁄∆E*ab│>0.9. 2.3. COLOR PAIRS WITH AVERAGE L* BELOW AND ABOVE 50 In this subsection our goal is to check the symmetry of the SL function proposed by CIEDE2000 with respect to the assumed lightness of the background, L*=50 (see Eq. 2). Tables 4 and 5 show STRESS results found using color pairs in complete datasets with average L* values (L*) below and above 50, respectively. The colors and fonts codes in the cells of Tables 4 and 5 are the same used in previous Tables 2 and 3. It can be noted that Tables 4 and 5 discarded the use of the ∆E00_M1 color-difference formula (and also the ∆E00 color-difference formula modified by the exponent 0.70), because this formula considers the lightness of the backgrounds in each dataset in place of (L*)=50 (Ho, 2006). Tables 4 and 5 also missed the NCSU-Blue (Lee et al., 2011) and NCSU- Black (Shamey et al., 2014) datasets because all color pairs in these datasets had (L*) values below 50 (see Table 1). From Tables 4 and 5 we can see that both, ∆E00_M2 and ∆E00_M3 were never statistically significantly better than ∆E00. It can be also noticed that the performance of these two CIEDE2000-modified color-difference formulas Table 4 - STRESS values for color pairs with average L* values above 50, considering 3 color-difference formulas and 11 visual datasets. The colors of the cells and the fonts for numbers in current Table 4 follow the same codes used in previous Tables 2 and 3 (see text). Table 5 - Idem to Table 4, but for color pairs with average L* values below 50. 63 Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Is it possible to improve the weighting function for lightness in the CIEDE2000 color-difference formula? ISSN 2384-9568 with respect to ∆E00 is better for color pairs with L* values below 50 (Table 5) than for color pairs with L* values above 50 (Table 4), but this result must be interpreted with prudence, because the number of available color pairs in the range L*≤25 is very small (see Table 1), and this difference is not statistically significant (see subsection 2.5). 2.4. NORMALIZED SCATTER PLOTS FOR COLOR PAIRS WITH MAINLY LIGHTNESS DIFFERENCES Figures 1 and 2 show the normalized ratios ∆E*ab⁄∆V for color pairs with │∆L *⁄∆E*ab│>0.9 (i.e. color pairs with mainly lightness differences) against average L* values below and above 50, respectively, for different visual datasets. Specifically, we considered 11 visual datasets, discarding from the 13 initial ones (Table 1) the two datasets with color pairs in only one region of the color space (Lee et al., 2001; Shamey et al., 2014). The normalization used in Figures 1 and 2 was to divide the mentioned ratios by its average in each individual dataset, as made by Berns (Berns, 2016). Figures 1 and 2 also show the two branches of the SL (V-shaped) function, corresponding to the predictions made by the original ∆E00 color-difference formula (ISO/CIE, 2014; CIE, 2001; Luo, Cui and Rigg, 2001). As in plots reported by Berns (Berns, 2016), Figures 1 and 2 show a considerable scatter. Therefore, in general, it cannot be stated that the SL function (Eq. 2) proposed by ∆E00 (Eq. 1) is a good predictor of experimental results for color pairs with almost only lightness differences in most currently available visual datasets. Figure 1 also shows that there are few pairs in the range L*≤25 (see Table 1), which claims for future studies using a higher number of dark color pairs. Figure 1 - Normalized ratios ∆E*ab⁄∆V for experimental color pairs with │∆L*⁄∆E*ab│>0.9 (i.e. color pairs with mainly lightness differences) in 11 visual datasets, against average L* values of color pairs in the range 0-50. The predictions made by the ∆E00 color- difference formula are indicated by the black line. Figure 2 - Idem to Figure 1, but for color pairs with average L* values in the range 50-100. 64 R. Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Melgosa M., Cui G., Oleari C., Pardo P.J., Huang M., Li C. and Luo M. R. ISSN 2384-9568 DOI: 10.23738/ccsj.i102018.07 2.5. SUMMARY OF RESULTS FOR COMPLETE DATASETS AND SUBSETS Table 6 summarizes the results shown in previous Tables 2-5, indicating the number of datasets (from a total of 13 or 11, depending on the row considered in such Table) with better or worse results than those achieved by the original ∆E00 color-difference formula, considering also statistically and not statistically significant differences at 95% confidence level. Note that, for the three CIEDE2000-modified formulas (∆E00_M1, ∆E00_M2 and ∆E00_M3) statistically significant better results than those found by using ∆E00 were found only for one dataset (Witt, 1999). However, the ∆E00 color-difference formula modified by an exponent 0.7 (Huang et al., 2015) produced very good results for all complete datasets (i.e. statistically significant or non-significant improvements for all datasets), and also good results for the subsets of color pairs with mainly lightness differences. In general, we can note that highest values in Table 6 are located in column 4, which means that for most datasets the CIEDE2000-modified formulas tested in the current paper were better, but, unfortunately, not statistically significantly better, than the original ∆E00 color-difference formula. With respect to the symmetry around L*=50, it can be noted in Table 6 that the CIEDE2000- modified formulas ∆E00_M2 and ∆E00_M3 performed better than the original ∆E00 formula in a majority of datasets for color pairs with average L* below 50, but not for color pairs with average L* above 50. However, from Wilcoxon sign rank test, the medians of STRESS values for color pairs with average L* values above and below 50 (Tables 4 and 5) were not statistically significantly different for ∆E00 (p=0.240), ∆E00_M2 (p=0.966), and ∆E00_M3 (p=0.240). 3. CONCLUSIONS The analyses presented in the current paper are complementary to those reported in Melgosa et al., 2017. From most currently available experimental datasets, our analyses do not allow a successful alternative proposal to the SL function proposed by ∆E00, because in most cases the improvements achieved by the candidate formulas are not statistically significant. In particular, our results show that replacing the SL function proposed in ∆E00 by SL=1, as done by CIELAB and CIE94 (CIE, 2004), and also recently suggested by Berns (Berns, 2016), is not a good choice. However, the replacement of CIELAB lightness L* by another lightness function, shown by the Eqs. 4 and 5, based on Whittle formula (Whittle, 1992), leads to promising results. Figures 1 and 2 also indicate that the SL function proposed by ∆E00 is not a satisfactory definitive result. However, it must be added that ∆E00 was recommended for a specific set of viewing conditions, the so- called ‘reference conditions’ (ISO/CIE, 2014; CIE, 2001; Luo, Cui and Rigg, 2001), which are not identical to those employed in all the visual datasets considered in the current paper (Table 1). We hope that advances on new color spaces, in particular those with physiological basis, as well as a sounder knowledge of the influence of specific viewing conditions (parametric factors) on perceived color differences will lead to future color-difference formulas with improved performance. Table 6 - Number of datasets or subsets with different kind of differences (columns 3-6) between several CIEDE2000-modified color- difference formulas (see text) and the original ∆E00 color-difference formula. 65 Cultura e Scienza del Colore - Color Culture and Science | 10 | 2018 | 59 - 65 Is it possible to improve the weighting function for lightness in the CIEDE2000 color-difference formula? ISSN 2384-9568 CONFLICTS OF INTEREST The authors declare no conflict of interest affecting the results reported in this paper. FUNDING This work was supported by Ministry of Economy and Competitiveness of the Government of Spain, research projects FIS2013-40661-P and FIS2016-80983-P, with support from European Regional Development Fund (ERDF). BIBLIOGRAPHY Berns, R. S., Alman, D. H., Reniff, L., Snyder, G. D. and Balonon-Rosen, M. R. (1991). ‘Visual determination of suprathreshold color-difference tolerances using probit analysis’. Color Research and Application 16 (5), pp. 297- 316. doi:10.1002/col.5080160505 Berns, R. S. and Hou, B. (2010). ‘RIT-DuPont supra- threshold color-tolerance individual color-difference pair dataset’. Color Research and Application 35 (4), pp. 274- 283. doi:10.1002/col.20548 Berns, R. S. (2016). ‘Modification of CIEDE2000 for assessing color quality of image archives”, 13th Annual IS&T Archiving Conference 2016 (Archiving 2016), pp. 181-185. Publisher: Society for Imaging Science and Technology. CIE. (2001). CIE Publication 142. Improvement to industrial colour-difference evaluation. CIE Central Bureau, Vienna, 2001. CIE. (2004). CIE 15:2004. Colorimetry (3rd Ed.). CIE Central Bureau, Vienna, 2004. CIE. (2016). CIE 217:2016. Recommended method for evaluating the performance of colour-difference formulae. CIE Central Bureau, Vienna, 2016. García, P. A., Huertas, R., Melgosa, M. and Cui, G. (2007). ‘Measurement of the relationship between perceived and computed color differences’. Journal of the Optical Society of America A 24 (7), pp. 1823-1829. doi: 10.1364/ JOSAA.24.001823 Ho, K. M. R. (2016). The development of colour-difference formula for parametric effects. Ph.D. Dissertation, University of Leeds, UK, 2006. Huang, M., Wang, L. and Liu, H. (2010). ‘Study on small color difference evaluation using printed samples with different gloss’. Acta Optica Sinica 30, 1851-1856. In Chinese. Huang, M., Liu, H., Cui, G., Luo, M. R. and Melgosa, M. (2012a). ‘Evaluation of threshold color differences using printed samples’. Journal of the Optical Society of America A 29 (6), pp. 883-891. doi:10.1364/JOSAA.29.000883 Huang, M., Liu, H., Cui, G. and Luo, M. R. (2012b). ‘Testing uniform colour spaces and colour-difference formulae using printed samples’. Color Research and Application 37 (5), pp. 326-335. doi:10.1002/col.20689 Huang, M., Cui, G., Melgosa, M., Sánchez-Marañón, M., Li, C., Luo, M. R. and Liu, H. (2015). ‘Power functions improving the performance of color-difference formulas’. Optics Express 23 (1), 597-610. doi:10.1364/OE.23.000597 ISO/CIE. (2014). ISO/CIE 11664-6:2014 (former CIE S 014–6/E:2013). Colorimetry — Part 6: CIEDE2000 colour- difference formula. CIE Central Bureau, Vienna, 2014. Kim, D.-H. (1997). The influence of parametric effects on the appearance of small colour differences, Ph.D. Thesis, University of Leeds (UK), 1997. Kim, D.-H. and Nobbs, J. H. (1997). New weighting functions for weighted CIELAB color difference formula’. Proc. AIC Colour 97 (AIC 1997), Vol. I, pp. 446-449. Lee, S. G., Shamey, R., Hinks, D. and Jasper, W. (2011). ‘Development of a comprehensive visual dataset based on a CIE blue color center: Assessment of color-difference formulae using various statistical methods’. Color Research and Application 36 (1), pp. 27-41. doi:10.1002/ col.20549 Luo, M. R., Cui, G. and Rigg B. (2001). ‘The development of the CIE 2000 colour-difference formula: CIEDE2000’. Color Research and Application, 26 (5), pp. 340-350. doi:10.1002/col.1049 Luo, M. R. and Rigg, B. (1986). ‘Chromaticity discrimination ellipses for surface colours’. Color Research and Application 11 (1), pp. 25-42. doi:10.1002/col.5080110107 Melgosa, M., Cui, G., Oleari, C., Pardo, P. J., Huang, M., Li, C. and Luo, M. R. (2017). ‘Revisiting the weighting function for lightness in the CIEDE2000 colour-difference formula’. Coloration Technology 133 (4), pp. 273-282. doi:10.1111/ cote.12294 Shamey, R., Lin, J., Sawatwarakul, W. and Cao, R. (2014). ‘Evaluation of performance of various color-difference formulae using an experimental black dataset’. Color Research and Application 39 (6), pp. 589-598. doi. org/10.1002/col.21844 Wang, H., Cui, G., Luo, M. R. and Xu, H. (2012). ‘Evaluation of colour-difference formulae for different colour-difference magnitudes’. Color Research and Application 37 (5), pp. 316-325. doi:10.1002/col.20693 Whittle, P. (1992). ‘Brightness, discriminability and the crispening effect’. Vision Research 32 (8), pp. 1493-1507. doi:10.1016/0042-6989(92)90205-W Witt, K. (1999). ‘Geometric relations between scales of small colour differences’. Color Research and Application 24 (2), pp. 78-92. https://doi.org/10.1002/(SICI)1520- 6378(199904)24:2<78::AID-COL3>3.0.CO;2-M