acceptability (pass/fail) method to judge colour differences. Two phases of experiments were conducted. Phase 1 carried out at the University of Derby, including ...
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
Verification of CIEDE2000 using industrial data Ming Ronnier LUO,* Carl MINCHEW,† Phil KENYON,‡ and Guihua CUI* * Department of Colour and Polymer Chemistry, University of Leeds, UK † Benjamin Moore, USA ‡ Colwell, USA
ABSTRACT CIEDE2000 colour difference formula was recommended by CIE in 2000 for industrial colour difference evaluation. A new set of paint samples was prepared and was assessed by a panel of observers from two companies and one university. The results were used to reveal observer uncertainty, to test different colour difference formulae and to set each formula’s colour tolerance. The results show that CIEDE2000 slightly outperformed the other formulae. 1. INTRODUCTION A new colour difference formula, CIEDE2000, was recommended by CIE in 2000 for industrial applications (Luo, Cui and Rigg 2001, CIE 2001). The formula was developed through an international collaboration between colour scientists from different countries. The formula is based upon CIELAB (CIE 1986) with five corrections, which include a lightness, a chroma and a hue weighting function, and an interactive term between chroma and hue differences for improving the performance for blue colours and a scaling factor for CIELAB a* scale for improving the performance for grey colours. It outperformed the other advanced formulae such as CMC (Clarke, McDonald and Rigg 1984), BFD (Luo and Rigg 1987), CIE94 (CIE 1993) using four independent experimental data sets. This paper describes a new set of experimental data based upon paint samples using the acceptability (pass/fail) method to judge colour differences. Two phases of experiments were conducted. Phase 1 carried out at the University of Derby, including thirty-eight pairs of samples surrounding a grey colour centre. Phase 2 used 60 pairs of samples surrounding 10 colour centres. These pairs were assessed by a panel of 10 professional assessors from two companies. The results were used to investigate the performances of CIEDE2000 and the other advanced formulae in predicting colour differences having a large range. 2. EXPERIMENTAL 2.1 Sample preparation The paint samples used in the experiment were prepared by the Cowell, a company also involved in the later visual assessments. The experiment was divided into two phases. Phase 1 investigates only a grey colour centre having the CIELAB coordinates of 51.0, 0.2 and 1.2 for L*, a* and b* respectively. Each sample had a size of 5 by 10 cm and was mounted onto a white card. In total, 38 pairs of samples were selected. Figure 1 shows the sample pair distribution surrounding the colour centre in ∆a* ∆b* (left) and ∆L* ∆C* (right) diagrams. It 97
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
1.2
1.2
1
1
0.8
0.8
∆C*
∆b*
can be seen that these pairs were chosen to give a good coverage of almost all directions. Phase 2 experiment used 60 colour difference pairs covering a wide range of colour regions as shown in Figures 2a and 2b for a*b* and L*C* diagrams respectively. Each sample was measured by a GretagMacbeth CE7000A spectrophotometer with a large aperture and an exclusion of specular and UV component. The spectro results were then converted to CIELAB values under illuminant D65 and 1964 standard colorimetric observer.
0.6
0.6
0.4
0.4
0.2
0.2
0 -0.6
-0.4
-0.2
0 0
0.2
0.4
0.6
-0.6
-0.4
-0.2
0
∆a*
0.2
0.4
0.6
∆L*
(a) (b) Figure 1. The sample pair distribution for phase 1 experiment in a) ∆a* ∆b* and b) ∆L* ∆C* diagrams.
50
60
40
50 30
20
40
C*
b*
10
0
30
-10
20 -20
-30
10 -40
-50
0 -50
-40
-30
-20
-10
0
10
20
30
40
50
0
a*
10
20
30
40
50
60
70
80
90
100
L*
(a) (b) Figure 2. Sample pair distribution for phase 2 experiment in a) a*b* and b) L*C* diagrams.
98
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
2.2 Visual assessment These pairs were assessed by 10 normal colour vision observers at the University of Derby. Each observer assessed each pair twice. Phase 2 was conducted at two companies with five professional assessors in each lab. Each observer did experiment twice. The instruction shown below is given to observers before each session. Imagine that you just painted half of the wall and you run out of the paint. You went back to buy another tin of that paint. When you paint the wall, you are asked to judge whether you “accept” or “reject” the match between each half of the wall. Three GretagMacbeth SpectraLight viewing cabinets were used in each site. It includes a filtered tungsten light source, an accurate D65 simulator with an A grade in visual range (Xu, Luo and Rigg 2003). 2.3 Measure of fit: wrong decision The data accumulated here described in terms of acceptability percentage, in which a batch sample is judged as pass or fail against a standard in percentage. The acceptability data is described in terms of acceptance % (A%) for a pair, e.g. a 30 A% means that 30% of observers accept the sample as a good match to the standard. For investigating observer uncertainty, “wrong decision” measure was used (McLaren 1970). For investigating observer accuracy, each individual’s results in terms of pass or fail against the panel results in terms of %A were compared. If an individual result is a “pass” against a panel result of 35% which is less than 50% (a “fail”), both results thus disagree with each other for this pair. Hence, it is counted as a wrong decision. The same principle applies to all pairs. Finally, the performance is expressed by WD%, which is the number of wrong decision pairs divided by the total number of sample pairs. For a perfect agreement, WD% should equal to zero. For examining observer repeatability, the WD% measure is also used to represent number of wrong decision made by the two repetitions from a single observer. For comparing colour difference equations’ performance, WD% measure is again used. Two examples are given in Figure 3 by plotting A% of the grey colour centre in phase 1 against the ∆E values calculated by a) CIELAB (2:1) and b) CIEDE2000 (2:1). Each of Figure 3 is divided into 4 regions (Q1 to Q4) by a horizontal line at 50 A% and a vertical line (∆Et). A trend can be found that a decrease of A% values with an increase of ∆E values. The wrong decision data is determined by adding the numbers in Q1 and Q3, i.e. the data in Q1 exhibiting small visual differences but large ∆E values of a colour difference equation. The data in Q3 are opposite to those in Q1. When calculating WD%, the line of ∆Et is varied from small to large colour differences. Each change will calculate a new WD%. Finally, the minimum WD% value corresponds to a particular ∆Et value. The WD% in Figure 3a and 3b are 26% and 18% respectively, i.e. 10 and 7 data points in Q1 and Q3 divided by total number of pairs (38) for CIELAB and CIEDE2000 respectively.
99
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
100%
100%
Q2
90%
90%
80%
80%
70%
70%
60%
60%
A%
A%
%
Q1
50%
Q1
50%
40%
40%
30%
30%
20%
Q2
Q4
Q3
20%
10%
Q3
0% 0.3
Q4
∆Et 0.4
0.5
10%
∆E t
0%
0.6
0.2
0.3
0.4
∆E CIELAB(2:1)
0.5
0.6
0.7
0.8
0.9
CIEDE2000(2:1) ∆ E
(b)
(a)
Figure 3. Method for determining the wrong decision for a colour-difference equation to predict an acceptability data set. 3. RESULTS AND DISCUSSION 3.1 Observer uncertainty Observer uncertainty results were analysed. As mentioned earlier, 10 university observers participated in phase 1 experiment and each observer did assessments twice. This resulted in 20 observations for each of 38 pairs. In phase 2, 5 observers from each company attended the experiment. Each one did assessment twice. The results are summarised in Table 1. Table 1. Summary of the observer uncertainty in WD% unit.
Observer repeatability Observer accuracy Between company errors
Phase 1 27 ranged from 16 to 37 28 ranged from 16 to 45 -
Phase 2 32 ranged from 12 to 43 32 ranged from 25 to 38 33
Table 1 results showed that there is not much difference between observer accuracy and repeatability. Phase 1 observers performed more accurate and more repeatable than phase 2 observers. However, this could be due to the different coloured sample pairs used in the experiment. Another analysis was carried out to compare the visual results between two companies. It was found that a wrong decision of 33% between two companies. The current finding indicates that observers could have one wrong decision out of 3 judgements. 3.2 Testing different colour difference formulae The visual results obtained from two phases were used to test four colour difference formulae: CIEDE2000, CMC, CIE94 and CIELAB. Their performances are given in Tables 2 and 3 for phases 1 and 2 results respectively. For each formula, two lightness parametric factors were used, 1 and 2.
100
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
Table 2. Performance (WD%) of formulae for phase 1 results (38 pairs). kL=1
Tolerance ∆Et WD%
kL=2
Tolerance ∆Et WD%
CIEDE2000 0.45 21% 0.45 18%
CIELAB 0.39 32% 0.37 26%
CIE94 0.38 32% 0.36 26%
CMC 0.52 29% 0.52 24%
Table 3. Performance (WD%) of formulae for phase 2 results (60 pairs). kL=1
Tolerance ∆Et WD%
kL=2
Tolerance ∆Et WD%
CIEDE2000 0.38 23% 0.29 43%
CIELAB 0.46 35% 0.37 48%
CIE94 0.41 25% 0.29 38%
CIELAB
100.00%
100.00%
90.00%
90.00%
80.00%
80.00%
70.00%
70.00%
60.00%
60.00% A%
A%
CIEDE2000
50.00%
50.00%
40.00%
40.00%
30.00%
30.00%
20.00%
20.00%
10.00%
10.00%
∆Et
0.00% 0.2
0.3
0.4
∆Et
0.00% 0.5
0.6
0.7
0.2
0.3
CIEDE2000 Colour Difference
0.4
0.5
0.6
0.7
0.6
0.7
CIELAB Colour Difference
(a)
(b)
CIE94
CMC
100.00%
100.00%
90.00%
90.00%
80.00%
80.00%
70.00%
70.00%
60.00%
60.00% A%
A%
CMC 0.42 28% 0.34 43%
50.00%
50.00%
40.00%
40.00%
30.00%
30.00%
20.00%
20.00%
10.00%
10.00%
∆Et
0.00% 0.2
0.3
∆Et
0.00% 0.4
0.5
0.6
0.7
0.2
CIE94 Colour Difference
0.3
0.4
0.5
CMC Colour Difference
(c)
(d)
Figure 4. The wrong decision for 4 colour-difference formulae.
101
AIC 2004 Color and Paints, Interim Meeting of the International Color Association, Proceedings
Table 2 showed that for each formula with a lightness parametric factor (kL) set to either 1 or 2, CIEDE2000 performed the best, followed by CMC and CIELAB and CIE94 the worst. For each formula, kL=2 version performed better than kL=1 version. However, both versions predict phase 1 results more accurate than the observer uncertainty about 30%. Figure 3 further confirmed the finding that the scatter for CIEDE2000 (3b) is much smaller than that of CIELAB (3a). Table 3 showed an opposite trend to Table 2, i.e. all formulae with kL=1 outperformed kL=2 by a large margin in Table 2. Also, the ∆Et values between the two tables are quite different. This discrepancy could be caused by the different sample pairs used in each phase and also different groups of observers used (professional and naïve observers). However, again CIEDE2000 slightly outperformed the other formulae. This is further confirmed in Figure 4 by plotting the phase 2 visual results (A%) vs four different formulae with kL=1. Again, it can be seen that CIEDE2000 formula had a smaller scatter than the others. 4. CONCLUSION Two phases of experiment were carried out to investigate the performance of CIEDE2000 colour difference formulae. The results imply that pass/fail judgements frequently used in industry are somewhat unreliable with one wrong decision out of three judgements. All colour difference formulae predict visual results more accurate than the observer uncertainty. However, CIEDE2000 consistently performed better than the other formulae. REFERENCES CIE (Commission Internationale de l’Éclairage). 1986. Colorimetry, 2nd edition, CIE Publ. 15.2. Paris: Central Bureau of the CIE. ——. 1993. Technical report: Parametric effects in colour-difference evaluation, CIE Publ. 101. Vienna: Central Bureau of the CIE. ——. 2001. Technical report: Improvement to industrial colour-difference evaluation, CIE Publ. 142. Vienna: Central Bureau of the CIE. Clarke, F. J. J., R. McDonald, and B. Rigg. 1984. Modification to the JPC79 colour-difference formula. Journal of the Society of Dyers and Colourists 100: 128-132 and 281-282. Luo, M. R., and B. Rigg. 1987. BFD(l:c) colour difference formula. Part I: Development of the formula. Journal of the Society of Dyers and Colourists 103: 86-94. Luo, M. R., G. Cui, and B. Rigg. 2001. The development of the CIE 2000 colour difference formula. Color Research and Application 26: 340-350. McLaren, K. 1970. Colour passing – visual or instrumental? Journal of the Society of Dyers and Colourists 86: 389-392. Xu, H., M. R. Luo, and B. Rigg. 2003. The evaluation of daylight simulators. Part I: Colorimetric and spectral variations. Coloration Technology 119: 59-69. Address: Ming Ronnier Luo, Department of Colour and Polymer Chemistry University of Leeds, LS2 9JT, England E-mail:
102
