ReseaRch PaPeR Journal of Agricultural and Marine Sciences Vol. 22 (1): 42-47 DOI: 10.24200/jams.vol22iss1pp42-47 Reveived 15 Feb 2016 Accepted 25 Oct 2016 In-situ colour correction for digital images acquired under non-standard lighting conditions P. M. K. Alahakoon2 and Annamalai Manickavasagan1* 1* A.Manickavasagan ( ) Department of Soils, Water and Agricul- tural Engineering, College of Agricultural and Marine Sciences Sultan Qaboos University, P.Box 34, PC 123, Sultanate of Oman. 2De- partment of Agricultural Engineering, Faculty of Agriculture, Univer- sity of Peradeniya, Sri Lanka. Introduction Computer vision techniques have been successful-ly used in various pre-harvest and post-harvest applications of agricultural and food products. Due to the common availability of digital imaging equip- ment and image processing software, many have been encouraged to develop their own techniques which are more inexpensive and easy to use (O’Neal et al., 2002) than depending on commercially available equipment. Further, the developments in digital imagery have opened many new trends in plant and food applications. Digital cameras in combination with computers and ap- propriate software can be used to image and evaluate the objects for their many surface qualities with relative ease and at an affordable cost. However, the research- ers are compelled to acquire images under laboratory conditions, by illuminating the test sample under stan- dard lighting conditions in order to achieve a consistent colour interpretation which is a must for the success of subsequent analyses. The selection of the illumination source must be done carefully which may drastically change the out- come (Luo et al., 1997; Brown and Timm, 1992; Man- ickavasagan et al., 2008). Due to this reason, most of the image related studies inevitably use either a laboratory تصحيح الصور الرقمية امللتقطة ابملوقع حتت ظروف اإلضاءة غري القياسية أالحكون2 ومنيكافا سجان أنامايل1* Abstract. Computer vision techniques using colour images are becoming popular in food and agriculture sector. Need of a standard illumination source is an important criterion in this approach to determine various attributes based on RGB values of the objects. In general, under laboratory conditions with standard lighting, an imaging system performs with high consistency in digitizing colour. However, in field conditions where the availability of a standard light source cannot be guaranteed, the colour interpretations may not yield accurate results. The objective of this study was to develop a simple algorithm to compensate for the variations in RGB values due to varying light conditions. It is intended to be useful in situations where taking digital images of objects without standard light sources is essential for a particular purpose. A set of quadratic transformation algorithms were developed to transform the RGB values of the images acquired under five different lighting conditions. The mean variance in RGB values of the image of a colour pal- ette (with 6 different colours) taken under five lighting conditions were in the range of 277 – 548. After implementing the developed algorithm, this was reduced to 34 – 142. Similarly, this variance was reduced from 180 – 294 to 63 – 128 in the test conducted with a plant material. This algorithm can be easily adopted in all computer vision applications where variations in colour interpretations due to nonstandard lighting sources are common. Keywords: Colour balancing; illumination; image correction; computer vision. امللخــص: أصبحــت تقنيــات الرؤيــة احلاســوبية باســتخدام الصــور امللونــة شــائعة يف جمــاالت الغــذاء والزراعــة. تعتــر احلاجــة إىل مصــدر ضــوء قياســي يف اســتخدام هذه التقنية معيارا مهما للتعرف على خمتلف الســمات لألشــياء اعتمادا على قيم األلوان الثالثة األمحر األخضر واألزرق )RGB( بشــكل عــام، وحتــت الظــروف املعمليــة وباســتخدام إضــاءة قياســية يســتطيع نظــام الرؤيــة احلاســوبية التعــرف علــى األلــوان بدرجــة عاليــة مــن التناســق. يف املقابــل، يف الظــروف الفعليــة ومــع عــدم ضمــان توفــر مصــدر للضــوء القياســي فــإن التعــرف علــى األلــوان باســتخدام هــذا النظــام قــد ال يعطــي نتائــج صحيحــة. اهلــدف مــن هــذه الدراســة هــو تطويــر خوارزميــة بســيطة حلــل مشــكلة التبايــن يف قيــم األلــوان )RGB ( والــذي تســببه تأثــرات اإلضــاءة املتغــرة. تكمــن أمهيــة هــذه اخلوارزميــة عنــد احلاجــة إىل التقــاط صــور لشــيء مــا بــدون اســتخدام مصــدر إضــاءة قياســي. مت تطويــر جمموعــة مــن اخلوارزميــات التحويليــة مــن )RGB( للصــور امللتقطــة حتــت مخــس ظــروف خمتلفــة للضــوء. وجــد أن متوســط التبايــن يف قيمــة األلــوان )RGB( الدرجــة الثانيــة لتحويــل قيــم األلــوان املأخــوذة عــن لوحــة األلــوان والــي حتــوي ســتة ألــوان خمتلفــة حتــت ظــروف اإلضــاءة اخلمــس كانــت يف حــدود 277-548. بينمــا وبعــد اســتخدام اخلوارزميــة املطــورة مت ختفيــض هــذا التبايــن ليصــل إىل 34-142. باملثــل، مت تقليــص التبايــن مــن 180-294 إىل 63-128 يف جتربــة مت تنفيذهــا علــى جــزء مــن نبتــة. ميكــن اســتخدام هــذه اخلوارزميــة يف كل تطبيقــات الرؤيــة احلاســوبية حيــث أن التبايــن يف التعــرف علــى األلــوان يعتــر شــائعا بســبب ظــروف اإلضــاءة غــر القياســية واملتغــرة. الكلمات املفتاحية: موازنة اللون؛ اإلضاءة؛ تصحيح الصورة؛ الرؤية احلاسوبية 43Research Article Alahakoon, Manickavasagan setup to acquire images under controlled lighting condi- tions or use a commercially available flat bed scanner to scan the objects for obtaining their images in the case of flat objects such as plant leaves (Murakami et al., 2005). Some of the light sources used are circular fluorescent lights (Paliwal et al., 2003; Majumdar and Jayas,1999), fluorescent tube lights (Manickavasagan et al., 2008), in- candescent lights, and infrared lights (halogen lamps) in special situations (Hehn and Sokhansanj, 1990; Liu et al., 2005). Digital imaging is very much dependent on the light source and the resulting interpretation of colours in RGB coordinates or as gray levels. However, in agriculture like many other natural sci- ence fields, there are many instances where the sample or the object needs to be imaged in the natural envi- ronment. In field conditions, the plant materials such as leaves, fruits and flowers have to be imaged without removing from the plant. Also in food handling and in supply chain applications, there are several situations where the background lighting cannot be made standard all the time. One of the challenges faced by researchers in taking digital images under such non-standard lighting situa- tions that may also change from one picture to another, is to maintain closeness in the representative colours in the images. Even though there are white balancing op- tions included into many modern digital cameras used today in order to provide a close approximation to the actual colours present in the scene, capabilities in the camera alone are not sufficient to yield the same ‘digi- tal’ colours for an object imaged under different lighting situations. Even though professional software packages provide excellent improvement in colour and contrast, the resulting colours may not yield the same RGB val- ues for the same object imaged under different environ- ments and lighting situations. On the other hand, since the colour of the sample or its specific regions of interest are represented as an RGB value, the researcher must be able to ensure that the acquired digital image does contain RGB values very close to the actual if not exact. The expected actual RGB values are the ones that will be obtained for a lighting situation that will provide 0, 0, 0 for black and 255, 255, 255 for white regions in the same scene. Therefore, it is essential that the digital images acquired are properly preprocessed in such a way that RGB values are closer to the actual as that will assist in proper diagnosis and drawing conclusions on the image properties, especially in food and agriculture related work which involves field images. As can be seen in some applications, measure- ment of different colour patches is important in mak- ing decisions in a large number of studies (O’Neal et al., 2002). Shahin and Symons (2003) developed an algorithm to adjust the colours recorded by digital scanners in order to establish uniform interpretation of colours in the event of using more than one scanner for scanning grains for research purposes. Chang and Reid (1996) presented a method for correcting for the variations in RGB values caused by vision system components used in acquiring the image. Different types of error sources have been identified and modeled in order to correct for the errors. Weng et al. (2006) proposed a new algorithm for automatic white balancing of digital images which provided more visually appealing high contrast images compared to some of the existing methods such as gray world method, perfecto reflector method, fuzzy rule method, and Chikane’s method. Conversely, Chikane and Fuh (2006) also presented an improved algorithm that can be incorporated into the image processing soft- ware in digital cameras, showing that white balancing is a prime requirement of all image acquisition operations. In order to carry out manual white balancing, a more artistic operation, it is necessary to acquire pictures in the raw format, and use professional software custom- ized to post-adjust the RGB components of the image based on the information recorded by the camera at the time of acquisition. In situations where raw image ac- quisition and the use of professional graphics processing software is limited, especially in industrial visits, and ac- quisition of images in agricultural fields, this approach posed limitations in use, thus justifying an attempt to develop techniques to address this issue. Takemura and Ishii (2011) presented a neural net- work based approach for determination of actual colour of objects for robot vision systems. Due to its very so- phisticated and demanding computational power re- quirement, normal users find it difficult to adopt such to their work. Besides, the calibrations performed are camera specific and may not be applicable for different cameras used under different conditions. Li et al. (2006) developed a specific colour adjustment system to obtain close to true colour representations of colposcopic im- ages used in auto diagnosis of cervical cancer and also in telemedicine related applications. Since the diagnosis is primarily based on the colour of the tissue observed under artificial light sources, they have found that vari- ations in the light source provided different colours in the digital image acquired, which may obstruct reach- ing a clear and fast diagnosis. Development of a simple algorithm that can be applied for each image would be highly beneficial to utilize in computer vision techniques for food and agricultural applications where huge varia- tions in lighting conditions are inevitable. Therefore the objective of this study was to develop an algorithm to correct the RGB values of an image based on the RGB values of three standard colour cards imaged under the same lighting condition. 44 SQU Journal of Agricultural and Marine Sciences, 2017, Volume 22, Issue 1 In-situ color correction for digital images acquired under non-standard lighting conditions Methodology Standard colour cards Black, gray and white standard colour cards (DGK Co- lour Tools, Boston, Massachusetts, USA) were used as the basis for correcting the RGB values of different re- gions of the images acquired under different lighting sit- uations. The homogeneous RGB values of these colour cards were (0, 0, 0), (128, 128, 128) and (255, 255, 255) for black, white and gray, respectively. Colour palette A flat colour palette comprising of 6 different colour re- gions along with the 3 standard colour cards was assem- bled by using uniform colour sheets as shown in (Fig. 1). The colours were selected by taking 3 stronger shades (blue, green and red) and three lighter shades (yellow, light green and pink) to test the effectiveness of the al- gorithm. Image acquisition A digital colour camera (Resolution: 4608 × 3456; Model: WB850F; Samsung Electronics Company Ltd., UK) was used to acquire images. The developed colour palette was imaged under five non-standard lighting conditions in order to simulate real field conditions: (1) shade of a building; (2) fluorescent tube lighting (inside the labora- tory; (3) natural cloudy (outside); (4) in a dark corridor (imaged with built-in flash); and (5) focused fluorescent lighting (test bench). Field test To investigate the performance of the developed algo- rithm, a simulated field experiment was conducted by taking images of a plant leaf under the same lighting sit- uations. In this experiment, the plant leaf was imaged along with the three standard colour cards in all lighting conditions. Results and discussion Algorithm development with standard colour cards The colour palette, comprising of 6 different colour squares and 3 standard colour cards were used together and exposed to 5 lighting situations. RGB values extract- ed from the standard colour cards were used to develop a quadratic transformation algorithm for each image us- ing the least squares method, so that the RGB values for the three cards would be translated to (0,0,0), (255, 255, 255), and (128, 128, 128), respectively for Black, White, and Gray, as prescribed by the manufacturers. All the mathematical operations related to algorithm develop- ment and numerical computations were done in MS Ex- cel environment. The algorithm development is briefly described by considering the matrices [Y], [X] and [h], where [Y] con- taining the expected gray levels for the standard colours [0 128 255], [X] containing the actual measured RGB values representing each colour, placed as [R2 R1] for the equation for the R component, and [h] matrix being the coefficients required as the solution for each equa- Table 1. Variance observed in the RGB values among the images acquired under 5 different lighting conditions before and after correction. Red Green Blue Card colour Before After Before After Before After Red 817 0 80 285 61 101 Yellow 868 0 595 11 0 0 Green 398 117 390 78 122 605 Light green 336 42 525 12 366 27 Blue 68 42 504 218 511 64 Pink 798 3 600 111 604 57 Mean 548 34 449 119 277 142 Figure 1. Image of the developed colour palette (first col- umn represents 3 standard colour cards). 45Research Article Alahakoon, Manickavasagan tion developed. Y⎡⎣ ⎤⎦ = X⎡⎣ ⎤⎦ h⎡⎣ ⎤⎦ h⎡⎣ ⎤⎦ = X TX⎡⎣ ⎤⎦ −1 X⎡⎣ ⎤⎦ T Y⎡⎣ ⎤⎦ (1) The coefficients thus derived by solving for [h] as shown in Equation 1, provided the algorithm for trans- forming a colour component R, G, or B. The same op- eration was repeated for the 3 colour components. The set of 3 functions developed were then used to trans- form the colour values of the other colour squares that represent different objects in the same image in order to obtain a colour adjusted image. The final RGB values for each colour region were compared to estimate the vari- ance among images obtained under different lighting situations. As a final measure to fine tune the algorithm, the gray level corresponding to the Gray card region (128) was adjusted to bring the total variance computed among RGB values of the new images to a minimum. RGB correction on colour palette regions The image of the colour palette with 3 standard colour cards and 6 refernce colours are shown in Fig. 1. The de- rived equations were used to correct the RGB values of the 6 colour regions in the colour palette. A sample set of such transformation functions derived for one lighting setting is given in Eqn 2-4. R New = −0.003108R Old 2 +2.1102R Old −46.8914 (2) G New = −0.002983G Old 2 +2.0802G Old −46.2674 (3) B New = −0.003705B Old 2 +2.2362B Old −48.2317 (4) Where R, G, BOld are the RGB values in the original im- age and R, G, BNew are the RGB values of the corrected image. In order to simulate and represent the field situations, some of the pictures were taken under natural light out- side as well as in the shade. The RGB values pertaining to 10 different points in the White, Gray, and Black re- gions were sampled and recorded together with those sampled from the other 6 colour regions. The variance values computed for each of the colour components R, G, and B, of selected regions representing each colour palette before and after the colour adjustment are given in this Table 1. The high variance values before adjusting the colours indicate that a colour is represented by sig- nificantly different RGB values depending on the light- Table 2. Variances observed in the RGB values among the images of a plant leaf acquired under 5 different lighting conditions before and after correction. Red Green Blue Card colour Before After Before After Before After Region 1 189 161 210 38 291 62 Region 2 226 4 247 159 0 0 Region 3 11 7 446 200 247 166 Region 4 533 79 402 83 42 94 Region 5 97 0 354 228 183 35 Region 6 246 129 105 59 315 164 Mean 217 63 294 128 180 87 Figure 2. Example of the RGB values representing three standard colour cards (White, Gray, Black) used under one lighting condition). 46 SQU Journal of Agricultural and Marine Sciences, 2017, Volume 22, Issue 1 In-situ color correction for digital images acquired under non-standard lighting conditions ing source under which it is imaged. After adjustment of colour values, the mean variances show a significant reduction. The only exception being with the RGB val- ues pertaining to green colour imaged under fluorescent lighting. This could be attributed to the unbalanced na- ture of spectral distribution in fluorescent light, which may significantly alter the R and B components com- pared to the G component. RGB correction on plant leaf regions The colours extracted from the plant leaf were trans- formed using the new algorithms derived from the RGB values sampled from the standard colour cards includ- ed in the same image. The analyses showed that the al- gorithms derived in following the developed approach were capable of transforming the image RGB values to new values giving lower variance among the same co- lours in different images. The variance values computed for each of the colour components R, G, and B, of se- lected regions of different colour patches on the plant leaf before and after the colour adjustment procedure are shown in Table 2. Pixel value averaging was used in both these representations to minimize any variations within the selected representative region, in each light- ing situation. The variance among the adjusted colour regions appear to be lower than that before adjustment, indicating the success of the method tested herein. The algorithm as well as the procedure to acquire images was expected to be relatively easy to implement on general purpose software platforms and in-situ im- age acquisition work, and present more flexibility in deciding on the gray level that corresponds to the level of gray used in the image compared to other commer- cially available image editing software, and reduce the inter-image variability in the final colours obtained after the transformation. Flexibility of the algorithm There were several advantages in the new approach com- pared to the use of traditional colour balancing methods for RGB values of acquired images. The developed algo- rithm corrected RGB values by taking into account 10 representative regions distributed over the whole image representing the three colour pallets in the image. This undoubtedly makes the algorithm more robust and rep- resentative of the existing RGB values, since a close ex- amination of the standard colour regions would reveal that there are variations in RGB values among the pixels regardless of the effort to provide uniform lighting at the time of taking the image. Further, an additional ‘tuning’ operation was also built into the new algorithm whereby the RGB value of the Gray colour card could be adjusted in a manner that would provide minimum variance among the same co- lour regions pictured under different lighting situations, after subjecting them to the colour transformations. Conclusion The developed algorithm for colour adjustment of im- ages provided less dispersed RGB values for a certain colour pictured under different lighting situations, com- pared to the original image. This made it possible to use images acquired in the field with minimal colour vari- ations for subsequent image analyses and feature rec- ognition steps. It was however necessary to acquire the image all the time with the three standard colour cards in the frame of the image together with the subject, to carry out this colour adjustment procedure. Acknowledgements We thank The Research Council (TRC) of the Sultanate of Oman for funding this study (Project No. RC/AGR/ SWAE/11/01-Development of Computer Vision Tech- nology for Quality Assessment of Dates in Oman). References Brown, G. K.and Timm, E. J., 1992. Lighting for fruit and vegetable sorting. ASAE paper no. 936069, St. Joseph, MI, USA. Chang, Y., Reid and J. F., 1996. RGB calibration for color image analysis in machine vision. IEEE Transactions on Image Processing. 5, 1414-1422. Chikane, V.and Fuh, C., 2006. Automatic white balance for dgital still cameras. Journal of Information Sci- ence and Engineering. 22, 497-509. Hehn, J. L. and Sokhansanj, S. 1990. Canola and mustard seed identification using mackintosh based imaging system. ASAE paper no. 903534, St. Joseph, MI, USA. Li, W., Thompson, M. S., Xiong, Y. and Lange, H. 2006. A new image calibration technique for colposcopic images. Medical Imaging 2006: Image Processing, edited by Joseph M. Reinhardt, Josien P. W. Pluim, in Proceedings of SPIE. 6144, 227-239. Liu, Z., Cheng, F., Ying, Y. and Rao, X. 2005. Identifi- cation of rice seed varieties using neutral network. Journal of Zhejiang University Science. 6B, 1095- 1100. Luo, X., Jayas, D. S., Crowe, T. G. and Bulley, N. R. 1997. Evaluation of light sources for machine vision. Cana- dian Agricultural Engineering. 39, 309-315. Majumdar, S. and Jayas, D.S. 1999. Classification of bulk samples of cereal grains using machine vision. Jour- nal of Agricultural Engineering Research. 73, 35-47. Manickavasagan, A., Sathya, G. and Jayas, D.S. 2008. Comparison of illuminations to identify wheat class- es using monochrome images. Computers and Elec- tronics in Agriculture. 63, 237-244. Murakami, P. F., Turner, M.R., Van den Berg, A. K. and Schaberg, P. G. 2005. An instructional guide for leaf color analysis using digital imaging software. Gen. 47Research Article Alahakoon, Manickavasagan Tech. Rep. NE-327. Newtown Square, PA: U.S. De- partment of Agriculture, Forest Service, Northeast- ern Research Station. 33 p. O’Neal, M., Landis, D. A.and Isaacs, R. 2002. An inex- pensive, accurate method for measuring leaf area and defoliation through digital image analysis. Journal of Economic Entomology. 95, 1190 – 1194. Paliwal, J., Visen, N. S., Jayas, D. S. and White, N.D.G. 2003. Cereal grain and dockage identification using machine vision. Biosystems Engineering. 85, 51-57. Shahin, M. A. and Symons, S. J. 2003. Color calibration of scanners for scanner-independent grain grading. Cereal Chemistry Journal. 80, 285-289. Takemura, Y, and Ishii, K. 2011. Auto color calibration algorithm using neural networks and its application to RoboCup robot vision. International Journal of Ar- tificial Intelligence. Weng, C., Chen, H. and Fuh, C. 2006. A novel automatic white balance method For digital still cameras. Jour- nal of Communication Engineering. 12.