Microsoft Word - 65benalia.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 58, 2017 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Remigio Berruto, Pietro Catania, Mariangela Vallone Copyright © 2017, AIDIC Servizi S.r.l. ISBN 978-88-95608-52-5; ISSN 2283-9216 Assessment of the Ripening of Olives Using Computer Vision Souraya Benaliaa, Bruno Bernardi*a, José Blascob, Antonio Fazaria, Giuseppe Zimbalattia a Department of AGRARIA, University Mediterranea of Reggio Calabria, Loc. Feo di Vito, 89122 Reggio Calabria, Italy b Agricultural Engineering Center, Instituto Valenciano de Investigaciones Agrarias (IVIA), CV-315, Km. 10,7 – 46113 Moncada (Valencia), Spain bruno.bernardi@unirc.it In the framework of a continuously evolving global market, olive and olive oil industries should introduce new and innovative technologies in order to enhance their productivity and improve their competitiveness. Computer vision systems (CVS) employed in automated processes for olive sorting and/or quality inspection constitute a promising tool that allow these industries to respond to the global market requirements. One of the application of CVS in this sector may be the prediction of olive ripening through data obtained from machine vision systems, in order to achieve a proper processing and obtain high quality products. Indeed, either for olive oil or table olives, ripening degree represents a key factor that influences the final product features. In this context, the present study aims to evaluate colour changes during olive ripening using a computer vision system. Experimental trials considered two olive (Olea europaea L.) cultivars, namely, ‘Carolea’ and ‘Nocellara’. First, experienced operators classified the olives visually in five different ripening classes for Carolea and six classes for Nocellara. After that, olive image acquisition was carried out employing a laboratory computer vision system consisting of a digital camera inside an inspection chamber under a controlled illumination. Images were then, pre-treated for white balance as well as chromatic correction using a profile specifically created with Colorchecker Passport Software (X-Rite Inc, USA), and subsequently analysed using Food-Color Inspector 3.5 (Cofilab) software, which allowed obtaining the segmentation models for colour olive images and the subsequent analysis of their features. The obtained data from image analysis expressed in terms of R, G, B, CIE L*, CIE a* and CIE b* colour coordinates, green area (%) and veraison area (%) were statistically analysed using ANOVA and PCA. Image analysis results show highly significant differences between the two studied cultivars as well as between the ripening classes. Moreover, PCA results illustrate that, for both cultivars, the main variability is expressed according to the first two components, with a different effect of colour coordinates on these latter. 1. Introduction Marketplace rules become ever more severe because of the globalization that imposes high productive standards. Moreover, the rising interest of the consumers towards the provenience, the productive process and especially toward the quality of the products they purchase, lead the agro-food industries to introduce new and innovative technologies that enable them to respond to consumers’ requirements and expectation, assuring their competitiveness (Bernardi et al., 2013). Olive and olive oil industries, as well as the entire sector should be up to date regarding these technologies. Non-destructive technologies, especially those based on computer vision system (CVS) constitute a promising tool (Benalia et al., 2015), which allow olive and olive oil industries to increase the marketed product quality especially if employed in automated processes for olive sorting and/or quality inspection. The scientific literature reports different applications of computer vision systems and the subsequent image analysis in olive and olive oil sector. Diaz et al. (2000 & 2004) developed and tested a fast algorithms for processed olive sorting; Riquelme et al. (2008) analyzed the images of olives Manzanilla sevillana considering both colour features and morphological characteristics of external defects; Furferi et al. (2010) predicted olive ripening index from image analysis data; Ram et al. (2010) developed predictive models of oil content of DOI: 10.3303/CET1758060 Please cite this article as: Benalia S., Bernardi B., Blasco J., Fazari A., Zimbalatti G., 2017, Assessment of the ripening of olives using computer vision, Chemical Engineering Transactions, 58, 355-360 DOI: 10.3303/CET1758060 355 Picual and Souri olive cultivars, Abdelhedi et al., (2012) built an algorithm aimed to verify oil level in the bottle and caps defects. One of the application of CVS in olive and olive oil sector may be the prediction of olive ripening through data obtained from machine vision systems, in order to achieve a proper processing and obtain high quality products. Indeed, either for olive oil or table olives, ripening degree represents a key factor that influences the final product features. Currently, the determination of olive maturity index is based on the visual colour assessment ranging from olives with a deep green skin to olives with a black skin and violet flesh (Essiari et al., 2014) according to the developed formula at the Venta del Llano Experimental Station, IFAPA, Mengíbar (Jaén, Spain) (COI, 2011). Fruit and vegetables ripening assessment using computer vision system provided reliable results such as demonstrated by Mendoza and Aguilera (2004) for bananas, Fadilah et al., (2012) for oil palm fresh fruit bunch, Vélez-Rivera et al., (2014) for mangoes cv. ‘Manila’, as well as Kanade and Shaligram (2015) for guava fruit. In this context, the present study aims to evaluate colour changes during olive ripening using a computer vision system considering two olive (Olea europaea L.) cultivars. 2. Material and methods 2.1 Samples The experimental trials considered olives cv. ‘Carolea’ and ‘Nocellara’. The drupes were harvested manually in different dates during 2015 campaign, in a commercial olive orchard situated in the Province of Reggio Calabria (Southern Italy). An experienced operator classified them according to their ripening degree, obtaining therefore five (5) classes for ‘Carolea’ and six ones (6) for ‘Nocellara’ based on the visual appearance of the olives (Figure 1). Figure 1: Visual classification of ‘Carolea’ and ‘Nocellara’ drupes according to the ripening degree by an experienced operator. 2.2 Image acquisition Image acquisition of each drupe was performed with a digital camera NIKON D5200 (Nikon Corp., Japan) that allow capturing images of 24.1 million effective pixels. The captured images had a size of 2286 x 3016 pixels and a resolution of 0.105 mm/pixel. The camera was provided with an AF Micro NIKKOR 60mm f/2.8D lens. Lighting was provided by eight fluorescent tubes (BIOLUX T8-18W/72, 6500 K, OSRAM, Germany) placed on two opposite sides of a square inspection chamber in a 0°/45° configuration (Figure 2). The camera was connected to a computer and was remotely managed using the digiCamControl V.1.2.0.0 software. Considering the used lamps, ISO value was set to 100, while the aperture was set according to Mendoza et al. (2006) and Valous et al. (2009). Once fixed these two parameters, the exposition time was adjusted until obtaining a correct exposition using the ColorChecker white balance target (X-Rite Inc, USA); the obtained value was 1/250 s. 356 Figure 2: The inspection chamber used for image acquisition. 2.3 Image pre-processing and analysis Since RGB colour model is device dependent (Menesatti et al., 2012), a previous calibration step was performed. Indeed, subsequently to image acquisition, all images underwent the white balance and the chromatic calibration according to a previously created profile with the ColorChecker Classic target (X-Rite Inc, USA), through the Colorchecker Passport Software. The calibrated images were then analysed with the Food-ColorInspector 3.5 Software (available at http://www.cofilab.com). This software allows to select training sets of pixels belonging to representative areas of the olives with different colours, and creating a segmentation model through Bayesian linear discriminant analysis that is later applied to the rest of the images. Seventy percent (70 %) of the images of each cultivar were segmented considering two color classes i.e. green and veraison moreover than the background (Figure 3). Three segmentation steps were carried out for each color class and for each image. At the end of the segmentation process, the obtained training model was applied to all the images in order to obtain drupe color features. Figure 3: Olive original image (left) and the image segmented (right). 2.4 Data analysis Data obtained from image analysis and expressed in terms of R, G, B, CIE L*, CIE a*, and CIE b* colour coordinates, green area (%) and veraison area (%) were statistically analyzed applying a two way analysis of variance ANOVA with R version 3.3.1 as well as a Principal Component Analysis (PCA) with PAST version 2.17c. 3. Results and discussion The analysis of variance of the veraison area shows highly significant differences between the two studied cultivars as well as between the ripening classes. The obtained results are reported in table 1. Moreover, Figure 4, representing the mean plot of the veraison area (%) according to the ripening class of both cultivars, shows that for the first three classes the drupes belonging to the studied cultivars may have a similar 357 behaviour in colour attribute changes. This tendency disappears when ripening reaches its advanced stages. Here, both of the cultivars and the ripening classes are clearly distinguishable. In addition, a fast colour change that occurs particularly between the second and the forth classes can be observed through the mean plot. Table 1: Two-way ANOVA of veraison area (%) of the drupes according to cultivar and ripening class. α=0.05. Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1. α=0.05 Sum Sq Df F value Pr(>F) Cultivar 3043 1 33.5671 9.111e-08 *** Ripening class Cultivar x Ripening class Residuals 170390 1384 8522 5 4 94 375.8900 3.8176 < 2.2e-16 *** 0.006403 ** Figure 4: The mean plot (± stand dev.) of the veraison area (%) according to the ripening class of both cultivars. As previously mentioned, the output of image analysis were expressed in terms of RGB and CIELAB colour coordinates, the green area (%) and the veraison area (%). Therefore, In order to compress and interpret the internal relationships between these variables, and at the same time check it out whether some of these are able to separate the analyzed ripening classes, PCA was applied as carried out by Benalia et al. (2016). The results show that for both cultivars the main variability is expressed according to the first two components (Figures 5 and 6). For cv. ‘Carolea’, the first component is positively influenced by the coordinate CIE b* of the green area (which represents the yellow when assuming positive values, as in this case, or blue when assuming negative value), as well as by the green area percentage, characterizing therefore, the first (unripen) class. Besides, this component is negatively influenced by veraison area percentage, CIE a* of both of the green area and the ripen area (which represents the red when values are positive or green when values are negative), as well as the coordinate B of the green area, describing the ripen class. The coordinates R, CIE L*, G and CIE b* of the ripen area exert a positive influence on the second component characterizing the ripening progress, while the coordinates R, CIE L*, G of the green area have a negative effect on this component. Concerning ‘Nocellara’, the PCA results show that the first component is mainly influenced by the coordinates G, CIE L*, CIE b* and R of the green area characterizing the unripen drupes, while it is negatively affected by CIE a* of the green area and the percentage of the ripen area distinguishing therefore the ripen drupes. On the other hand, the second component is affected by the coordinates R, CIE L*, G and B of the ripen area characterizing ripening progress. 358 Both of PCA scatter diagram illustrate that the visually categorized ripening classes are somehow overlapped over the ripening process, indicating the inaccuracies of such a segregating method. This can be solved using numerical data from image analysis and applying multivariate analysis to look for those variable that enable an accurate segregation. Figure 5: PCA scatter diagram of the ripening classes for olives cv. ‘Carolea’ Figure 6: PCA scatter diagram of the ripening classes for olives cv. ‘Nocellara’. 4. Conclusions The present study aimed to assess colour attributes of two olive (Olea europaea L.) cultivars, i.e., ‘Carolea’ and ‘Nocellara’ according to their ripening stage using a computer vision system and image analysis. The obtained results show that the employment of computer vision system provide useful preliminary information regarding the determination of those parameters that better allow the classification of the drupes according to their ripening degree. Indeed, according to these results CIE b* of the green area could be used to 359 characterize the unripen olives, while the coordinate CIE a* describes more the ripen drupes, independently from the cultivar. Further studies are needed to confirm the obtained results in view to develop an algorithm able to separate correctly the drupes in an automated process. Acknowledgments The present study was carried out in the framework of a Post-Doc scholarship provided by the Commerce Chamber of Reggio Calabria and the University Mediterranea of Reggio Calabria. Authors are thankful to Olearia San Giorgio farm owners for their support as well as for providing the olives needed in the experimental trials. Reference Abdelhedi S., Taouil K., Hadjkacem B., 2012, Design of Automatic Vision-Based Inspection System for Monitoring in an Olive Oil Bottling Line, International Journal of Computer Applications, 51(21), 39–46. Benalia S., Bernardi B., Cubero S., Leuzzi A., Larizza M., Blasco J., 2015, Preliminary trials on hyperspectral imaging implementation to detect mycotoxins in dried figs, Chemical Engineering Transactions, 44, 157- 162. DOI: 10.3303/CET1544027 Benalia S., Cubero S., Prats-Montalbán J.M., Bernardi B., Zimbalatti G., Blasco J., 2016, Computer vision for automatic quality inspection of dried figs (Ficus carica L.) in real-time, Computers and Electronics in Agriculture, 120, 17-25, Doi.org/10.1016/j.compag.2015.11.002. Bernardi B., Zimbalatti G., Proto A.R., Benalia S., Fazari A., Callea P., 2013, Mechanical grading in PGI Tropea red onion post-harvest operations, Journal of Agricultural Engineering, 44(e63), 317-322. DOI: 10.4081/jae.2013.(s1):e63 COI (International Olive Council), 2011, Guide for the determination of the characteristics of oil olives, COI/OH/Doc. No 1 November 2011 < www.internationaloliveoil.org> accessed 20.04.2017 Díaz R., Faus G., Blasco M., Blasco J., Moltó E., 2000, The Application of a Fast Algorithm for the Classification of Olives by Machine Vision, Food Research International, 33(3-4), 305–9. Díaz R., Gil L., Serrano G., Blasco M., Moltó E., Blasco J., 2004, Comparison of three algorithms in the classification of table olives by means of Computer Vision, Journal of Food Engineering, 61, 101-107. Essiari M., Zouhair R., Chimi H., 2014, Contribution to the study of the typical characteristics of the virgin olive oils produced in the region of Sais (Morocco), OLIVÆ, 119, 8-21. Fadilah N., Mohamad-Saleh J., Halim Z.A., Ibrahim H., Ali, S.S.S., 2012, Intelligent Color Vision System for Ripeness Classification of Oil Palm Fresh Fruit Bunch, Sensors, 12(10), 14179–14195. Doi.org/10.3390/s121014179 Furferi R., Governi L., Volpe Y., 2010, ANN-Based Method for Olive Ripening Index Automatic Prediction, Journal of Food Engineering, 101(3), 318–28. Doi.org/10.1016/j.jfoodeng.2010.07.016). Kanade A., Shaligram A., 2015, Development of machine vision based system for classification of Guava fruits on the basis of CIE1931 chromaticity coordinates, 2nd International Symposium on Physics and Technology of Sensors (ISPTS), Pune, 2015, pp. 177-180. Doi: 10.1109/ISPTS.2015.7220107 Mendoza F., Aguilera J. M., 2004, Application of image analysis for classification of ripening bananas, Journal of Food Science, 69, E471–E477 Mendoza F., Dejmek P., Aguilera J.M., 2006, Calibrated Color Measurements of Agricultural Foods Using Image Analysis, Postharvest Biology and Technology, 41(3), 285–95. Menesatti P., Angelini C., Pallottino F., Antonucci F., Aguzzi J., Costa C., 2012, RGB colour calibration for quantitative image analysis: The "3D Thin-Plate Spline" warping approach, Sensors, 12, 7063-7079 Ram T., Wiesman Z., Parmet I., Edan Y., 2010, Olive Oil Content Prediction Models Based on Image Processing, Biosystems Engineering, 105(2), 221–32. Doi.org/10.1016/j.biosystemseng.2009.10.011. Riquelme M.T., Barreiro P., Ruiz-Altisent M., Valero C., 2008, Olive Classification according to External Damage Using Image Analysis, Journal of Food Engineering, 87, 371–79. Valous N.A., MendozaF., Sun D.W. , Allen P., 2009, Colour Calibration of a Laboratory Computer Vision System for Quality Evaluation of Pre-Sliced Hams, Meat Science, 81(1), 132–41. Vélez-Rivera N., Blasco J., Chanona-Pérez J., Calderón-Domínguez G., Perea-Flores M.J., Arzate-Vázquez I., Cubero S., Farrera-Rebollo R., 2014, Computer Vision System Applied to Classification of “Manila” Mangoes During Ripening Process, Food Bioprocess Technol, 7, 1183. Doi:10.1007/s11947-013-1142-4 360