Microsoft Word - 1murphy.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 58, 2017 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Remigio Berruto, Pietro Catania, Mariangela Vallone Copyright © 2017, AIDIC Servizi S.r.l. ISBN 978-88-95608-52-5; ISSN 2283-9216 New Solutions for the Automatic Early Detection of Diseases in Vineyards through Ground Sensing Approaches Integrating LiDAR and Optical Sensors Raimondo Gallo*a, Gianluca Ristortoa, Gabriele Dagliob, Nadia Massab, Graziella Bertab, Massimo Lazzaric, Fabrizio Mazzettoa aFaculty of Science and Technology, Free University of Bozen (FUB), Piazza Universitá 5, 39100 Bolzano (Italy) bDipartimento di Scienze e Innovazione Tecnologica (DISIT), Università del Piemonte Orientale, Viale T. Michel, 11, 15121 Alessandria, Italy cDepartment of Health, Animal Science and Food Safety (VESPA), University of Milan, Via Celoria – 20133 Milano (Italy) raimondo.gallo@unibz.it Flavescence dorée and Esca are two of the most important diseases that can affect grapevine. These diseases, if not properly treated in time, are the cause of vegetative stress or death of the attacked plant, with the consequence of losses in production as well as a rising risk of propagation to the closer grapevines. Nowadays, the detection of Flavescence dorée and Esca is carried out manually through visual surveys usually done by agronomists. These activities require enormous amount of time. Up to now, a solution for a fast and early disease detection of these bacterial and fungal attack was not yet developed. Aim of this research was to test if the use of sensors typically employed in precision agriculture and robotics, mounted on different vehicles, can be a useful tool for crop monitoring purposes, principally for the recognition of disease symptoms. Therefore, two prototypes of a mobile laboratory, the ByeLab (Bionic eye laboratory) and the ATV- LAB, equipped with sensors, were tested on vineyards. The ByeLab is a remote controlled tracked vehicle, able to move between vineyards rows in an easy way also in case of difficult terrain conditions (such as slope or mud). An aluminum frame, specifically designed, was installed on the Byelab. Here, two Lidar sensors (one on the top and the other on the bottom) and six multispectral sensors (three per side, placed at different height) are fixed, respectively on frontal and lateral sides. Thanks to these sensors, the volume of the monitored grapevines and their NDVI were assessed. All the collected values are geo-localized thanks to the presence of a RTK-GNSS receiver, mounted on the top of the frame. Indeed, on the ATV-LAB only two multispectral sensors, one per side and positioned at the same height, and a RTK-GNSS receiver were installed on a support placed frontally. Both vehicles were equipped with a data-logger, in order to store all the collected data. With the aim to identify the grapevine diseases with these two solutions, a visive survey was also conducted to obtain a reference for a comparison. Through this survey, the presence of Flavescence dorée and Esca were assessed and classified according to the stage of the infection. Also the empty spaces along the row, due to missing plants or to the presence of rootstocks, were monitored. From the comparison among manual survey and the assessed NDVI collected by the ByeLab and by the ATV- LAB, the preliminary results show a capability of these systems to detect respectively 80% and 40% of the low vigor condition, as well as diseases and plant absence. From these preliminary results, the ByeLab shows a pretty good capability to identify the presence of low vigor condition along the row. 1. Introduction Flavescence dorée (FD) and Esca are two of the most important diseases of Vitis vinifera L.. FD is a phytoplasmosis belonging to the Grapevine Yellows group and it has a high epidemic behavior. Indeed, from the infection of one plant, the FD can be transmitted very quickly to the rest of the implant and to DOI: 10.3303/CET1758113 Please cite this article as: Gallo R., Ristorto G., Daglio G., Massa N., Berta G., Lazzari M., Mazzetto F., 2017, New solutions for the automatic early detection of diseases in vineyards through ground sensing approaches integrating lidar and optical sensors, Chemical Engineering Transactions, 58, 673-678 DOI: 10.3303/CET1758113 673 the nearby vineyards. The symptom of the FD is a foliage color alteration, which tend towards yellow. Besides, FD can attacks also the grape shoots and bunches (Balsari, 2007). Esca is a disease caused by specific fungi. These organisms attack the woody part of the vine causing a reduction in the transmission of organic components into the plant. The low supplying of nutritive elements determines a partially or a complete desiccation of the foliage with the death of the plant. As for FD, also Esca shows discoloration of the canopy with yellowy leaves (Balsari, 2007). Both diseases, if not properly treated in time, are the causes of a vegetative stress of the attacked plants, with the consequence of small bunches of grapes and berries. In the worst situation, FD and Esca illness can lead to death the attacked plants. Therefore, the presence of these diseases determines losses in production as well as the rising of the risk of propagation to the neighbor grapevines. Generally, against these illnesses, the actions adopted by the agronomists consist in removing the branches or the explant of the infected plants. So to preserve the vineyards, all these operations of crop monitoring to detect the pathogens are very important, in order to direct agronomic and control strategies in vineyards on time. Nowadays, more times along the vegetative season, agronomists or technicians usually plan a survey in order to detect the plants health state. All the information collected during the surveys are used to decide and to plan all the required interventions to control the presence of these pathogens. Usually, during these operations, the surveyors carried out visual evaluation on the vigor of each plant, assessing the presence or the absence of the illness. These operations are highly time-consuming. Remote and proximal sensing applications are widely used in viticulture with the purpose to obtain helpful operative parameters in a fast way and on large-scale thanks to the use of specific sensors. The collected data are employed to calculate vegetative indices to represent the vigor of the crop, or yield production to edit management and productive maps (Mazzetto et al., 2010; Meggio et al., 2010; Johnson et al., 2003; Hall et al., 2003). However, due to the distances between sensors and targets, the assessment of specific aspects is not always possible. For instance, presence of leaf discoloration symptoms are not detectable with aerial images. Nevertheless, thanks to the ground-based monitoring technologies, these kind of more accurate assessments are feasible because the sensors are closer to the plant (Mazzetto et al., 2010; Rosell et al., 2009; Llorens et al., 2011). Aim of this research is to test and compare the capability of two machineries to perform crop monitoring operations through the equipment with specific sensors for a ground-based sensing approach. The main goal of the research was to detect the presence of symptoms of FD and Esca diseases as well as of openings along the vine rows. Besides this, a preliminary approach for data interpretation was also developed. The conclusive goal of the study was to develop a complete system, of crop monitoring and data interpretation, able to monitor the biomass vigor by identifying in a semi-automatic way the presence of low vigor condition along the vine rows. All the obtained results were then validated with a manual survey. 2. Materials and methods The test was planned on a Vitis vinifera var. Dolcetto plantation during the summer of 2016. The vineyards was located in the municipality of Carpeneto, Alessandria Province (N-W of Italy). The test was conducted in a vineyard trained with Guyot system where a 2.8x1 m of planting layout was adopted. The surveys were done in four neighbour vine rows. 2.1 The equipment During the test, different sensors for ground-based sensing were used. The employed sensor were developed to operate mainly in outdoor activities. To investigate the vigour of the canopy, commercial multispectral sensors were employed (OptRx™ - Ag Leader, South Riverside Drive Ames, Iowa). The functioning of sensor is based on the reflectance, which is the ratio of reflected and emitted light. This active sensor emits light at known wavelength, at VIS (670 nm), RedEdge (730 nm) and in the range of NIR (775 nm), and it calculates reflectance values for each point detected. Hence, this tool is capable to compute in real-time the NDVI (Normalized Difference Vegetation Index) of the monitored crop. Instead, to calculate the canopy thickness, its external shape and volume, a 2D-LiDAR (SICK® LMS111 LiDAR) was used. This sensor is based on ultrasonic technology and, knowing the flying time of the laser, it is able to calculate the distances between targets and laser source. At the end, all mobile lab’s displacements and all the collected points are geo- referenced thanks to a RTK-GNSS receiver (Aschtech MobileMapper® 120 - Spectra precision). The employed mobile labs were developed by being installed on two commercial vehicles, an utility ATV (ATV-Lab) and a bins fruit carrier (ByeLab – Bionic Eye Laboratory). These solutions were designed and developed to carry out crop monitoring operations mainly in orchards and vineyards. Both vehicles were properly modified with specific metal frames, on which the sensors were installed in different configuration and 674 layouts. The ByeLab is a tracked remote controlled vehicle, electrically powered. On the lateral metal structure, 6 pairs of NDVI sensors (three for each side) and two 2D-LiDARs were installed. The NDVI sensors were mounted at different heights, in order to carry out a side-view monitoring of the entire canopy face of two adjacent vine rows. Instead, on the frontal support, two 2D-LiDARs were installed, to carry out a stereoscopic scanning of the vegetative face. in order to calculate shape and volume of the monitored canopy (Bietresato et al., 2016; Rossell et al., 2009). Meanwhile, on the top of the frontal support, a RTK-GNSS receiver was installed. The sampling frequency acquisition for the RTK-GNSS and NDVI sensors were set at 1 Hz, while for the LiDAR was at 25 Hz. Besides, the ByeLab is also equipped with a field computer, necessary to manage all the sensors and to elaborate all the raw data collected in field. As reported in specific literature, a specific diagnostic algorithm was developed to extrapolate useful information from the raw data collected with the ByeLab (NDVI, canopy thickness and geographical positions) about the vigor of the monitored crop. This information are suitable to draw thematic maps to summarize if any low vigor conditions are present. For each monitoring side, the algorithm computes an NDVI matrix from the collected NDVI indexes through a "Kriging algorithm with spherical model”. Then the algorithm spread the computed NDVI matrix on the point cloud simultaneously scanned by the LiDARs. Thus, the algorithm combines NDVI and thickness values in order to discriminate the presence of healthy, low stressed, high stressed or absence of the biomass, following well determined rules (BIetresato et al., 2016). In this study, the discrimination was done considering the presence of situations with low vigor condition or not. In these conditions were considered all those situations such as the absence of plants along the row, death plant or plant with disease symptoms. At the contrary, on the ATV-Lab were mounted only a couple of NDVI sensors, one for side, while it did not employed the LiDAR technology. Both sensors were mounted on a frontal support at an adjustable width and high. During the test, the sensors were placed at 0.3 m on the left and right side of the middle support and at a height of 0.9 m. These distances were chosen in order to monitor a representative portion of the canopy. At the top of the frontal support, the RTK-GNSS receiver was fixed. The sampling frequency acquisition was set at every 0.5 m of displacement. Besides, the ATV-Lab was also equipped with an on-board computer for the sensor controlling and management of the entire datasets acquired during the surveys (Figure 1). Figure 1: The left figure shows the ByeLab, while the right shows the operator during the crop monitoring operation using the ATV-Lab. The displacement speeds of the systems were set at 0.5 m/s and 2 m/s for the ByeLand and ATV-Lab, respectively. The middle NDVI sensors mounted on the ByeLab are installed at the same height on the NDVI sensor installed on ATV-Lab. The validation of the results obtained by the before mentioned equipment were done thanks to the comparison with data manually collected by visive survey. Indeed, the ground-based sensing was planned a visive survey. During this operations all the plants which showed symptom of FD and Esca disease was recorded and georeferenced. Beside this, also all the plant absences and the vine shoots were georeferenced. 2.2 Data elaboration During the field tests was always used the same GNSS system, but it was applied on different technologies. Data collected by the ByeLab were rescaled to 1Hz, while those collected with the ATV-LAB were set with an acquisition every 30 cm. Besides, manual survey assessed and collected only the information of low vigor condition founded along the vine rows. To compare manual survey and other elaborations, was requested a 675 synchronization procedure. The synchronization started computing the distances between each records and the beginning of the vine row. From the starting point, the vine row was divided into spatial windows (SWs) of well-defined length, arbitrary chosen from 1 to 5 meter. Authors chosen these lengths as they were the most suitable to do a fast visive surveys on field. For each SW identified the synchronization methodology evaluated, for the applied techniques, the state presented inside them, as follow: • Presence of low vigor condition (openings along the row or diseased plants detected into SW); • Absence of low vigor condition (healthy plants detected into SW); • False-positive (identification of low vigor condition instead of healthy situation); • Unidentified low vigor condition (SW where low vigor conditions were not assessed by the technology). The validation of the proposed solutions was done analyzing the correspondence of the identified SWs with a low vigor or healthy condition with those assessed by the manual survey. Possible discrepancies were analyzed and classified as false-positive or undefined low vigor condition. At each of these recognitions, a weight was assigned in order to calculate a Multi-Attribute Indexes (MAI). The weights were assigned according to the importance of the parameter assessed and they were 5, 1, -1 and -2 for SWs with low vigor condition, SWs without low vigor condition, false positive and no-identified SWs with low vigor condition, respectively. The MAI was used to evaluate the accuracy among the proposed technologies. 3. Results and discussions During this study, the vigor of four vine rows was assessed by means of different technological solutions of ground sensing. The total length of the assessed rows was 72 m, the number of evaluated SWs were 73, 37, 25, 19 and 16 (rounded up and with the addition of 1 meter to ensure the SWs’ fitting) for a SWs length of 1, 2, 3, 4 and 5 meters, respectively. The original version of the algorithm developed for the ByeLab crop monitoring was opportunely modified to take into account contexts where the canopy has a specific vertical development due to the training system. Indeed, in the second version, to avoid wrong detection and to have more representative measures the NDVI was computed without applying the Kriging algorithm with spherical model, but: i. as it was collected by the middle sensor; ii. averaging the values collected by the two highest sensors; iii. averaging the values collected by all the sensors. The data acquired by the LiDARs were used as reported in literature and interpolated with the achieved NDVI (BIetresato et al., 2016), obtaining three diagnostic algorithms. The modified diagnostic algorithms can be described by the following matrix (Figure 2). Figure 2: The matrix considered for the second version of the diagnostic algorithm is here shown. Authors arbitrarily chose the NDVI and canopy thickness threshold. Four classes were identified: i) canopy with very good vigor (green), ii) states with medium levels of stress due to the possible presence of diseases or thin canopy (yellow), iii) states with high levels of stress (red), iv) death plants or openings along the row (black). Anyway, in this preliminary field study, the algorithm was partially used. Indeed, the obtained results were grouped in healthy and critical conditions (where disease and absence of the plant were considered together). To evaluate the accuracy of the compared solutions the MAI was calculated. The higher the Multi-Attribute Index, the more accurate the technology. In Figure 3 the index, aggregate for SW lengths and vine rows, are reported. The elaboration of the values acquired with the ATV-LAB configuration recorded the lowest values of MAI, considering all the SWs lengths. This aspect highlighted that this system, in the tested configuration, is not enough capable to properly detect the vegetative vigor stress and it requires improvements in order to increase its efficiency. Probably a single NDVI sensor is not enough or it is necessary to review the mounting height. The installation height of 0.9 m maybe it is not the most proper height for a representative scanning of vineyards with this training system, if only one sensor is used, because it scans only a portion of the canopy. 676 Comparing the results obtained by using the two technologies, when only one NDVI sensor was used, significant differences were obtained. Figure 3: Summary of the result obtained by MAI computation, aggregate for SW lengths and vine row. This result can be explained by the use of the thickness information, as well as by the different advancement speeds. The ATV-LAB’s displacement speed was set at about 2 m/s, while the ByeLab’s speed was set at 0.5 m/s. So the higher speed of the ATV-LAB could have negatively influenced the precision of the NDVI survey. Hence, both the canopy thickness and the advancement speeds are important parameters to be considered in order to carry out a more reliable assessment, whereas the results obtained by the ByeLab elaborations highlighted that the higher the number of NDVI sensors considered in the algorithm, the higher the reliability of the assessment. In all the SWs considered, the highest MAI score was collected for the diagnostic algorithm computed using all the sensors. In absolute, the highest MAI was collected for the SW with 2 m length for any technologies and procedures analyzed. It means that this outline is the most suitable for reliable crop monitoring operations. In the Table 1, are reported all the results found by the monitoring done at the SWs length of 2 m. As reported in the following table, it is possible to observe that the survey carried out with the ATV-LAB configuration recognized less than 40% of the low vigor conditions. Also in this case, the general low reliability of this system to detect critical state was confirmed. Table 1: Summary of the crop monitoring results obtained by the different configurations analysed. The elaborations refer at SW with 2 m length. Data are divided for vine row, and show the number of SWs recognized as healthy (H), low vigor condition (LVC), false positive (FP) or unidentified as critical (NO id). Vine Row 1 Vine Row 2 Vine Row 3 Vine Row 4 H LVC FP No id H LVC FP No id H LVC FP No id H LVC FP No id Manual survey 16 21 - - 13 24 - - 23 14 - - 17 20 - - Disease detection – ATV-LAB 15 9 1 12 12 9 1 15 21 2 2 12 17 4 0 16 BeyLab diagnostic algorithm – 1 sensor 9 17 7 4 6 18 7 6 19 4 10 4 15 3 17 2 BeyLab diagnostic algorithm – 2 sensors 8 20 8 1 4 21 9 3 15 6 8 8 12 5 15 5 BeyLab diagnostic algorithm – 3 sensors 8 18 8 3 5 19 8 5 5 11 18 3 4 19 13 1 On the contrary, the diagnostic algorithm shows the capability to assess more than 80% of the situations where low vigor conditions are present. This is more evident for that elaboration where the NDVI was collected by three sensors. Besides this, the results obtained by the ByeLab diagnostic algorithm with three sensors are the best in terms of less SWs with unidentified low vigor condition, lower than 14%. It is important to underline that in the vine row 3 and 4 was assessed a higher number of false positive than in rows 1 and 2, compromising the accuracy of the methodology. This situation can be due to the soil roughness conditions, slightly different between the two inter-rows. The terrain unevenness generates vibrations at frame level – and consequently to the sensors – compromising the data acquisition. Probably during the vine row 3 and 4 monitoring, the high level of vibrations affect the LiDAR measurements causing a mismatch in data 677 synchronization. Therefore, for further surveys, the use of an Inertial Measurement Unit (IMU) must be planned. This sensor, in real time, will correct the acquired data from the noises due to vibrations transmitted to the frame during the advancement. 4. Conclusions Aim of this work was to test two solutions of ground-based sensing, suitable to carry out operation of crop monitoring and to detect the presence of stress conditions in vineyards. The results highlighted low values of critical state identification (~40%) when one NDVI is employed (ATV-LAB). On the contrary, the diagnostic algorithm, when NDVI collected by the whole sensors is used, recognized more than 80% of the low vigor conditions manually identified. However, a large number of false positive was detected for the vine row 3 and 4, probably because of the characteristic of the operative environment. In conclusion, the ByeLab proved to be the most reliable and capable solution for crop monitoring task, using ground-based sensing technologies in comparison with the ATV-LAB, even if it is more time-consuming. Indeed, the ByeLab required 75% more time as consequence to the lower advancement speed. In the end, thanks to its construction features, the ByeLab can be implemented with automatic guidance application in order to accomplish automatic crop monitoring operations. Further studies will be conducted in order to considerer the most appropriate threshold of NDVI and canopy thickness to implement in the diagnostic algorithm in order to be able to assess also the presence of diseases. Acknowledgments Research funded by the Autonomous Province of Bozen – TN2203, MONALISA Project. The Authors would to thanks the ARVATEC as well as the Fondazione Agrion - Centro sperimentale per la vitivinicoltura - Tenuta Cannona - Carpeneto (AL) for their collaboration and support during the research activities. References Balsari P., 2007. La vite e il vino: botanica, storia e arte, alimentazione, paesaggio, coltivazione, ricerca, utilizzazione, mondo e mercato. Eds. Bayer CropScience, Milano, Italy. Bietresato, M., Carabin, G., D'Auria, D., Gallo, R., Ristorto, G., Mazzetto, F., Vidoni, R., Gasparetto, A., Scalera, L., 2016, August. A tracked mobile robotic lab for monitoring the plants volume and health. In Mechatronic and Embedded Systems and Applications (MESA), 2016 12th IEEE/ASME International Conference on (pp. 1-6). IEEE. Hall, A., Louis, J. and Lamb, D., 2003. Characterising and mapping vineyard canopy using high-spatial- resolution aerial multispectral images. Computers & Geosciences, 29(7), pp.813-822. Llorens, J., Gil, E. and Llop, J., 2011. Ultrasonic and LIDAR sensors for electronic canopy characterization in vineyards: Advances to improve pesticide application methods. Sensors, 11(2), pp.2177-2194. Mazzetto, F., Calcante, A., Mena, A. and Vercesi, A., 2010. Integration of optical and analogue sensors for monitoring canopy health and vigour in precision viticulture. Precision Agriculture, 11(6), pp.636-649. Meggio, F., Zarco-Tejada, P.J., Núñez, L.C., Sepulcre-Cantó, G., González, M.R. and Martín, P., 2010. Grape quality assessment in vineyards affected by iron deficiency chlorosis using narrow-band physiological remote sensing indices. Remote Sensing of Environment, 114(9), pp.1968-1986. Johnson, L.F., Roczen, D.E., Youkhana, S.K., Nemani, R.R. and Bosch, D.F., 2003. Mapping vineyard leaf area with multispectral satellite imagery. Computers and electronics in agriculture, 38(1), pp.33-44. Rosell, J.R., Llorens, J., Sanz, R., Arno, J., Ribes-Dasi, M., Masip, J., Escolà, A., Camp, F., Solanelles, F., Gràcia, F. and Gil, E., 2009. Obtaining the three-dimensional structure of tree orchards from remote 2D terrestrial LIDAR scanning. Agricultural and Forest Meteorology, 149(9), pp.1505-1515. 678