Microsoft Word - 54catania.docx CHEMICAL ENGINEERING TRANSACTIONS VOL. 44, 2015 A publication of The Italian Association of Chemical Engineering Online at www.aidic.it/cet Guest Editors: Riccardo Guidetti, Luigi Bodria, Stanley Best Copyright © 2015, AIDIC Servizi S.r.l., ISBN 978-88-95608-35-8; ISSN 2283-9216 Online Tomato Inspection Using X-Ray Radiographies and 3- Dimensional Shape Models Mattias van Dael M.a,*, Seppe Rogge a, Pieter Verboven a, Wouter Saeys a, Jan Sijbersb,Bart Nicolai a,c a BIOSYST-MeBioS, Catholic University Leuven, Willem De Croylaan 42, B-3001 Leuven, Belgium b iMinds-Vision Lab, Department of Physics, University of Antwerp, Universiteitsplein 1, B-2610 Wilrijk, Belgium c Flanders Centre of Postharvest Technology, Willem De Croylaan 42, B-3001 Leuven, Belgium mattias.vandael@biw.kuleuven.be A method is proposed that fits a 3-dimensional shape model (SM) on X-ray radiographies of tomatoes on a conveyor belt to allow for inspection of internal tomato quality using X-ray. For the training of the SM a set of computed tomography (CT) scans is used. From these scans, the surfaces of the fruits are extracted. Corresponding points on all these surfaces are located after which the variation in position of every point can be determined using principal component analysis (PCA). The result of this process is a mean shape with various modes of variation, which represent the variability of the shape. Any shape can then be reconstructed through a linear combination of the mean shape and its modes of variation. During runtime, the contour of every tomato is extracted onto which the SM is fitted. This allows us to accurately estimate tomato volume and 3-dimensional shape, and assess the presence of defects and other unwanted properties from X-ray radiographies in an online application. Results are promising, but show that improvement can be made by simulating radiographs from the shape model and fitting these directly to the measured radiograph. 1. Introduction Because of an ever increasing demand for a high and uniform food quality, fresh produce is subject to strict quality requirements. Tomatoes can be affected by physical defects like exocarp punctures, insect infestations or shape and firmness abnormalities. A suitable detection method for automated quality control is the use of X-ray radiographs, since both external and internal quality properties can be assessed. Additionally, the side of the fruit oriented away from the detector will be included in assessment as well, unlike in traditional camera systems utilizing the visual part of the electromagnetic spectrum. X-rays cover wavelengths between 10 and 0.01 nanometers and are typically produced by accelerating electrons in a vacuum tube through a potential difference and subsequently hitting a metal target. The electrons that hit the target release X-rays as they slow down (braking radiation or bremsstrahlung). When passing through an object, these X-rays physically interact with the object material and are partly absorbed, scattered or reflected (Barrie Smith and Webb, 2010; Nicolaï et al., 2014). The remaining X-rays are recorded on a detector, resulting in an X-ray radiograph, an image containing superimposed information or a projection of the 3D object in a 2D plane. The goal of this work is to extract as much of the 3D fruit shape information as possible from the radiographs. To this end, a shape mode (SM) is constructed using a database of tomato CT-scans. This SM is used as prior knowledge and is fitted to the radiograph by minimizing error between the projected SM and radiograph contours. In future work, the estimated shape and position of the fruit can be used to compare the measured radiographs to simulated radiographs, identifying possible defects. DOI: 10.3303/CET1544007 Please cite this article as: Van Dael M., Rogge S., Verboven P., Saeys W., Sijbers J., Nicolai B., 2015, Online tomato inspection using x-ray radiographies and 3-dimensional shape models, Chemical Engineering Transactions, 44, 37-42 DOI: 10.3303/CET1544007 37 2. Materials 24 tomatoes (Solanum lycopersicum) grown in greenhouses in the Netherlands were scanned in the Microfocus Computer Tomography AEA Tomohawk X-ray CT scanner (Philips) at the Department of Materials Engineering (MTM), KU Leuven (Belgium), equipped with an area detector. The tomatoes were scanned with a source voltage of 85 kV at 390 mA with 60 ms of exposure time and a pixel size of 104 µm. Radiographies were recorded with an angular interval of 0.5°. At every angular position, 16 frames were averaged to minimize the noise level in images. 3. Method 3.1 Shape extraction The result of CT reconstruction is a 3-dimensional matrix in which the value of every voxel is related to the local material density. Basic image processing techniques were used to extract the shape of the scanned object from this matrix, as shown in Figure 1. The first step was to threshold the matrix and create a binary mask. A 3-dimensional Canny edge detection algorithm (Canny, 1986) was applied to extract the boundary voxels of the mask. The coordinates of these points were converted from Cartesian to spherical coordinates and resampled to linearly spaced azimuth and elevation values. The result was a 2-dimensional matrix, where each pixel value represented the distance from the center of the object to the surface. Rows indicated the elevation angle, ranging between –π/2 and π/2 while columns indicated the azimuth angle, ranging between – π and π. Figure 1: Top: Slice through a reconstructed CT scan (and image processing steps in a 2D representation. Bottom: Resulting surface representation (distance to center) in the spherical and Cartesian domain. 3.2 Statistical shape model The shapes extracted from different tomato scans were manually aligned in the Cartesian domain to match stem and calyx. Subsequently PCA was applied on the aligned surfaces. To this end the surface matrices were reshaped to create feature vectors. This effectively modeled surface-to-center distance for all points, represented as the deviation from the mean shape (Heimann and Meinzer, 2009). This means that new shapes could be represented as a sum of the mean and a linear combination of the components that resulted from the PCA analysis. 3.3 Model Fitting The next step was to fit the statistical shape model to a measured radiograph. Two modes of variation should be optimized for this: the orientation of the model with respect to the source-detector pair and the weights assigned to all components in the shape model i.e. its deformation. To do this, the method shown in Figure 2 was proposed, roughly equivalent to Irving et al. (2013). In a first step, the silhouettes of the model and the radiograph were determined. The deviation between these will serve as an error function to be minimized during optimization through rotating and deforming the model. 38 Figure 2: The proposed optimization protocol. The initial mean shape model is rotated and deformed until its silhouette matches the radiograph-derived silhouette. Model projection silhouette An instance of the shape model can be represented using a point cloud. To determine its silhouette, this point cloud was projected onto a plane by removing the component normal to this plane. For example: to project the point cloud onto the YZ-plane, the X component is removed, as shown in Figure 3. To extract the silhouette from the resulting 2-dimensional point cloud, an alpha shape is computed (Edelsbrunner et al., 1983). Alpha shapes are essentially a generalization of the convex hull i.e. every convex hull is an alpha-shape but not every alpha shape is a convex hull. The value of α relates to the radius of allowed convexity: when α equals infinity, the alpha shape of a point cloud is its convex hull. The points describing the resulting alpha shape are resampled to a set number of equi-angular points. Figure 3: A point cloud representation of a surface (left). Its projection onto the YZ-plane by removing the X- component (middle) and its α-shape with α = 100 (right). Radiograph silhouette The radiograph silhouette was determined using some basic image processing techniques as shown in Figure 4. The sequence consisted of thresholding and Canny edge detection. The points describing the resulting edge were resampled to a set number of equi-angular points. 39 Figure 4: The original radiograph (a) first is thresholded (b) and the edge is detected using a Canny edge detector (c). The resulting edge was resampled onto 100 equi-angular points (d). Optimization The error function to be minimized during optimization was the deviation between model and radiograph silhouette. This was determined as the RMSE of the distance between points of both silhouettes at the same angular position, as shown in Figure 5. Figure 5: Silhouttes derived from a radiograph (left) and a model (middle).The error function is defined as the the RMSE of the distance between points at the same angular position of the different silhouettes. and -2 to +2 standard deviations for deformation parameters since 95% of variation lies within this range. Iteration ends if a maximum number of iterations or an acceptable error level was reached. Optimization of rotation and deformation happens identically. Every possible combination of parameter levels was assessed and the solution with minimum error is chosen. In the next iteration, parameter levels were centered around the solution of the previous iteration and the range was reduced. Since all surfaces and contours were defined relative to their centers, object translation in regard to the detector didn’t influence the error function. This implicates that it would be omitted from the optimization. 4. Results Figure 6 shows the mean shape of the shape model and its combination with +3 and -3 standard deviations of the first three components on the left side. The first component is related to the overall size of the model, while the second and third component skew the model in different directions. Figure 6 (right) shows the cumulative percentage of explained variance. 40 Figure 6: Left: PCA shape model. Mean shape and first 3 components, plus and minus 3 standard deviations. Right: Cumulative percentage of explained variance. Figure show the result of fitting the shape model to a single and two orthogonal radiographs respectively. Optimization settings are shown in Table 1. Figure 7: Result of fitting a model on a single radiograph (left) and the original shape (CT scan) from which the radiograph was derived (right). Figure 8: Result of fitting a model on two orthogonal radiographs (left) and the original shape (CT scan) from which the radiograph was derived (right). 41 Table 1: Optimization parameters for the results as shown in figures 7 and 8. Number of Iterations Initial angular range Angle levels Number of components Component levels Component range Remaining error Single radiograph 2 -π to π 4 5 5 -2sd to +2sd 3.5 pixels Dual radiographs 2 -π to π 4 5 5 -2sd to +2sd 5.7 pixels 5. Discussion Figure 7 shows that the optimization, i.e. the fitting of the two silhouettes to each other had good results. There is no guarantee though that the resulting 3D model actually has the same shape as the sample the radiograph originated from. The main reason is that multiple 3-dimensional shapes can actually result in the same 2-dimensional projection. To reduce the influence of this effect, the model was fitted to two orthogonal radiographs as shown in Figure 8. The sole change to the optimization process was to redefine the error function as the mean deviation of the two silhouette pairs. An important observation here is that the silhouette of the deformed model fits the radiograph silhouette better than the silhouette of the shape from which the radiograph actually was generated. This indicates that the way the silhouette is deduced from the model and/or the radiograph is not completely valid. This might improve if instead of a parallel projection a perspective projection is used when extracting the model silhouette, taking into account the geometry in which the radiograph was acquired. The problem remains though that a lot of useful information is discarded when fitting only the two silhouettes. A better alternative might be simulating a radiograph from the model and directly fitting this to the measured radiograph. After optimization, any remaining, local discrepancies might indicate sample defects, resulting in rejection. 6. Conclusion The proposed method accurately fits 3-dimensional shape models to 2d silhouettes derived from a radiograph with a better result if two orthogonal radiographs are used. The resulting model does not fully match the shape the radiograph was derived from. This is probably related to the way the silhouettes are derived from the model. Improvements are expected when simulated radiographs will be used to directly fit to the measured radiograph, omitting the need for silhouette deduction. This has the added benefit that it will allow for defect detection. Acknowledgements This research was financially supported by the European Comission (FP7 project Pick’n’Pack (nr. 311987), www.picknpack.eu) and the Flemish Agency for Innovation by Science and Technology (IWT project TomFood (SBO 120033), www.TomFood.be). This research was carried out in the context of the European COST Action FA1106 (‘QualiFruit’). References Barrie Smith, N., Webb, A., 2010. Introduction to Medical Imaging: Physics, Engineering and Clinical Applications. Cambridge University Press. Canny, J., 1986. A Computational Approach to Edge Detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI- 8, 679–698. Edelsbrunner, H., Kirkpatrick, D., Seidel, R., 1983. On the shape of a set of points in the plane. IEEE Trans. Inf. Theory 29, 551–559. Heimann, T., Meinzer, H.-P., 2009. Statistical shape models for 3D medical image segmentation: a review. Med. Image Anal. 13, 543–63. Irving, B., Douglas, T., Taylor, P., 2013. 2D X-ray airway tree segmentation by 3D deformable model projection and registration. Proc. Fifth Int. Work. Pulm. Image Anal. Nicolaï, B.M., Bulens, I., De Baerdemaeker, J., De Ketelaere, B., Lammertyn, J., Saeys, W., Verboven, P., Hertog, M.L.A.T.M., 2014. Postharvest Handling, Postharvest Handling. Elsevier. 42