HUNGARIAN JOURNAL OF INDUSTRY AND CHEMISTRY Vol. 49(2) pp. 3–7 (2021) hjic.mk.uni-pannon.hu DOI: 10.33927/hjic-2021-13 3D SCANNING AND MODEL ERROR DISTRIBUTION-BASED CHARAC- TERISATION OF WELDING DEFECTS JÁNOS HEGEDŰS-KUTI *1,2 , JÓZSEF SZŐLŐSI1 , DÁNIEL VARGA1 , GÁBOR FARKAS1 , TAMÁS RUPPERT2 , JÁNOS ABONYI2 , AND MÁTYÁS ANDÓ1 1Savaria Institute of Technology, Eötvös Loránd University, Károlyi Gáspár tér 4, Szombathely, 9700, HUNGARY 2Complex Systems Monitoring Research Group, Research Centre for Biochemical, Environmental and Chemical Engineering, University of Pannonia, Egyetem u. 10, Veszprém, 8200, HUNGARY The inspection of welded structures requires particular attention due to many aspects that define the quality of the product. Deciding on the suitability of welds is a complex process. This work aims to propose a method that can support this qualification. This paper presents a state-of-the-art data-driven evaluation method and its application in the quality assessment of welds. Image processing and CAD modelling software was applied to generate a reference using the Iterative Closest Point algorithm that can be used to generate datasets which represent the model errors. The results demonstrate that the distribution of these variables characterises the typical welding defects. Based on the automated analysis of these distributions, it is possible to reduce the turnaround time of testing, thereby improving the productivity of welding processes. Keywords: Industry 4.0, Welding technology, 3D scanner, Iterative Closest Point, defect analysis 1. Introduction During the Fourth Industrial Revolution, more modern industrial processes and tools, e.g., various testing in- struments, are present in many technological fields. Ad- vanced industrial tools are used in these technological environments [1, 2]. In addition to this new economic approach, the implementation of welding with a higher degree of automation has been a technological facilita- tion [3], while the complexity of the system has required the involvement of new disciplines such as applied in- formatics. The need to achieve autonomous production also means that quality control of the welding task car- ried out is also a significant concern [4]. In the mean- time, human workers are also utilized [5]. The issues of compliance and standards of quality have become more prominent following adoption of the quality management principle [6]. The standard includes three distinct quality categories as well as indicates acceptability by the terms “not per- mitted” and “permitted”. The acceptability of a welded joint is based on the visible absence of defects in the weld and the adequacy of all the types of tests described in the relevant standard test procedure [4]. The basic physi- cal principle of the various non-contact inspection tools, *Correspondence: hj@inf.elte.hu which are successfully used in many industrial applica- tions to detect quality non-conformities, relies on image recognition and image processing research. The future- oriented results of this research have also appeared in new fields such as face recognition by comparing three- dimensional objects or even in the development of image processing systems for self-driving cars [7, 8]. These applications have opened up new dimensions in the context of digitaLIsation, making industrial processes faster as well as easier and, therefore, more efficient [9]. Advanced image processing technologies are available in materials testing laboratories, helping researchers to study changes in material structures [10]. These method- ological developments support the basic idea of digitisa- tion, as they provide alternatives to the manual analysis of the past. Structured light scanners work in a similar manner to laser scanners, but the light source is a high-resolution projector that projects a raster mesh onto the surface of the object. The optics measure both the reticle distor- tion and intensity of the reflected light, providing a much more accurate result than variants based on lasers - with a resolution of up to 10 microns. One possible solution for processing images from structured light scanners is to use the Iterative Closest Point (ICP) algorithm, which is used to create pairwise correspondences between two https://doi.org/10.33927/hjic-2021-13 mailto:hj@inf.elte.hu 4 HEGEDŰS-KUTI, SZŐLŐSI, VARGA, FARKAS, RUPPERT, ABONYI, AND ANDÓ Figure 1: The steps of the proposed weld defect analysis method cloud points. The method was developed in studies from the early 1990s by Besl and McKay [11] as well as Yang and Medioni [12]. This iterative algorithm is known in the field of matching three-dimensional shapes, a process in which the points of the source point cloud are matched to the points of the target point cloud (also known as the reference cloud). The matching is based on the root mean square error method, where originally a point-point met- ric was to be used, while other techniques use a point- plane metric [7]. In the following, the method is presented in Section 2, where sampling, point cloud generation, the use of the Fast Global Registration and ICP algorithm as well as the Root Mean Square Error is described. Finally, Section 3 presents the results of the developed framework. 2. The proposed CAD model–based method of the generation and analysis of error distributions The proposed framework starts by scanning with a 3D scanner before reducing the number of points in the re- sulting point clouds. This is followed by a coarse and then a fine fitting, which uses a histogram containing a general comparison of the points and a colour map visualization. Its applicability in a specific technological field has been demonstrated. The objective is to investigate the weld ge- ometry of selected welded samples using the available imaging tool to detect discrepancies. A point cloud is generated on the CAD models and the welded seams pre- pared using a camera system. The technique of matching point clouds by fast global registration is presented. An iterative closest point algorithm and a histogram of point features are used to perform the refined transformation. The comparison is carried out within a Python framework to solve the problem of matching the evaluation method to a neural network in topic processing. Finally, the devi- ations resulting from the comparison are evaluated in the light of the relevant standard. The test method is illustrated in detail in Fig. 1. The test starts by generating a reference point cloud, that is, a scanned point cloud of a welded workpiece (or a point cloud of a part created by CAD software), which is con- sidered to be flawless. For the purpose of defect detection, the principle is to fit the original point cloud to the point cloud to be inspected, the geometry of which is nearly identical. As a second step, CAD models were created for which a single defect was modelled [13]. These de- fect modes were compared to the flawless CAD model. The analysis aims to obtain the deviations from the point cloud comparison in terms of the specific deviations and explore their correlations. The scanning process is as fol- lows: • When calibrating the tray, the grid is projected by the projector; • Background scanning, after inserting a turntable; • Positioning of the workpiece at the centre of the turntable; • Start the scanning process; • Create the three-dimensional model from the im- ages; • Fusion process from the image fragments can be ex- ported in other (.stl/.obj) formats. For optimal point matching, distance-based density distribution (sampling) is required. This is important because of the selected order of magnitude (250, 000 points). The differences between the point clouds can be detected. An approximate - feature-based - registration, FGR (Fast Global Registration), is first performed based on the paired points to compare the two point clouds. This corresponds to an approximate estimate, which looks for similar corners, points and parts before pairing them. Al- though the approach used is faster than ICP, it still re- quires a local refinement algorithm. Since the perfor- mance is only flawless on a heavily sampled point cloud, a more accurate fitting procedure is also used to further refine the fitting. The ICP algorithm is a refined trans- formation that more closely aligns the two point clouds, Hungarian Journal of Industry and Chemistry 3D SCANNING AND CHARACTERISATION OF WELDING DEFECTS 5 which usually iterates over two steps. First, a match is searched for from the target point cloud and then the transformation is updated by minimizing the objective function defined by the correspondence set. The normal vector of a plane is the unit vector per- pendicular to its plane and the normal vector is the vec- tor perpendicular to the surface at a point on the surface. Calculating the normal vector of the point of detection involves a solution method based on the surface grid and a solution method based on the distribution of points in the surrounding neighbourhood. The axis, on which the normal vector is located, is the most dispersed direction of the neighbourhood distribution. A final geometric feature is computed using the Point Feature Histogram (PFH), which is obtained by calcu- lating the variation ratio between all pairs of points in the neighbourhood of the points. This high-dimensional space provides useful feature representations as well as can cope with different sampling densities and noise lev- els of neighbours. The PFH stores these variables to- gether with the Euclidean distance between points and generates a histogram after computing all the pairs. The final descriptor is the sum of the histograms of each vari- able. Then, after selecting a percentage value (in this case 97%), a colour scale-based (green-near; red-distant) no- tation is applied according to the location of the pairs of points within a pixel distance of each other. The deviation from the reference area is detected using the mean square deviation formula. The recom- mended setup parameters observed during the study and the results are detailed in the next section. 3. Results A Telwin Inverpulse 320 welding machine was used to create welded samples. 8 − 8 specimens, 50 mm long with a5 welds (dimension of the height dimension of the triangle enclosing the corner weld), were T-welded. One side of the test specimens had a good weld configuration, while the other side had an unsatisfactory weld configu- ration. The welded samples were scanned using a HP 3D Structured Light Scanner 5 Pro Edition (software version 5.2.0.790) and the resulting point clouds compared to the point cloud of the reference part. It is important to note a few details about the 3D scanning process: • Environmental preparation (from the projector light, from the harsh light in the room); • System preparation (position the camera 6-8 cm to the left of the projector, adjust the camera angle to 10-12◦ from the guide rail perpendicular to the pro- jector); • Tripod adjustment up to 30◦ top view of the object. In the second step of the test series, CAD mod- els using the Solid Edge Academic Edition (version 221.00.00.00.114) were created for which a single defect was generated and compared to the flawless CAD model. An algorithm running in Python (version 3.7.7) and using the Open3D (version 21.1.3) visualization interface was used to create a framework for the comparison. The study aims to determine the deviations from the point cloud comparison and to write correlations between them. The analysis method is similar to the first process step, but here no scanning process is required. The PFH in Fig. 2 indicates whether the points can be grouped according to their distribution or identified by their location. The histogram shows the distances of the deviations. The typical distance values from the histogram data are expected to be close to the calculated deviation limits. A significant achievement in welding quality management is the application of the standard ISO 5817 : 2014, which lists surface and volume deviations with the display of de- fect categories and codes. The standard includes limits in three quality categories or indicates acceptability by the terms not permitted and permitted. The acceptability of a welded joint is based on the visible absence of defects in the weld and the adequacy of all the types of tests de- scribed in the relevant test procedure standard. The qual- ity assessment of a welded joint is, therefore, a complex task. As shown in Fig. 2, the evaluation is defined with different marginal deviations in each of the three qual- ity categories (B, C, D). By progressing down to BCD, the standard becomes more permissive; for example, the height of the excessive convexity can be up to 3 mm for category B, up to 4 mm for category C and up to 5 mm for category D. The summary results for each of the three root sizes are as follows: • 1. Crack: no borderline deviation is allowed in any category. However, for the test, a "brick body cavity" of 0.5 mm × 0.2 mm × 0.1 mm was created in the CAD model. The deviation could not be detected by the algorithm used; • 2. Cavities: modelled as a 2 mm deep cone-shaped material void in category D and a 1 mm deep cone- shaped material void in category C. No deviation is allowed in category B. The deviation cannot be detected even by the most sensitive adjustments; • 3. Solid inclusion: 1 mm in category D, 0.5 mm in categories C and B. The algorithm detects special distributions. 4. Summary An image and data analyses-based method for the qual- ification of welding defects was developed. The corner welded joints were studied and the quality deviations of such welded joints observed, these deviations being de- fects that can be detected by visual inspection. For the detection of volumetric defects, the industry has practi- cally and consistently used X-ray and ultrasonic testing. A results-oriented approach to the CAD visualisation of 49(2) pp. 3–7 (2021) 6 HEGEDŰS-KUTI, SZŐLŐSI, VARGA, FARKAS, RUPPERT, ABONYI, AND ANDÓ Figure 2: Exemplary distributions, CAD models of welding deviations . defects has been adopted. Since several defects can oc- cur on a real welded part, it can be said that by including CAD models of each deviation in the analysis, the de- viations could be kept under control. Even though this tool alone cannot be used to perform a comprehensive analysis of a weld, when complemented with other pro- cedures, the steps of the welding process analysis can be accelerated. Our investigations have led to the conclusion that, apart from defective joints, all the other categories of defects listed can be detected with certain limitations. The results, treated as differences between components of point clouds, can be subsequently identified and grouped. REFERENCES [1] Mishra, D.; Pal, S.K.; Chakravarty, D.: Industry 4.0 in Welding. Welding Technology, 2021, 253–298 DOI: 10.1007/978-3-030-63986-0 [2] Farkas, A.: Impact of Industry 4.0 on robotic welding, XXIII International Conference on Manufacturing (Manufacturing 2018), IOP Conf. Ser.: Mater. Sci. Eng., 2018, 448, 012034 DOI: 10.1088/1757-899X/448/1/012034 [3] Frolov, A.: Automation the Welding Trajectory Control. 2020 International Multi-Conference on Industrial Engineering and Modern Technolo- gies (FarEastCon) (IEEE), 1–5 DOI: 10.1109/FarEast- Con50210.2020.9271607 [4] Stavridis, J.; Papacharalampopoulos, A.; Stav- ropoulos, P.: Quality assessment in laser welding: a critical review. Int. J. Adv. Manuf. Tech., 2018, 94(5), 1825–1847 DOI: 10.1007/s00170-017-0461-4 [5] Ruppert, T.; Abonyi, J.: Worker movement dia- gram based stochastic model of open paced convey- ors. Hung. J. Ind. Chem., 2018, 46(2), 55–62 DOI: 10.1515/hjic-2018-0019 [6] László, S.: Hegesztés és rokon technológiák. Gépi- pari Tudományos Egyesület, Budapest, 2007 , 191– 204 [7] Zhang, Z.: Iterative point matching for registration of free-form curves and surfaces. Int. J. Comput. Vis., 1994, 13(2), 119–152 DOI: 10.1007/BF01427149 [8] Lee, M.: An analysis of the effects of artificial in- telligence on electric vehicle technology innovation using patent data. World Pat. Inf., 2020, 63, 102002 DOI: 10.1016/j.wpi.2020.102002 [9] Urminsky, J.; Marônek, M.; Jǎna, M.; Morovič, L.: Analysis of weld joint deformations by optical 3D scanning. Acta Polytech. 2016, 56(1), 76–80 DOI: 10.14311/APP.2016.56.0076 Hungarian Journal of Industry and Chemistry https://doi.org/10.1007/978-3-030-63986-0 https://doi.org/10.1088/1757-899X/448/1/012034 https://doi.org/10.1088/1757-899X/448/1/012034 https://doi.org/10.1109/FarEastCon50210.2020.9271607 https://doi.org/10.1109/FarEastCon50210.2020.9271607 https://doi.org/10.1007/s00170-017-0461-4 https://doi.org/10.1515/hjic-2018-0019 https://doi.org/10.1515/hjic-2018-0019 https://doi.org/10.1007/BF01427149 https://doi.org/10.1016/j.wpi.2020.102002 https://doi.org/10.14311/APP.2016.56.0076 https://doi.org/10.14311/APP.2016.56.0076 3D SCANNING AND CHARACTERISATION OF WELDING DEFECTS 7 [10] Dastres, R.; Soori, M.: Advanced Image Processing Systems. Int. J. Imaging Robot., 2021, 21(1), 27–44 https://hal.archives-ouvertes.fr/hal-03168963 [11] Wold, S.; Esbensen, K.; Geladi, P.: Principal com- ponent analysis, Chemometr. Intell. Lab. Syst., 1987, 2(1-3), 37–52 DOI: 10.1016/0169-7439(87)80084-9 [12] Chen, Y.; Medioni, G.: Object modelling by reg- istration of multiple range images. Image Vis. Comput., 1992, 10(3), 145–155 DOI: 10.1016/0262- 8856(92)90066-C [13] Bęczkowski, R.; Gucwa, M.: Defects appearing in the surfacing layers of abrasion resistant. Arch. Foundry Eng., 2016, 16(4), 23–28 DOI: 10.1515/afe- 2016-0077 49(2) pp. 3–7 (2021) https://hal.archives-ouvertes.fr/hal-03168963 https://doi.org/10.1016/0169-7439(87)80084-9 https://doi.org/10.1016/0262-8856(92)90066-C https://doi.org/10.1016/0262-8856(92)90066-C https://doi.org/10.1515/afe-2016-0077 https://doi.org/10.1515/afe-2016-0077 Introduction The proposed CAD model–based method of the generation and analysis of error distributions Results Summary