AP08_3.vp 1 Introduction Despite significant advances in micro-technology lasting recent decades, precise and cost-efficient welding still poses a big challenge. Usually, tedious and error-prone manual in- teraction is required to adjust currently available welding machines to precisely join two parts with a laser beam. This adjustment is additionally exacerbated if flexible parts or parts with loosely fixed counter pieces (see Fig. 1) are to be welded. In these cases each part has to be adjusted indi- vidually or clamped precisely, which leads to an increase in production costs. One of the major applicatios of laser welding is the pro- duction of medical devices, e.g., welding the housing for cardiac pacemakers or endoscopes. Other applications are in environmental engineering, e.g., water filters that are capable of filtering harmful microorganisms out of drinking water, see Fig. 1. To provide these high-tech goods for emerging markets, a production technology is needed which meets the require- ments concerning accuracy and production costs. We there- fore aim to simplify the production process by equipping an existing assembly system with sensors and intelligence. To avoid an excessive increase in costs, we try to make efficient use of system-immanent components. More precisely – firstly, we enable the machine to acquire high-resolution images of the workpieces with a single camera, secondly we enable the positions of the workpiece to be estimated automatically, and finally we adapt the position of the laser to possible offsets be- tween nominal and detected positions. The paper is organized as follows: First, the hardware setup for welding and image acquisition is described. Second, the concept for position estimation and for transforming the welding contours is introduced. Finally, an evaluation of posi- tion estimation and a typical welding result is presented. The paper concludes with a summary and conclusions. 2 Hardware setup To perform laser welding with an assembly machine, one of the most important problems concerns the positioning of the focused laser beam on the object to be welded. The solu- tions to this problem are highly diverse, ranging from moving workpieces via translation stages to static workpieces with fi- ber lasers moved by robots. We chose a constellation which is close to the latter. The workpiece is immobile and the laser adapts to the position of the workpiece. To achieve a high dynamics of the assembly machine, a so-called scanner is em- ployed instead of a robotic arm. This scanner consists of two orthogonal turnable mirrors, which are capable of precisely positioning the laser beam in the plane of the workpieces (see Fig. 2). Corresponding to the reduced moving mass of the mirrors in comparison to a robotic arm, the dynamics and thus the throughput rates of the welding parts can be signifi- cantly increased. © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 55 Acta Polytechnica Vol. 48 No. 3/2008 Automatic Compensation of Workpiece Positioning Tolerances for Precise Laser Welding N. C. Stache, T. P. Nguyen Precise laser welding plays a fundamental role in the production of high-tech goods, particularly in precision engineering. In this working field, precise adjustment and compensation of positioning tolerances of the parts to be welded with respect to the laser beam is of paramount importance. This procedure mostly requires tedious and error-prone manual adjustment, which additionally results in a sharp increase in production costs. We therefore developed a system which automates and thus accelerates this procedure significantly. To this end, the welding machine is equipped with a camera to acquire high resolution images of the parts to be welded. In addition, a software framework is developed which enables precise automatic position detection of these parts and adjusts the position of the welding contour correspondingly. As a result, the machine is rapidly prepared for welding, and it is much more flexible in adapting to unknown parts. This paper describes the entire concept of extending a conventional welding machine with means for image acquisition and position estimation. In addition to this description, the algorithms, the results of an evaluation of position estimation, and a final welding result are presented. Keywords: Video-based position estimation, automatic laser adjustment, compensation of positioning tolerances, precise welding, robust fitting, contour point detection. Fig. 1: Prototype of a clear water filter: (a) filter membrane, (b) fil- ter body with deepening, (c) membrane loosely positioned within the deepening. To prevent it from warping, the membrane has to be welded exactly along its edge. In addition to laser positioning, this setup can be ben- eficially used for high-resolution image acquisition with a standard camera. To this end, we employ specially designed beam splitter optics, which allows the optical path of the laser through the scanner to be shared with that of the camera (see Fig. 3). Thus, the camera field-of-view (FOV) can be moved within the plane of the workpiece just like the laser. With this system we acquire high resolution image tiles and corre- sponding information about the FOV positions in order to stitch the tiles to an overall image. In contrast to systems which use static cameras, we can hence create an overall picture with much greater resolution or size. High resolution is required because the size of the workpieces is approxi- mately 6 orders of magnitude higher than the sought accu- racy of position detection. Since the diameter of the welding spot is approximately 300 �m, an offset of �50 �m is in most cases tolerable. 3 Structure of the software framework In order to implement the adaptation of the welding laser to the position of the workpiece, several steps have to be car- ried out, as follows: 1. Image acquisition and stitching. 2. Feature extraction. 3. Feature based position estimation and computation of the offset to nominal position. 4. Transformation of the welding contour. 5. Welding. First, as already mentioned, image tiles have to be ac- quired and stitched to an overall image. Since the scanner knows the positions of these tiles with sub-pixel accuracy, we can omit time-consuming matching and position estimation steps and directly stitch the tiles at the correct positions. How- ever, to avoid artifacts at the image junctions due to different brightnesses, single-band blending is used, which smoothly blends one image into the adjacent image. The second step is to define and extract the features to be used for identifying the position of the workpiece. These features have once to be defined by the user, since it is difficult to automatically omit features which are not relevant for esti- mating the subsequent position. The filter part in Fig. 1 provides a good example of this: The aim is to weld the loosely positioned membrane along its rim to the body of the filter. Therefore, it is essential to use only features of the mem- brane, e.g. its edge, instead of features of the body of the filter, which would hamper the correct detection of the membrane’s position. A further aspect that motivates the proposed feature based position estimation is related to the third step in the item list. Here, the difficulty is to obtain nominal values for the positions which are later used to determine the offsets, and to transform the welding contour correspondingly. Since these nominal values are usually available from CAD-data (Computer Aided Design), it is meaningful to use a procedure that is “compatible” with this data. As the appearance of surfaces is still only coarsely modeled in present-day CAD- -Programs, and it is often far from reality, only edges and their geometric representations appear to be suitable for being 56 © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ Acta Polytechnica Vol. 48 No. 3/2008 Fig. 2: Scheme of a galvanometer-driven scanner [4] Fig. 3: Scanner-based welding and image acquisition system. The beam splitter optics comprises a beam splitter mirror and a dichroit. The dichroit is a wavelength dependent mirror, which, in our case, reflects the Nd:YAG laser for welding and lets the illumination rays for image acquisition pass through. Fig. 4: Image tiles and stitched panorama of the clear water filter, Fig. 1 matched with real images. Thus, the nominal values for lines, circles, arcs of circles, corners and ellipses can be extracted from CAD-data without excessive effort. 4 Feature extraction The feature extraction algorithm is required to achieve high robustness, high accuracy and a high throughput rate. Since the user is required to explicitly define the features to be selected for position estimation, for the reasons given above, it is not too time-consuming to also select a region of interest (ROI) that is big enough to cover position tolerances. Within the ROI, the desired feature is extracted robustly and with sub-pixel accuracy, as described in greater detail below. The detection of only one feature per ROI may be seen as a restric- tion, but it inherently avoids ambiguity of feature detection and matching within one ROI. In addition, it has not shown to be adverse in practice. After defining the ROI and selecting one of five possible features – such as line, circle, arc of a circle, ellipse, corner – the next step is robust and sub-pixel accurate extraction. Since the size of the overall image can exceed 10 million pix- els, the ROI-size may still be about 1 megapixel and more. Well-known algorithms such as the Canny edge detector [2] in combination with the Hough Transform [7] for line or circle fitting are too costly, since they are not perfectly adapted to the problem at hand and they evaluate the entire ROI. To ob- tain the required throughput in detection, further speedup is needed. To this end, we have developed an approach which is time-efficient and robust. This approach consists of two steps: estimating the positions of contour points, and fitting a chosen model to the points determined in this way. This approach can be used for robust estimations of the parame- ters of the mentioned features, except for the edge, which is determined using the SUSAN corner detector [12]. 4.1 Contour point estimation To estimate the contour points, a predetermined number of line profiles is evaluated. These profiles are roughly po- sitioned orthogonally and equally spaced to the sought contour, when defining the ROI. Thus, the sought contour is sampled by only considering the pixels of the line profiles (and their neighborhoods), instead of evaluating the entire ROI. To make a robust estimate of the contour points within these profiles, we applied two different models. 4.1.1 Step edge model This model consists of a set of step edges with pre- defined edge positions, which are correlated with the line profiles [10]. The resulting position of the contour point is determined by evaluating which step edge model corre- sponds to the maximal correlation value. Thus, the position of the step edge in this model defines the position of the sought contour point. Since the models always cover entire profiles and, consequently, all pixels are considered, a high degree of robustness is achieved – outliers due to noise are canceled out. In addition, the step edge approach is well tailored to the problem, since it finds the one sought contour point for each profile. The step edge approach is most suitable to robustly detect the transition between two adjacent areas of slightly different gray values, even in noisy images. 4.1.2 Sobel edge model The Sobel edge model utilizes the well-known filter kernel from the Sobel operator [1]. Here, it is assumed that an edge is defined as a sharp transition orthogonal to the edge and a small transition in the direction of the edge. This model is transferred to the Sobel filter kernel, which consists of Gaussi- an smoothing combined with a derivation in the orthogonal direction. To apply this model to contour point detection, we first rotated the filter kernel so that the derivation is com- puted in the direction of the profile and the smoothing is computed orthogonally to the profile. Then, this rotated filter is applied to each pixel of the profile and its neighborhood. This procedure yields an optimal response to edges orthogo- nal to the profiles. A simplification of this procedure may be to compute the dot product of the gradient and the unit vector pointing in the direction of the profile. However, this procedure would lack the directional smoothing of the ro- tated Sobel kernel, and would therefore yield a lower degree of robustness. The Sobel edge model is suitable for detecting edges, when the step edge model cannot be applied because the constraint of large adjacent and homogeneous areas is violated. 4.2 Model fitting Since the detected contour points may not be free from outliers, it seems reasonable to use robust methods for fitting the geometric models to the points. To this end, the RANSAC algorithm (RANdom SAmple Consensus) [3, 11] is used. This algorithm is able to eliminate outliers and hence use the points that are assumed to be undisturbed for fitting. To this end, we need mathematical definitions of the geometric primitives to be fitted and a measure for assessing the dis- tance of each contour point to the given model. A description of all four models (circle, circle arc, ellipse, line) would go beyond the scope of this paper. We therefore elaborate more on the example of circle detection, since the basic steps are similar to the algorithms for the other models. The algebraic model of a circle in the plane can be ex- pressed as [5] a cT Tp p b p� � � 0 (1) with a � 0, p � ( , )x y T and b � ( , )b b T1 2 , p b, � � 2. The center of the circle c and the radius r can be determined from these parameters as c � � � � � � b a b a T 1 2 2 2 , , (2) r b b a c a � � �1 2 2 2 24 . (3) The insertion of n � 3 contour points p i i i Tx y� ( , ) ( i n� 1� ) in (1) results in an overdetermined linear system of equations, which is Bu 0� (4) © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 57 Acta Polytechnica Vol. 48 No. 3/2008 with B x y x y x y x yn n n n � � � � � � � � � � 1 2 1 2 1 1 2 2 1 1 � � � � , u � � � � � � � � � � a b b c 1 2 . B is the matrix of the contour points and u is the vector of the sought circle parameters. Thus, the problem to be minimized can be formulated as Q B B B B BT T T( ) ( ) minu u u u u u� � � �2 2 , (5) with the constraint of u 2 1� to avoid obtaining the trivial so- lution. This can be reformulated as Q B Bi i T T i i( ) minv v v� � �� , (6) where the eigenvector vopt of B TB, which belongs to the smallest eigenvalue �4, equals the sought parameter vector u. Thus, the circle parameters c and r can be determined. As already mentioned, the detected contour points may be subject to gross outliers, which can corrupt the fitting result and thus deteriorate the accuracy of the position estimation. Therefore, we utilize the RANSAC algorithm for robust out- lier elimination. In the case of circle fitting, the procedure is as follows [3]: 1. Select a set of at least three determined contour points randomly, 2. Construct a circle through these (e.g. with least squares circle fitting), 3. Count the number of points which lie within a pre- determined error tolerance to the circle (so-called inliers), 4. If the number of inliers is greater than some pre- defined threshold, do least squares circle fitting (as explained above) for all inliers, else repeat the pro- cess, i.e. start again at 1., until a maximum number of trials is reached. With this procedure, a circle can be correctly fitted even in challenging images. The use of line-shaped profiles for contour point de- tection instead of entire images or ROIs permits a speed up, which is required for executing one detection and welding cycle per second (detection with a standard PC, 2.6 GHz, Dual-Core, C�� implementation). In addition, sub-pixel ac- curacy is accomplished by means of least squares fitting to the RANSAC-Inliers. 5 Offset computation and correction In the case of extracting a single feature, e.g. a circle, it is easy to determine an offset between the detected value and a given nominal value. Since the system is restricted to the cor- rection of translation and rotation, one circle only contributes to translation estimation. Thus, correction of the welding contour is a simple shift in its coordinates. In the case of two or more detected circles with different center positions, a rotation may be additionally determined. To avoid approaches such as chamfer matching [6], which re- quire costly iterative optimization for position estimation, we used a direct algebraic and model-oriented way. In contrast to (local) optimization approaches, we thus avoid missing the global optimum, which represents the sought position. In order to apply our direct algebraic approach, which uses the RANSAC algorithm and the similarity transform [9], the different features have first to be brought to a uniform level. To this end, an examination of the descriptions of the feature is replaced by considering (center) points and angles. Thus, e.g. an ellipse is represented by its centroid and an angle, a circle is represented by its center, and so on. In the next step, the RANSAC algorithm is used to elimi- nate outliers. Then, at first, a rotation is computed by only considering the mentioned points and their corresponding nominal values (angles are considered later). Hence, we use the similarity transform in order to obtain a linear set of equations and thus avoid sine and cosine terms. The structure of the linear equation set is u u u u v v v v x y xm ym x y y x 1 1 1 1 1 1 1 0 0 1 � � � � � � � � � � � � � � � � � � � � v v v v a b c d xm ym ym xm � � � � � � � � � � � � � � � � � � � � � � � 1 0 0 1 � , (7) where u i xi yi Tu u� ( , ) , and vi xi yi Tv v� ( , ) , i m� 1� , build a set of m point pairs, which are composed of the detected points such as the centers of the circles and their corresponding nominal values. The parameters a, b, c, d form the matrix of the similarity transform and encode the sought rotation and a scaling. The rotation angle � can be computed by determin- ing the pseudoinverse of A, e.g., via singular value decompo- sition and the computation of � � arctan ( , )2 b a . (8) Between this value and the angles, possibly determined directly by the features (e.g. the angles of lines and ellipses), the weighted average is computed. After the computation of this result, the points of the feature are rotated inversely and the translation is determined as the mean of the distances between the point pairs. Hence, the sought rotation and the offset are determined robustly and without iterative optimization steps. Finally, the welding contour has to be transformed correspondingly. This is straightforwardly implemented as a multiplication of the welding coordinates with an adapted transformation matrix. 6 Experimental results To evaluate the accuracy of the position estimation, im- ages of real workpieces were acquired using a galvanometer scanner and stitched (see Fig. 5). These images were syntheti- cally rotated and shifted with sub-pixel accuracy. Hence, a reliable ground truth is created. Within these images, the pro- 58 © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ Acta Polytechnica Vol. 48 No. 3/2008 Fig. 5: Stitched images used for evaluating the accuracy in posi- tion estimation posed algorithms for feature extraction and offset estimation are applied. The results obtained with the images in Fig. 5 are presented in the boxplot [8] in Fig. 6. Despite the challenging real world images, the range of deviations for circle detection is smaller than 2 pixels, which is in our case less than 13 �m. In case of the arc of a circle, the same image is used but only a quarter of a circle is employed for position detection. This is more time efficient, since only a quarter of the overall image has to be acquired, but the accu- racy is significantly decreased. In position estimation with two orthogonally intersecting lines, the worst case scenario is a de- viation of 9 pixels, which corresponds to 60 �m. However, the image used for line detection depicts an SMD-chip (Surface Mounted Device), where the lines are fitted to the rows of pins. The rows are fragmented due to the gaps between the pins. This complicates the task of fitting a line with such high accuracy. The image of the ellipse is affected by strong arti- facts which hamper higher accuracy. In addition, the ellipse has the highest number of degrees of freedom. Since the number of contour points which are used for fitting is con- stant, the relative number (related to the degrees of freedom) for the ellipse is lower. This causes a decay in accuracy. The results of the algorithm for offset computation are used for rotating and shifting the welding contour corre- spondingly. The final aim is to execute the welding process at the correct position. In order to obtain preliminary evidence, three of the filter membranes in Fig. 1 have been welded at position variations of about 1 mm. The final position devia- tion of the overall system did not exceed 30 �m. An example of successive welding obtained with our system is presented in Fig. 7. 7 Summary, conclusions This paper addresses the problem of tedious and er- ror-prone adjustment of present-day laser welding machines. Especially for high-tech goods in medical or environmental applications, high precision is required at moderate levels of production costs. To this end, a conventional welding machine has been equipped with the capability to react auto- matically to varying positions of the parts to be welded. In this way, costly manual adjustment can be reduced to a minimum and high-tech goods could become more attractive for emer- ging markets. The proposed procedure can be subdivided into the steps of image acquisition, stitching, feature extraction, position es- timation, transformation of the welding contour and, finally, successful welding. These steps have been described and eval- © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ 59 Acta Polytechnica Vol. 48 No. 3/2008 Fig. 6: Boxplot, illustrating the deviations of 20 measurements from ground truth (“Circle x” e.g. denotes the deviations in the x-direc- tion, when a circle is used for position estimation. “Lines rot.” denotes the deviation of the determined rotation from ground truth, in degrees). The lines in the boxes mark the medians, the boxes encompass the two inner quartiles of the quantity of re- sults, while the “whiskers” and crosses mark the two remaining outer quartiles. Fig. 7: Welding result: Filter part and its weld seam uated in detail. The evaluation results show that high stan- dards of accuracy can be achieved by the entire system. Acknowledgments The authors would like to thank Professor Dr.-Ing. Til Aach, Institute of Imaging and Computer Vision, RWTH Aachen University, and Dr.-Ing. Alexander Olowinsky, Fraun- hofer Institute for Laser Technology, who supervised this project. The authors would also like to thank Dipl.-Ing. Jens Gedicke, Fraunhofer Institute for Laser Technology who, among others, was involved in creating the hardware setup, for his kind support. This work is funded by the “INTAKT” collaborative re- search project in the “InnoNet” program, funded by German Federal Ministry of Economics and Technology (BMWi) with VDI/VDE-IT. The authors gratefully acknowledge this sup- port as well as the active cooperation of all project partners. References [1] Jähne, B.: Digitale Bildverarbeitung. Springer Verlag, Berlin, Heidelberg, 6. edition, 2005. [2] Canny, J.: A Computational Approach to Edge Detec- tion. IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 8 (1986), p. 679–714. [3] Fischler, M. A., Bolles, R. C.: Random Sample Consen- sus: a Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM, Vol. 24 (1981), No. 6, p. 381–395. [4] Furlong, B., Motakef, S.: Scanning Lenses and Systems. News form CVI Melles Griot, November, 28 2007. [5] Gander, W., Golub, G. H., Strebel, R.: Least-Squares Fit- ting of Circles and Ellipses. In Numerical analysis (in hon- our of Jean Meinguet), 1996 p. 63–84. [6] Borgefors, G.: Hierarchical Chamfer Matching: A Para- metric Edge Matching Algorithm. IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 10 (1988), No. 6, p. 849–865, November 1988. [7] Hough, P. V. C., Arbor, A.: Method and Means for Recognizing Complex Patterns. United States Patent and Trademark Office, 3069654, 1962. [8] McGill, R., Tukey, W. J., Larsen, W. A.: Variations of Box Plots. In The American Statistician, Vol. 32 (1978), p. 12–16, Feb. 1978. [9] Werman, M., Weinshall, D.: Similarity and Affine Dis- tance Between 2D Point Sets. In Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 1 (1994), p. 723 – 725, Jerusalem, Israel, October 1994. IEEE. [10] Stache, N. C., Zimmer, H., Gedicke, J., Olowinsky, A., Aach, T.: Robust High-Speed Melt Pool Measurements for Laser Welding with Sputter Detection Capability. In DAGM07: 29th Annual Symposium of the German Associa- tion for Pattern Recognition, Heidelberg, September 2007. German Association for Pattern Recognition, Springer Verlag. [11] Hartley, R., Zisserman, A.: Multiple View Geometry in Com- puter Vision. Cambridge University Press, 2000. [12] Smith, S. M., Brady, J. M.: SUSAN – A New Approach to Low Level Image Processing. Technical Report TR95SMS1c, Oxford University, Chertsey, Surrey, UK, 1995. Nicolaj C. Stache e-mail: Nicolaj.Stache@lfb.rwth-aachen.de Thi Porn Nguyen Institute of Imaging & Computer Vision RWTH Aachen University D-52056 Aachen, Germany 60 © Czech Technical University Publishing House http://ctn.cvut.cz/ap/ Acta Polytechnica Vol. 48 No. 3/2008