Digital tools as part of a robotic system for adaptive manipulation and welding of objects ACTA IMEKO ISSN: 2221-870X September 2022, Volume 11, Number 3, 1 - 8 ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 1 Digital tools as part of a robotic system for adaptive manipulation and welding of objects Zuzana Kovarikova1,2, Frantisek Duchon1, Andrej Babinec1, Dusan Labat2 1 Institute of Robotics and Cybernetics, Faculty of Electrical Engineering and Information Technology, Slovak University of Technology in Bratislava, Ilkovicova 3, 812 19 Bratislava, Slovakia 2 VUEZ, a. s., Hviezdoslavova 35, 934 39 Levice, Slovakia Section: RESEARCH PAPER Keywords: Industrial robot; simulation; digital twin; robotized welding; SQL server; HMI; database; 2D scanner; 3D scanner; industrial web; automatized measuring system; Industry 4.0 Citation: Zuzana Kovarikova, Frantisek Duchon, Andrej Babinec, Dusan Labat, Digital tools as part of a robotic system for adaptive manipulation and welding of objects, Acta IMEKO, vol. 11, no. 3, article 11, September 2022, identifier: IMEKO-ACTA-11 (2022)-03-11 Section Editor: Zafar Taqvi, USA Received March 26, 2022; In final form July 22, 2022; Published September 2022 Copyright: This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This article was written thanks to the support under the Operational Program of Integrated Infrastructure for the following projects: "Robotic Workplace for Intelligent Welding of Small Volume Production (IZVAR)", code ITMS2014 +:313012P386, and "Digitization of Robotic Welding Workplace (DIROZ)", code ITMS2014 +:313012S686 (both co-financed by the European Regional Development Fund). Corresponding author: Zuzana Kovarikova, e-mail: kovarikova@vuez.sk 1. INTRODUCTION Traditionally it has been difficult to use automation in small batch production with hight variation in volumes and high mix of products [1]. There is a great potential for small batch producers to use flexible automation in manufacturing operation to remain competitive [1]. To achieve flexibility, it is crucial to design the structure of digital tools so that the greatest possible adaptability is achieved with tools that require a minimum time of conversion of technological units. The interaction between automation controllers and Computer-aided Design/Manufacturing (CAD/CAM) systems capable of offline programming, is generally a way to decrease production down time due to programming [1]. The technology components modelled in CAD tools are an important input for the creation of the digital twin. Digital twin (DT) is the technical core for establishing cyber- physical production system (CPPS) in the context of industry 4.0 [2]. The importance of digital twin (DT), which is characterized by the cyber–physical integration, is increasingly emphasized by both academia and industry [3]. Through data modelling, data are stored according to certain criteria and logic, which can facilitate data processing [3]. Theories of service modelling are useful for the identification, analysis, and upgrade of services [3]. Simulation theories are useful for operation analysis (e.g., structural strength analysis and kinetic analysis) in a simulation environment [3]. ABSTRACT The aim of this article is to describe the design and verification of digital tools usable for sharing information within a team of workers and machines that manage and execute production carried out by a robotic system. The basic method is to define the structure of the digital tool and data flows necessary to enable an exchange of data needed to perform robotic manipulation and robotic welding of variable products minimizing at the same time strenuous human activity. The proposed structure of data interconnects a set of intelligent sensors with control of 18 degrees of freedom of 3 robotic manipulators, a welding device, and a production information system. Part of the work was also to verify the functionality of the proposed structure of the digital tools. In the first phase, simulations using a digital twin prototype of the workplace for robotic manipulation and robotic welding were performed to verify the functionality the digital tools. Subsequently, a digital tool was tested in the environment of a real prototype workplace for robotic manipulation and robotic welding. Simulation results and data obtained from the prototype tests proved the functionality of the digital tool inclusive of the production information system. https://www.fei.stuba.sk/ mailto:kovarikova@vuez.sk ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 2 Digital Twins are more than just pure data, they include algorithms, which describe their real counterpart and take a decision in the production system. [4]. DTs can make a production process more reliable, flexible, and predictable [3]. Above all, DTs can visualize and update the real-time status, which is useful for monitoring a production process [3]. 2. ROBOTIC SYSTEM FOR ADAPTIVE MANIPULATION AND WELDING OF OBJECTS The robotic system for adaptive manipulation and welding of objects shown in Figure 1 consists of three robots. There are two robots for manipulating the to-be-welded parts and one robot for tungsten inert gas welding. The welding robot can scan the welding gap and the surface parameters of the final weld using a 2D laser scanner installed on the robot body. There are also two warehouses of parts equipped with 3D laser scanners, each of them installed above the concerned warehouse. The 3D laser scanners give feedback to the control system of robotically positioned to-be-welded parts. The source of energy needed for welding in the robotic system for adaptive manipulation and welding of objects is a robotic welding machine. The robotic welder is equipped with a digital interface enabling to set its parameters from the central control system by digital communication. Adaptation of the workplace for handling of parts and products of various shapes is enabled by a quick-change system of robotic grippers. Each robotic manipulator has one robotic gripper stand with six positions for setting up effectors of different types. The robotic system for adaptive manipulation and welding of objects also includes an automated system for measuring process quantities which is connected to the production and quality information system by digital communication. Coordination of workplace components is ensured by a central control system which is a mediator and provider of process variables scanned at the workplace. The process variables are provided to the human-machine interface and the SQL database where digital data are registered and archived. Power and media distribution systems provide operating power distribution, air distribution for operation of grippers and inert gas distribution to create a protective atmosphere. 3. DIGITAL TOOLS AND FLOW OF DIGITAL DATA The diagram in Figure 2 shows the digital tools and the flow of digital data. An important source as well as consumer of the data is the prototype of the workplace itself. The data of the Figure 1. Robotic system for adaptive manipulation and welding of objects. Figure 2. Digital tools and data flow in prototype of the robotic system for adaptive manipulation and welding of objects. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 3 robotized process is read, and the required parameters are set using a programmable logic controller. The programmable logic controller is a central control system of the robotic system for adaptive manipulation and welding of objects. 3.1. Robot controllers There are three robot controllers in the prototype of the robotic system for adaptive manipulation and welding of objects. Two controllers control robots for manipulation of to-be-welded objects, one of them also manipulation of products after welding operation. One robot controller controls trajectory of the tool centre point, i.e. tungsten electrode tip. 3.2. Robotic removal of to-be-welded parts from 3D laser scanners Robotic positioning of the to-be-welded parts removed from the parts warehouse to a position close to the welding position is performed automatically based on the feedback from 3D laser scanners. Each robotic manipulator has one 3D laser scanner able to obtain point cloud data characterizing the state of the warehouse of the stored parts before every single removal. This automatic robotic manipulation process is shown in Figure 3. 3.3. 2D laser measurement system The welding robot is equipped with a 2D laser sensor that reads data needed to evaluate the geometry of the weld gap. The measured data are used to correct the weld gap, to generate the welding trajectory, and to automate the quality control of the performed welds. 3.4. FTP server The FTP server contains data in the file format. CAD data corresponding to the design of the prototype workplace (including the design of variable robotic grippers and support constructions for setting up parts in warehouses) is stored in this data repository. Data on the required design of the positioned parts, to-be-welded parts, and welded parts defining the desired product are also stored here. The FTP server contains also robotic system simulation files for adaptive manipulation and welding of objects as well as thermodynamic simulation files. In addition, the FTP server stores files with measured data obtained from the 2D laser scanner, thermovision camera measurements as well as historical trends from an automated system for process measurements, and photo documentation. 3.5. SQL server The database of the robotic system for adaptive manipulation and welding of objects is implemented and operated in the MS SQL server environment. The database contains records of robotic welds, data on required technological parameters of robotic welding and measured data of process variables. 3.6. Web server The web server of the robotic system for adaptive manipulation and welding of objects provides data from the database in MS SQL through web pages in the form of tables and behaviour of selected quantities in graphical form. It also allows the welding technologist to enter the required values of welding parameters to optimize robotic welding. 3.7. Human-machine interface The human-machine interface (Figure 4) for controlling and monitoring the state of the robotic system for adaptive manipulation and welding of objects provides windows for setting the robotic welding parameters and for monitoring the robotic welding process, a control panel for controlling the prototype workplace, and tools for viewing measured process variables in both tabular and graphical forms. The human- machine interface is installed on operator panels and computers with visualization immediately next to the prototype workplace. 3.8. Quality control The quality control of the final product is implemented in two ways. The first one is an automated weld inspection performed immediately after welding by robots using the 2D laser scanner installed on the welding robot. Automated quality control data is recorded and archived automatically via measurement data files. The second one is a manual quality control of the final product performed by a person who enters the control results into pre- prepared protocols archived in the FTP server. 4. SIMULATION OF PROTOTYPE ROBOTIC SYSTEM FOR ADAPTIVE MANIPULATION AND WELDING OF OBJECTS Simulation of both the robotic positioning of parts and the robotic welding is preceded by testing of the feasibility to obtain a quality product in the prototype. For this simulation, a digital twin of the robotic system for adaptive manipulation and welding of objects is used. Figure 3. Robotic removal of to-be-welded parts from 3D laser scanners. Figure 4. Human-machine interface of the robotic system for adaptive manipulation and welding of objects. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 4 The advantage of verifying the design by means of a digital twin consists in the possibility of tuning a correct synchronization of the movements of three robotic manipulators which operate in a common working space during robotic positioning and welding of the product. In this way, it is possible to prevent potential collision events and to optimize the cycle time of welding and handling processes. In the digital twin, it is also possible to verify correctness of the design of robotic grippers. When simulating the robotic manipulation of a given part, it is necessary to take into account its geometry and weight. In the digital twin, dynamic events that occur during the positioning of welded parts can also be simulated. Figure 5 shows the verification of the automated robotic grip of a selected robotic gripper from a quick-change robotic gripper system. Attachments of robotic grippers of various types can be verified by simulation via a digital twin which contains a quick- change system of robot effectors. Before testing the robotic welding in the prototype workplace, a thermodynamic simulation is performed simulating the propagation of heat from the weld site into the welded body. A render of the thermodynamic simulation is shown in Figure 6. Thermodynamic simulations are compared with temperature data measured by a thermovision camera during tests of welding in the prototype workplace and help to optimize preparation of specifications of robotic TIG welding parameters. 5. SIMULATION AND TESTING OUTPUTS The simulation of the process of robotic positioning and robotic welding for both fillet welds and butt welds was verified by the simulation. The design of the robotic positioning of the final product into the robotic ultrasound diagnostic workplace was also verified by means of a digital twin. Figure 7 shows robotic manipulation of two to-be-welded flat sheets (UP) and robotic welding during performing a fillet weld of them (DOWN). With the same robotic fingers, the sheets were robotically positioned and held also when performing a butt weld as shown in Figure 8. Figure 5. Simulation of changing robot’s grippers in digital twin. Figure 6. Thermodynamic simulation of TIG-welding two tubes together. Figure 7. UP: Robotic manipulation of two to-be-welded flat sheets - simulation. DOWN: Robotic holding of two to-be-welded flat sheets - simulation. Figure 8. Robotic positioning of to-be-welded parts in a prototype robotic workplace. Position before weld gap correction from data obtain by 2D laser scanner. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 5 Testing of robotic positioning of to-be-welded parts in a prototype robotic workplace showed sliding of the sheets in this type of fingers, which had a negative impact on the feasibility and quality of the weld. For this reason, a different type of fingers was designed, as shown in Figure 9. During the design of these fingers, the Ansys calculation program was used in which the design was optimized so that the finger was as light as possible and at the same time sufficiently strong. Figure 9 on the left shows the designed robotic finger before performing the optimization. The finger after optimization is shown on the right. The described robotic system was designed for adaptive manipulation and welding. Thus, with the help of robotic manipulators, it is possible to position and hold objects of different sizes and shapes during robotic welding. The designs of all considered scenarios of robotic positioning are verified in advance by the digital twin of the workplace of robotic manipulation and robotic welding. Figure 10 shows the output of the simulation of robotic manipulation and robotic welding of cylindrical objects by realization of butt welding. The upper part shows a simulation of the simultaneous positioning of three robotic manipulators during the welding. After welding, the welding robot is repositioned to its home position by the central control system. The first robotic manipulator is instructed to open the gripper that held the part during the welding process. Subsequently, with the second robotic manipulator, the final product is robotically positioned in the output warehouse. The output of the robotic positioning simulation of the final product by the second robotic manipulator is shown in Figure 10 below. After obtaining suitable robotic trajectories of both robotic manipulators for positioning and holding to-be-welded parts as well as trajectories of the welding robot, the parameters obtained in the digital twin technology were verified in a prototype robotic positioning and welding workplace. Photographs from the testing of the robotic process in the workplace prototype are shown in Figure 11. The upper part shows the robotic positioning of to-be-welded parts and the automated measurement of the weld gap by a laser scanner before its correction. After automatic correction of the weld gap, Figure 9. Design of the robotic fingers. Left – before optimization in Ansys. Right – after optimization in Ansys. Figure 10. UP: Robotic holding during welding of two to-be-welded cylindrical objects - simulation. BELOW: Robotic manipulation of welded cylindrical object - simulation. Figure 11. Cylindrical objects before their automatic weld-gap correction and during robotized welding in the workspace. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 6 synchronized positioning of robotic manipulators holding parts and robotic welding is performed as shown in Figure 11 below. The robotic workplace of handling and welding is followed by the workplace of robotic ultrasonic inspection for proving the quality of welds by a non-destructive method. One of the robotic manipulators of the welding workplace is used for positioning of the welded and cooled products in the workplace of robotic ultrasound diagnostics as it is shown in Figure 12. Positioning processes are synchronized with each other. After performing robotic ultrasound diagnostics, the tested product is again positioned by a robotic manipulator and placed in a warehouse of quality products or failures, depending on the results of the implemented non-destructive inspection. Figure 13 shows robotic manipulation of the to-be-tested object in real workspace. 6. TESTING THE FUNCTIONALITY OF PROTOTYPE DIGITAL DATA TOOLS Testing of robotic manipulation and robotic welding as well as testing of functionality of digital tools as an effective means of data exchange between the workplace and the workplace operators, welding technologists, quality control workers, designers, robot programmers, and the central control system and the production management was carried out in the prototype Figure 12. Robotic manipulation of the to-be-ultrasonic-robotic-diagnostic- testing objects in digital twin. Figure 13. Robotic manipulation of the tested objects in the robotic ultrasonic diagnostic workspace. Figure 14. Testing of robotic manipulation and robotic welding in prototype of the robotic system for adaptive manipulation and welding of objects. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 7 workplace of the robotic system for adaptive manipulation and welding of objects. The prototype of the robotic system for adaptive manipulation and welding of objects is shown in Figure 14. In the prototype of the robotic system for adaptive manipulation and welding of objects, several welded objects of different sizes and shapes were tested. One of them is shown in Figure 15. Using a digital twin, it was possible to simulate robotic positioning and welding using a digital twin of the workspace. Digital tools as part of the robotic system for adaptive manipulation and welding of objects allow to automatically record data about each tested sample and store them in an SQL database. From the database, measured values of process variables can be displayed directly at the prototype workplace on the HMI system monitor. At the same time, these measured values are available to the welding technologist to optimize future robotic welding procedures as well as to the quality control staff via a web interface in both tabular and graphical forms. Measured values of welding voltage and welding current from the robotic welding of the test sample in a graphical form are shown in Figure 16. The graph is drawn from web application of the prototype. 7. CONCLUSION The research presented in the article shows that digital tools as part of a robotic system for adaptive manipulation and welding of objects can be effectively used in modelling or design verification by simulations through a digital twin prototype of the robotic workplace. The exchange of digital data takes place between the components of the prototype consisting of one welding robot, two handling robots equipped with a quick- change robotic gripper system, 3D scanners, a 2D scanner, an automated process variable measurement system, a SQL database, a FTP server, and a web portal. Implementation of the Figure 15. One type of welded samples. Figure 16. Graph of measured values of welding voltage (blue) ang welding current (violet) for testing sample in Figure 15. ACTA IMEKO | www.imeko.org September 2022 | Volume 11 | Number 3 | 8 digital tools into the prototype enables to adapt the workplace for production of various products of different shapes and sizes. In this prototype, the web portal allows comfortable entering and mining of data characterizing the process of robotic manipulation and robotic welding. ACKNOWLEDGEMENT This article was written thanks to the support under the Operational Program of Integrated Infrastructure for the following projects: "Robotic Workplace for Intelligent Welding of Small Volume Production (IZVAR)", code ITMS2014 +:313012P386, and "Digitization of Robotic Welding Workplace (DIROZ)", code ITMS2014 +:313012S686 (both co-financed by the European Regional Development Fund). REFERENCES [1] M. Lofving, P. Almstrom, C. Jarebrant, B. Wadman, M. Widfeldt, Evaluation of flexible automation for small batch production, ScienceDirect, 2018, pp. 177-184. DOI: 10.1016/j.promfg.2018.06.072 [2] Ch. Liu, P. Jiang, W. Jiang, Web-based digital twin modeling and remote control of cyber-physical production systems, Robotics and Computer Integrated Manufacturing, 2020, pp. 1-16. DOI: 10.1016/j.rcim.2020.101956 [3] F. Tao, H. Zhang, A. Liu, A.Y.C. Nee, Digital Twin in Industry: State-of-the-Art, IEEE Transactions on industrial informatics, vol. 15, no 4, April 2019, pp. 2405-2415. DOI: 10.1109/TII.2018.2873186 [4] R. Ferro, H. Sajjad, R. E. C. Ordonez, Steps for Data Exchange between Real Environment and Virtual Simulation Environment, ICCMS ’21, 25-27 June 2021, Melbourne, VIC, Australia, ISBN 978-1-4503-8979-2/21/06. DOI: 10.1145/3474963.3474988 https://doi.org/10.1016/j.promfg.2018.06.072 https://doi.org/10.1016/j.rcim.2020.101956 https://doi.org/10.1109/TII.2018.2873186 https://doi.org/10.1145/3474963.3474988