DOI: 10.3303/CET24114133 Paper Received: 27 May 2024; Revised: 6 October 2024; Accepted: 5 December 2024 Please cite this article as: Szűrös Z., Ördög R.R., Károlyfi K.A., Szalai D., Bukovics Á., 2024, Building Diagnostics Options for Existing Buildings – Innovative Methods, Chemical Engineering Transactions, 114, 793-798 DOI:10.3303/CET24114133 CHEMICAL ENGINEERING TRANSACTIONS VOL. 114, 2024 A publication of The Italian Association of Chemical Engineering Online at www.cetjournal.it Guest Editors: Petar S. Varbanov, Min Zeng, Yee Van Fan, Xuechao Wang Copyright © 2024, AIDIC Servizi S.r.l. ISBN 979-12-81206-12-0; ISSN 2283-9216 Building Diagnostics Options for Existing Buildings – Innovative Methods Zsolt Szűrös*, Rita Rebeka Ördög, Kitti Ajtayné Károlyfi, Dóra Szalai, Ádám Bukovics Széchenyi István University, Győr, Hungary szuroszs98@gmail.com The increasing number of buildings, due to erosion and extraordinary effects, has increasingly made technological advancements focused on innovative methods possible. This has become more prominent alongside traditional practices such as visual inspection and destructive building diagnostic methods. Thanks to the continuous development of tools, technologies and methodologies, the accurate and detailed digital mapping of buildings (including all their aspects) is available. Among these solutions, the laser scanner point cloud is the most popular due to its high level of detail. The newly emerging and now widespread survey procedures based on laser, photogrammetric remote sensing reflect a real-time, current state of the buildings, which maximizes the accuracy of the Building Information Model (BIM) - Historic Building Information Model (HBIM) completed in the later phases and helps detect structural or surface defects. This study presents the analyzing techniques of point clouds to diagnose buildings, assess their condition, identify errors, and develop sustainability strategies. It also explores areas that require further research and development to enhance the effectiveness of these methods. In the first phase of the paper, the currently used point cloud generation technologies (e.g.: laser scanning, photogrammetry, light detection and ranging (LiDAR)) will be presented with their advantages and disadvantages. The research methodology was conducted using the VOSviewer software to visually analyze bibliometric networks from a dataset of scientific publications, focusing on trends in structural health monitoring and highlighting key areas such as damage detection, computer vision, and AI-based techniques. In the second phase of the study, the possibilities of the analysis of point clouds and image processing-based survey options for structural diagnostic purposes will be explained and presented. The evaluation of the three damage detection methodologies (Geometrical features, Color and intensity information, Combined Method) highlights their complexity, technological requirements, cost, practical applicability, and accuracy. During the analysis, the goal is to map and systematize innovative methods supported by digital tools for diagnostic tasks in existing buildings. 1. Introduction Engineering facilities such as buildings, bridges, various engineering structures, and surface or underground infrastructure are essential for the efficient functioning of a modern society. These facilities are built of complex structural systems designed to take into account the circumstances of the situation in which the facility is located. Engineering structures have an impact on the level of comfort of daily life and on the social and economic development of countries. However, the installations themselves and the structural systems that make them up are designed for a period of about 50 to 100 y. This lifetime can be greatly influenced by the extent and nature of structural damage. Whether external (e.g., environmental factors, human activity, meteorological effects) or internal (e.g., load changes, material fatigue), structural damage has serious economic, social, and environmental consequences. The investigation of structural damage can be divided into two broad categories depending on its nature. There are destructive DT (Destructive Testing) structural health assessment procedures, which include Structural Health Monitoring (SHM) tests (tensile strength test, Rockwell hardness test, Charpy indentation test) and non-destructive NDT (Non-Destructive Testing) structural health assessment procedures (contact ultrasonic; X-ray scanning; IRT thermal imaging). In the former case, mechanical damage is caused to the 793 structure/materials/surfaces from which their physical condition is determined, and in the latter case, the condition assessment is determined from measurements of the structure/material/surface without external damage. The latter category includes Vision-based Structural Health Monitoring (VSHM). For both inspection methods, several subgroups have emerged depending on the technology and methodology (Chuan-Zhi and F Necati, 2020). VSHM is a technique using non-destructive optical imaging techniques to help detect/inspect and assess the physical condition of buildings, engineering structures, facilities, line-side infrastructure, and even aircraft and pipelines. VSHM systems use cameras and other optical or light- and distance-sensing technologies to capture visual condition data instead of sensors and detectors, such as laser scanning, photogrammetric point cloud, and LiDAR (Light Detection and Ranging). Image processing technologies based on distance sensing for detecting damaged areas are often referred to as active methods (e.g., LiDAR, laser scanner), and methods based on image processing alone are referred to as passive methods (Yan et al., 2023). The rationale behind VSHM is to optically analyze surface and behavioral changes associated with structural aging, which is indicative of potential defects, structural damage, and deformation. Vision-based structural health monitoring methods can be evaluated through several workflows depending on the passive or active monitoring procedure, but the general processing procedure and steps of these methods are the same (Mohammad et al., 2019). The advantage of VSHM over SHM is its cost-effectiveness. Although SHM systems have many advantages in terms of detection capability, the deployment, maintenance, and upkeep of these technologies present additional challenges and costs. The aim of the research is to map VSHM systems in light of recent research findings comprehensively. The research will categorize the methodologies identified from each case study and use them for the studies according to their nature. Then, the effectiveness of the results will be evaluated. At the end of the article, the increasingly precise VSHM-based methodologies and research findings are summarised and compared, leading to conclusions on the most relevant areas of vision-based research in terms of applicability. 2. Research methodology The research methodology was based on the VOSwiever software, using publication data extracted from the Web of Science website. VOSviewer is a software for building and visualizing bibliometric networks. These networks include journals, researchers, or individual publications and can be constructed based on citations, bibliographic appendices, co-citations, or co-author links. The data used for the research analysis was provided by a "Tab delimited file" downloaded from the Web of Science interface, which aggregates scientific publications based on predefined search criteria and keywords. Figure 1: Overlay bibliometric network visualization of structural health monitoring. 794 Journals published between 2020 and 2024 were taken into account in the research to ensure that the topic is up-to-date. The original dataset of 500 publications was further filtered by reducing the common keywords detected by the software to 8 common occurrences. The resulting 60 publications already provided a relevant picture of the topics related to the research topic. From the bibliometric network analysis plot (Figure 1), it can be seen that keywords closely related to structural health monitoring, such as damage detection, identification, computer vision, and various damage detection techniques based on artificial intelligence, are included, suggesting further areas of research. It can be concluded that recent research has mainly moved towards methodology, framework, and vision-based studies. 3. Conclusions Damage detection methods 3.1 Damage detection process The processing steps for vision-based impairment assessment methods are the same for passive and active methods (Figure 2): 1. Data collection: The first step is to ensure that the survey data (photos, laser scanner/photogrammetric point cloud) have a sufficient level of detail and accuracy. 2. Preprocessing: The second step is to prepare the collected data for processing. In this step, the images are corrected, and the point clouds are removed. 3. Damage detection methods: In the following, the clean data corrected in the pre-processing are processed. The processing flow in this step can be divided into three main processing methodologies: (1) damage detection based on the geometric method, (2) damage detection based on color and intensity, and (3) damage detection based on the combined method. 4. Damage evaluation: In the fourth step, the data processed using one of the previous three methodologies is evaluated. 5. Identified damage: Based on the evaluation, the injury is identified in the fifth step, where it is categorized according to its nature and extent (Mohammad et al., 2019). In the following, the 3 VSHM methodologies presented in step 3 are described with the help of case studies. These three types of methodologies investigate surface and structural damage using different approaches, technologies, and results. Figure 2: Steps of a Vision-based structural health monitoring 3.2 Damage detection based on geometrical features Of the three different types of inspection listed in step 3 (Damage detection based on geometric features, Color and intensity information, Combined method), the geometric method is the point cloud-based inspection method. Point cloud-based structural damage detection can be divided into three main approaches based on the technology and the nature of the point cloud generation: laser scanning, photogrammetry, and LiDAR. All three methods provide real geometric and often color data of the structure under investigation. Defect detection processes analyze local spatial variations of the point cloud peaks using geometric surface descriptors so that structural and surface damage can be well interpreted digitally (Mohammad et al., 2019). Laser scanner point cloud-based measurements were carried out on a concrete bridge on the A16 Naples- Canosa motorway (E842 European road) in the province of Avellino, Italy (Giordano et al., 2009), focusing on the automatic identification of concrete damage. The procedure was based on the calculation of the mean and Gaussian curvatures of the surfaces and the comparison of their distributions. Damage detection was performed in three steps: (1) subdivision of the point cloud into sub-areas, (2) performing curvature calculations, and (3) 795 identifying damaged areas based on standard deviation. The effectiveness of the procedure was confirmed by the agreement with the real data. The test on the concrete bridge pier and main girder showed that with long- and medium-range instrument data, the test works and has realistic computational costs that do not depend on the light conditions. In another study (Giovanni et al., 2017), a case study was carried out at the San Felice sul Panaro fortress in Italy, where an innovative numerical and three-dimensional finite element (FE) modeling strategy was implemented to investigate damage using point clouds. The point cloud generated by Terrestrial Laser Scanning (TLS) was converted into a 25 x 25 x 25 cm FE mesh model, providing the necessary detail for global structural analysis. Four different material properties were assigned to the model, and linear and nonlinear analyses were performed for the whole fort and the main tower. The aim of the study was to develop a comprehensive numerical analysis based on simplified TLS models. It was shown that an increase in the automation of the mesh generation process for damage detection could be achieved, and the time required is greatly reduced compared to CAD-based modeling procedures. Photogrammetric vision-based detection methods are one of the most widely used inspection methods for monitoring and detecting damage to structures and industrial facilities (Enoc et al., 2018). A photogrammetric point cloud-based inspection was performed on a reinforced concrete pillar (Hongchang et al., 2022). For this purpose, a three-step computational framework was developed to digitally assess the damaged surfaces. (1) First, noise filtering was applied to the point cloud and fixed in the coordinate system. (2) Then, the point cloud was divided into horizontal segments to examine the cross-sections of the pillar. In the VA analysis, the undamaged cross-sectional segments that remained intact were compared to the damaged ones, which allowed the detection of the damaged surfaces. (3) In the third step, the damaged surfaces and the concrete losses were calculated by linear interpolation. The results proved that even photographs taken by a smartphone (iPhone XR) are suitable for diagnostic tests with millimeter accuracy. In a case study (Rosella and Fabio, 2019), a photogrammetric point cloud study was used to quantitatively assess the deterioration on the Palazzo Palmieri in Monopoli. The research consisted of three phases: (1) a photogrammetric survey and 3D model of the building, (2) the detection of cracks and material loss using surface descriptor software, and (3) the development of monitoring procedures to track changes over time. The results show that damage can be detected with millimeter accuracy using photogrammetric point clouds, although misinterpretations may occur in the case of facade decorations. 3.3 Damage detection based on color and intensity information Of the test types listed in step three, color and intensity-based damage detection rely exclusively on color and its change information. These tests are mostly image-based, but since the point cloud also contains image data, they can also be used (Mohammad et al., 2019). A multi-pixel level damage detection method (Shengyuan et al., 2019) based on the Fully Convolutional Network (FCN) has been developed for concrete structure damage detection. For the study, a database of 2750 images of different building structures - structural damage (crack, spalling, efflorescence) was set up and manually labeled for each damage. The FCN architecture was modified, trained, and tested on the images in the database. The test resulted in a pixel accuracy (PA) of 98.61 %. One of the significant advantages of FCN is that it learns the lesion characteristics from a large amount of training data. However, a comprehensive database and time-consuming annotation are required to train a large-scale FCN model. A further study (Alireza and Andrew, 2015) investigated roof sheathing damage in buildings due to wind effects based on automatic detection. The tests were carried out under controlled laboratory conditions to maximize the accuracy of the algorithm settings used. The clustering of intensity and color of the images produced by the ground-based lidar (GBL) technology was based on the spectral data feature method. Of the intensity clustering studies performed, LiDAR produced the most accurate results, with less than 5 % false damage detection, which was only minimally affected by shadows, roof color, and the extent of damage (Alireza and Andrew, 2015). 3.4 Damage detection based on combined method The last of the types of tests listed in step three combines the methods described above to refine the tests (Mohammad et al., 2019). In one study (Valença et al., 2017), surface damage of reinforced concrete girder bridges was investigated using MCrack-TLS methodology, which includes information based on the color intensity of images and geometric data from ground-based laser scanning TLS technology. Based on the geometric data collected by TLS, the captured images are orthorectified, which eliminates one of the main drawbacks of image processing, the detection of cracks on large surfaces. The detection of surface continuity disturbances and damage was based on the reference plane of the point cloud and the distance between the vertices. The vertices that were defined as damage were then defined as those that were larger than the standard deviation of the two values. The main advantage of MCrack-TLS compared to conventional methods 796 is the automatic processing of information, resulting in higher speed, efficiency, reliability and quantity, as well as better data quality. 4. Results and Discussion From the case studies reviewed, it was found that vision-based damage detection methods are highly applicable to real-world practical projects. Point cloud-based surveys are preferred for larger and more complex industrial installations due to the precise geometric detail that maximizes the accuracy of the detection algorithms. The test methods described in the case studies are typically able to detect the level of visible surface damage with an accuracy of 5%, since the point cloud-based damage detection provides centimeter-accurate information about the examined area, even in total darkness. These surveys are perfectly suited for damage detection due to geometric changes, such as the determination and calculation of material losses, geometric distortions, deformations, and monitoring of various damaging deformations. 3D survey data has the advantage of integrating the detected damage into a BIM (Building Information Modeling) work environment, which allows for long-term monitoring studies. Pixel and color information-based measurements can be effectively used to detect damage due to changes in surface patterns, textures, and color information, such as cracks, soaking, thermal bridges, lime efflorescence, cracked plaster, and rusting on small-scale surfaces and components. The essence of the process is the detection of visible changes compared to the ideal state, mainly on homogeneous and structured surfaces, from which the damaged areas can be measured in percentage terms. From the point of view of damage detection, the most effective measurements were those that used a hybrid approach combining the advantages of both methods to accurately detect both geometric deformations of complex spatial masses and their damaged sub-areas (Table 1). As shown in the Research methodology, the latest research in the field of structural health monitoring applies artificial intelligence, including deep learning and machine learning, to the structural analysis of software damage detections, which will be able to maximize the accuracy of building diagnostic tests and the efficiency of the process. However, this requires a significant database and a lot of live work due to the annotation of images. Table 1: Evaluation of damage detection methods (advantageous: ++; normal: +; disadvantageous: -) Method type Complexity Technological demand Costliness Accuracy Applicability (1) Geometrical features + - + + ++ (2) Color and intensity information ++ ++ ++ + + (3) Combined Method + - - ++ ++ 5. Conclusions Aging engineering and infrastructural facilities present gradual challenges to operators, to which solutions are provided by Nondestructive Testing and Evaluation (NDT&E) and VSHM tests. These technologies provide a cost-effective and user-friendly framework for the practical application of non-contact condition assessment. Using the bibliometric neural network visualization model, it is possible to clearly select the key areas and sectors where and to which the point cloud data can be used in vision-based damage detection procedures. The effectiveness and practical applicability of the VSHM procedures and technologies are confirmed by the studies and methodologies tested in real-world conditions, which are presented through case studies. A hybrid solution using a combination of the investigated methods has the potential to improve the level of detection by exploiting the advantages of the technologies. The measurement results and their evaluation can be further optimized by applying artificial intelligence (AI)-based algorithms. Deep learning and machine learning, closely related to AI, can filter out inaccuracies due to distractions and help in structure recognition, contributing to further research and advancement in the field of building diagnostics. References Gao Y., Li H., Fu W., Chai C., Su T., 2023, Damage volumetric assessment and digital twin synchronization based on LiDAR point clouds, Automation in Construction, 157, 105168. Dong C.Z., Catbas F.N., 2020, A review of computer vision–based structural health monitoring at local and global levels. Structural Health Monitoring, 2, 692 – 743. Mohammadi M.E., Wood R.L., Wittich C.E., 2019, Non-Temporal Point Cloud Analysis for Surface Damage in Civil Structures. International Journal of Geo-Information, 8(12), 527. 797 Wu C., Yuan Y., Tang Y., Tian B., 2022, Application of Terrestrial Laser Scanning (TLS) in the Architecture Engineering and Construction (AEC) Industry. Sensors, 22(1), 265. Teza G., Galgaro A., Moro F., 2009, Contactless recognition of concrete surface damage from laser scanning and curvature computation. NDT & E International, 42(4), 240-249. Castellazzi G., D'Altri A.M., de Miranda S., Ubertini F., 2017, An innovative numerical modeling strategy for the structural analysis of historical monumental buildings. Engineering Structures, 132, 229-248. Sabato A., Dabetwar S., Kulkarni N.N., Fortino G., 2023, Noncontact Sensing Techniques for AI-Aided Structural Health Monitoring: A Systematic Review. IEEE Sensors Journal, 23, 4672–4684, DOI: 10.1109/JSEN.2023.3240092. Sanz-Ablanedo E., Chandler J.H., Rodríguez-Pérez J.R., Ordóñez C., 2018, Accuracy of Unmanned Aerial Vehicle (UAV) and SfM Photogrammetry Survey as a Function of the Number and Location of Ground Control Points Used. Remote Sensing, 10(10), 1606. Zhang H., Zou Y., Castillo E. del R., Yang X., 2022, Detection of RC Spalling Damage and Quantification of Its Key Properties from 3D Point Cloud. KSCE Journal of Civil Engineering, 26, 2023-2035. Galantucci R.A., Fatiguso F., 2019, Advanced damage detection techniques in historical buildings using digital photogrammetry and 3D surface anlysis. Journal of Cultural Heritage, 36, 51-62. Li S., Zhao X., Zhou G., 2019, Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Industrial Application, 34, 616–634. Kashani A.G., Graettinger A.J., 2015, Cluster-Based Roof Covering Damage Detection in Ground-Based Lidar Data. Automation in Construction, 58, 19-27. Valença J., Puente I., Júlio E., González-Jorge H., Arias-Sánchez P., 2017, Assessment of cracks on concrete bridges using image processing supported by laser scanning survey. Construction and Building Materials, 146, 668-678. 798