Microsoft Word - numero_58_art_03_3050 A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 33 Wavelet-based multiresolution analysis coupled with deep learning to efficiently monitor cracks in concrete Ahcene Arbaoui University of Bouira, Department of Civil Engineering, Bouira, Algeria a.arbaoui@univ-bouira.dz Abdeldjalil Ouahabi UMR 1253, iBrain, Université de Tours, INSERM, Tours, France University of Bouira, Department of Computer Science, LIMPAF, Bouira, Algeria ouahabi@univ-tours.fr Sébastien Jacques University of Tours, GREMAN UMR 7347, CNRS, INSA Centre Val-de-Loire, Tours, France sebastien.jacques@univ-tours.fr Madina Hamiane College of Engineering, Royal University for Women, Bahrain mhamiane@ruw.edu.bh ABSTRACT. This paper proposes an efficient methodology to monitor the formation of cracks in concrete after non-destructive ultrasonic testing of a structure. The objective is to be able to automatically detect the initiation of cracks early enough, i.e. well before they are visible on the concrete surface, in order to implement adequate maintenance actions on civil engineering structures. The key element of this original approach is the wavelet-based multiresolution analysis of the ultrasonic signal received from a sample or a specimen of the studied material subjected to several types of solicitation. This analysis is finally coupled to an automatic identification scheme of the types of cracks based on artificial neural networks (ANNs), and in particular deep learning by convolutional neural networks (CNNs); a technology today at the cutting edge of machine learning, in particular for all applications of pattern recognition. Wavelet-based multiresolution analysis does not add any value in detecting fractures in concrete visible by optical inspection. However, the results of its implementation coupled with different CNN architectures show cracks in concrete can be identified at an early stage with a very high accuracy, i.e. around 98%, and a loss function of less than 0.1, regardless of the implemented learning architecture. Citation: Arbaoui, A., Ouahabi, A., Jacques, S. and Hamiane, M., Wavelet-based multiresolution analysis coupled with deep learning to efficiently monitor cracks in concrete, Frattura ed Integrità Strutturale, 58 (2021) 33-47. Received: 26.03.2021 Accepted: 01.07.2021 Published: 01.10.2021 Copyright: © 2021 This is an open access article under the terms of the CC-BY 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. https://youtu.be/5ddJPhWe_No A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 34 KEYWORDS. Crack monitoring; Concrete; Non-destructive ultrasonic testing; Wavelet-based multiresolution analysis; Deep learning. INTRODUCTION ince its creation, concrete, the flagship material of civil engineering structures, has been the basis of construction techniques, whether industrial (factories, warehouses, ...) or hydraulic (dams, dikes, ...), but also of infrastructures such as transportation (bridges, tunnels, ...) or urban infrastructure (aqueducts, ...). This success is due to several factors: concrete is an economical material, easy to work with, resistant to compressive stress, durable, sound and heat insulating, and it contributes to architecture through the shapes, textures and colors it provides [1]. Coupled with a reinforcement usually made of steel, reinforced concrete can compensate for the low tensile strength of concrete. The service life of reinforced concrete structures is conditioned by the response to chemical (e.g. carbonation, corrosion, etc.), physical (e.g. freeze-thaw cycle) and mechanical (e.g. overloading) aggressions of the environment, as well as by the capacity of the constituent materials to protect themselves against these aggressions. In this work, we are interested in the corrosion of reinforcement which is one of the main causes of degradation of reinforced concrete structures [2]. This corrosion induces a modification of the steel-concrete bond, a reduction in section of the steel bars, a decrease of the ductility of the steel as well as a peripheral damage of the concrete due to the pressure of the corrosion products. More precisely, Bhaskar et al. showed that when the porous zone at the interface between steel and concrete (a zone whose volume depends on the surface of the reinforcement, the water/cement mass ratio and the degree of hydration) is completely filled by the corrosion products, pressures are exerted on the concrete cover and can generate cracks [3]. As shown in Figure 1, monitoring the propagation of these cracks is therefore essential to stop them from reaching critical sizes that could lead to the reduction of the bearing capacity of the concrete or reinforced concrete structure or even to its failure [4–8]. This research topic is still of great interest and recent studies have shown that it is possible to limit interfacial micro-cracks in concrete and its composites subjected to dynamic loads, for instance by adding fly ash and/or silica fume at a rate of a few thenths of the weight of the cement [7, 8]. Figure 1: Examples of cracks in reinforced concrete structures [provided by the authors]. S A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 35 Visual inspection of civil engineering structures is now complemented by high-performance, high-definition scanner or photogrammetry surveys, and artificial intelligence (AI), in particular deep neural networks, can detect defects, classify them and propose a diagnosis. The Internet of Things (IoT) and new generations of sensors (e.g. fissurometers, inclinometers) make it possible to instrument infrastructure and continuously monitor a number of structural health indicators from 24/7 control centers [6]. There are many methods to evaluate the detection of anomalies in materials or components of a civil engineering structure. Non-destructive testing (NDT), which is a long-standing, common and mandatory practice in many industries such as aeronautics or aerospace, is an important category [10]. NDT methods consist in causing a disturbance in the material to be studied, here of an ultrasonic nature, and recording its response. This response to the ultrasonic excitation is a function of the state of the material or the component of the structure to be controlled. These techniques are therefore important tools to help detect cracks, characterize the degree of corrosion of reinforcement in reinforced concrete, determine the thickness of concrete slabs, etc. [11, 12]. However, a critical point of this type of method is the extraction of relevant information on the state of the material from its response. Multiresolution analysis (MRA) is the original method implemented in this work to extract this key information by decomposing the signals at different levels of resolution. In particular, we will use wavelets, i.e. an extension of Fourier analysis, as an analytical tool to mathematically describe the increment of information required to move from a coarser approximation of the material response to a higher resolution approximation. Wavelet-based multiresolution analysis, which has received significant attention in recent years in various fields, is therefore a powerful tool for efficiently representing signals and images at multiple levels of detail [13, 14]. The last key point of the work described in this paper is to build a classifier to detect cracks from the images obtained at the spatial scale. In this regard, an automatic crack type identification scheme, based on artificial neural networks (ANNs), is proposed. Crack detection techniques based on deep ANNs, i.e. deep learning, are currently under active research due to their renowned outstanding performance [15]. In particular, some authors have recently proposed improved convolutional neural networks (CNNs) that can extract crack patches in an image with 99% accuracy [16]. The structure of this article is as follows. First, we will point out the fundamental concepts, as well as the experimental procedures, associated with each of the three key points of the method proposed here: non-destructive ultrasonic testing to obtain an ultrasonic signal identifying the defect; multiresolution wavelet-based analysis to preserve the important elements of the signal, i.e. the cracks, at high resolution and produce a scalogram localizing the defect; and finally classification by CNNs. The results obtained will then be analyzed and discussed. Finally, we will emphasize the originality of this work, namely the multiresolution analysis based on wavelets, as input to the deep neural network, which allows us to obtain a high level of classification accuracy, independently of the chosen CNN architecture. RESEARCH SIGNIFICANCE AND PROPOSED METHODOLOGY Research signification his work solves the important problem of detecting the onset of cracks inside concrete structures. These cracks are optically invisible from the outside and may propagate unexpectedly until structure failure. In some sensitive infrastructures, such as nuclear power plants, dams or bridges, a concrete failure can lead to very serious disasters. Although this type of disaster remains unusual, each occurrence can generate serious human, environmental and technical consequences. This is why it is important to have a protocol for detecting and monitoring cracks in their early stages in order to secure structures of vital interest. What is interesting is to know the cost of our proposed protocol to practically evaluate its implementation in the field. The cost of our investigations is low since all that is required is an on-site portable ultrasonic device and an ordinary processor, either a DSP card or a laptop computer since we are implementing an architecture that has already learned to detect and track possible internal cracks or the beginnings of cracks. To quantify the hardware implementation of our approach, we recall the instruments used: The instrument used is the Pundit PL-200. It allows first class ultrasonic pulse velocity tests to examine the quality of concrete: to estimate the compressive strength of concrete or to measure the surface velocity and the depth of cracks. Our software supports settings directly accessible in real time from the measurement screen. The developed software is implemented on an electronic board with DSP or directly implemented in a laptop. The global cost of all this instrumentation is about $5,000.00. T A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 36 Methodology The methodology implemented in this paper is composed of three main steps (see Figure 2) that we will detail in the rest of this section. The objective of the proposed methodology is the detection and monitoring of internal cracks in concrete structures. Such cracks will be detected by ultrasonic NDT and analyzed by the wavelet transform providing a spatially scaled image allowing to localize the crack in space and at each resolution. The resulting multi-resolution image is then subjected to a deep learning-based crack/non-crack classification process (AlexNet, VGG16). This methodology comprises three steps: The first step consists of performing a non-destructive ultrasonic test (NDT) on aging concrete samples of different compositions. The objective is to collect the ultrasonic signal received. In the second step, a wavelet-based multiresolution analysis is conducted on the received ultrasound signal. This is the key step of the work presented in this article since it will allow to highlight the initiation of cracks within the material. From the multi-resolution analysis, a B-scan mapping will then be obtained. Finally, the obtained image will be the input of a deep learning algorithm based on Convolutional Neural Networks (CNNs). Two well-known architectures in the literature, namely AlexNet and VGG16, will be tested. AlexNet won the ImageNet competition in 2012, and VGG16 won the same competition in 2014. These are the two networks that were used in our experiments due to the fact that they are behind the explosive emergence of Deep Learning. They will be the basis for evaluating the performance of our approach for crack detection in concrete structures. Other neural networks exist, as powerful as AlexNet and VGG16 and maybe more, which are used in pattern recognition. However, the goal of this work is not to find the network that will give the best accuracy in detecting an internal crack in concrete from optical images. Indeed, the aim is to demonstrate that with wavelet-based multiresolution analysis, the detection of a crack in concrete at an early stage will be very accurate independently of the type of deep learning architecture used. Figure 2: Methodology for effective monitoring of cracks in concrete after non-destructive ultrasonic testing. A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 37 EXPERIMENTAL PROCEDURE Preparation of concrete specimens n this section, the four main steps in the preparation of concrete specimens will be detailed, i.e. fabrication, casting, curing and end grinding. As shown in Figure 3 a, the following raw materials were used to manufacture the concrete specimens:  cement (Sour El-Ghozlane cement plant in Algeria) dosed at 350 kg/m3;  sand (Bou-Sâada sand in Algeria) characterized by a grain size between 200 µm and 500 µm;  three types of gravel i.e., grain size 3/8 mm, grain size 8/15 mm and grain size 15/25 mm. Five batches of standardized size specimens, corresponding to five different concrete mixes (see Table 1), were made. Each batch is composed of one cubic specimen of 150 mm side and six cylindrical specimens of 160 mm diameter and 320 mm height. Each of the thirty-five specimens dedicated to this study was cast in a galvanized steel mold on a vibrating table (see Figure 3 b. The final mass of a concrete specimen is 15 kg. As shown in Figure 3 c, each specimen was stored at 20 °C and 98% humidity in a curing chamber for 28 days to simulate the fabrication of concrete columns. Finally, as shown in Figure 3 d, a “Deluxe Hi-Kenma TSURU-TSURU” type end grinding machine from the manufacturer MARUI & CO., LTD. was then used so that each concrete specimen had a perfect surface, i.e., low roughness after machining. Figure 3: Main steps in the manufacture of concrete specimens: a) Raw materials; b) Casting; c) Curing; d) End grinding [provided by the authors]. Batch No. Characteristic strength [MPa] Water/cement ratio Cement [kg.m-3] Sand [kg.m-3] Gravel [kg.m-3] 1 10 1.20 150 1,016 1.11 2 15 0.80 225 954 1.11 3 20 0.65 277 911 1.11 4 25 0.62 290 900 1.11 5 30 0.55 327 870 1.11 Table 1: Proportion of constituents of the five batches of concrete mixtures and associated characteristic compressive strengths. I A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 38 Ultrasonic tests on concrete specimens There are many standardized processes related to NDT that can be classified into two categories: the first allowing to evaluate the strength and its variation in time; the second to evaluate characteristics other than strength (e.g. dimensions of structural elements, corrosion, dampness etc.) [10], [17–20]. In the first category, we find sclerometric methods [21] (whether static or dynamic), acoustic methods [22, 23] (e.g. ultrasound) or pull-off methods [24] which are semi-destructive. In the second category, the techniques are much more numerous: we can mention, among others, acoustic methods [25, 26] (e.g. acoustic emission, echo, ultrasonic, impact echo), electromagnetic methods [27] (e.g. continuous wave eddy current testing), physical methods [28] (e.g. methods based on measuring thermal properties, electric methods such as linear polarization resistance) or radiological methods [29] (e.g. techniques based on X-rays). Unfortunately, civil engineering cannot benefit from all the technical advances of NDT in the mechanical industries because the nature of the materials used and the concerns differ. Unlike metals, concrete is a composite material that originally contains a large number of defects in the form of small cavities, pores and gaps. It is also a material whose mechanical properties are not rigorously reproducible, even under the best conditions. Moreover, these properties degrade more or less rapidly over time due to increased service loads, climatic conditions, alkali- aggregate reaction, etc. Therefore, only acoustic techniques, infrared thermography, penetrant testing and corrosion rate measurement (e.g. linear polarization) methods are generally used in civil and mechanical engineering. Many authors have designated acoustic NDT methods, used individually (e.g. ultrasonic tomography or impact-echo, both used individually) or in combination (e.g. impulse response combined with impact-echo), as particularly well suited for testing structures or building materials, especially concrete. In this article, we have implemented an acoustic method based on the measurement of the ultrasonic pulse velocity. This type of method is only suitable for the study of concrete consistency, discontinuities, cracks and crack depth but is not reliable for strength determination, except for the determination of Poisson's ratio and Young's modulus with reasonable accuracy [30]. By correlating ultrasonic pulse velocity and the concrete compressive strength, this latter can also be determined. The equipment used to perform the tests is the Pundit® PL-200 from Proceq (see Figure 4). Figure 4: Experimental procedure for ultrasonic pulse velocity tests. Two P-wave ultrasonic pulse velocity transducers with a frequency of 54 kHz are used. The ultrasonic pulse velocities are between 100 Vpp and 400 Vpp. The pulse echo range is from 0.1 µs to 1,200 µs. A 7 inch 800 × 480 pixel touch screen with very high resolution is available with the equipment to analyze the measured waveforms. To determine the compressive strength, a 2,000-3,000 kN one-piece compression testing machine from 3R is used (see Figure 5). According to Table 1, with the five batches of concrete mixtures made, the estimated values of compressive strength are 10 MPa, 15 MPa, 20 MPa, 30 MPa and 35 MPa. The cylindrical specimens were first tested ultrasonically, using 54 kHz longitudinal and transverse wave transducers. This method consists in measuring the transit time of an ultrasonic pulse passing through the concrete sample under test. The A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 39 higher the speed, the better the quality of the concrete in terms of density, uniformity, homogeneity, etc. The axial compression tests were performed immediately after the ultrasonic tests. Figure 5: Experimental procedure for compressive tests. Wavelet-based multiresolution analysis Wavelets are an interesting analytical tool to describe mathematically the increase in information required to go from a coarse approximation to a higher resolution approximation. Through a multiresolution analysis (MRA), a signal can be decomposed and reconstructed as a series of approximations of decreasing scale, completed by a series of details [13]. To illustrate this concept, let us consider an image built from a succession of approximations; the details enrich this image. Thus, thanks to the MRA based on wavelets, the coarse vision becomes finer and more precise. Engineers, practitioners and researchers are confronted daily with increasingly difficult technological problems at multiple scales of analysis, in terms of classification, segmentation, detection of contours or parameters of interest, noise reduction or elimination, compression for transmission or storage, synthesis or reconstruction, etc. This concerns many fields such as astrophysics [31], finance [32], fluid mechanics [33], thermodynamics [34], medicine and biology [35–37], multimedia [38], telecommunications [39, 40], signal and image processing, and of course, the monitoring of cracks and detection of fractures in materials [41–46]. MRA based on wavelets can thus become an essential tool for solving the difficulties encountered in the above-mentioned fields. This tool, sometimes described as miraculous, produces an immediate, easily interpretable and exploitable result. However, for specific applications requiring the extraction of targeted information, it is clear that advanced methods will have to be developed and “merged” in order to effectively utilize existing techniques or to optimize the analyses (for example in compression) by taking into account edges or contours, using 2nd and 3rd generation wavelets such as peaks, curves [47], contours [48], bands [49], etc. Indeed, these anisotropic wavelets are automatically oriented and extended by unifying the geometry of a given edge or contour. This conceptualization of MRA is comparable to a camera that moves closer to a subject or uses a zoom lens to distinguish details, and further away to capture larger features– the famous concept of the mathematical microscope. The principle of wavelet-based MRA is illustrated in Figure 6. Three levels of resolution are considered here. At the first level of resolution, the signal S is decomposed into an approximation 1A and a detail 1D . At the second level of resolution, the 1A approximation is decomposed into an 2A approximation and a 2D detail. Finally, at the third level of resolution, the 1A approximation is in turn decomposed into an 3A approximation and a 3D detail. Thus, the signal S can be expressed as shown in (1).    3 3 2 1S A D D D (1) Let   t denote a reference pattern called the mother wavelet. It is generally requested that   t has jointly highly concentrated time and frequency supports.   t satisfies (2), when n controls the number of oscillations of   t . This relation means that   t is orthogonal to polynomial components of degree less than n . A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 40 Figure 6: Principle of wavelet-based multiresolution analysis with three levels of resolution.        0,    0pt t dt p n (2) The wavelet transform  ,XW u s of a signal X at time u and scale s is defined by (3), where  * denotes the complex conjugate of  .           *,X t u W u s X t dt s (3) Looking closely at equations (2) and (3), it is clear that  ,XW u s will be insensitive to the most regular behaviors of the signal assimilated to a polynomial of degree less than n (the number of vanishing moments of  ). Conversely,  ,XW u s takes into account the irregular behavior of polynomial tendencies. This important property plays a central role in the detection of signal singularities, especially in the detection and tracking of cracks. The discrete wavelet transform (DWT) is given by (4).            , 2 , 2 ,    ,j jX Xd j k W u k s j k (4) Clearly, to reduce or eliminate redundancy, the family      2, ,j k j k must be an orthonormal basis of   2 , where   2 denotes the vector space of one-dimensional measurable, square-integrable functions. This property of the wavelet makes it possible to obtain a fast wavelet transform. The fast wavelet transformation is calculated by a cascade of low-pass filtering by h and high-pass filtering by g followed by a downsampling (or decimation) by a factor of 2 (see Figure 7). In Figure 7, ja (or  ,Xa j k , where k is time) and jd (or  ,Xd j k , where k is time) are referred to as approximation coefficients and wavelet coefficients (or details) of the signal at level j , respectively. Moreover, the symbol represents the decimation by a factor of 2, i.e., keeping every other sample. The impulse response of the mirror low-pass filter is     h k h k and that of the mirror high-pass filter is     g k g k . These two impulse responses are linked by        1 1kg k h k whose coefficients are obtained directly from the chosen wavelet  [13]. Figure 8 shows an example of signal decomposition over three levels of resolution using MATLAB. Note that the original signal has 1,000 samples while the detail (and approximation) signals have been decimated by a factor of 2 at each resolution A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 41 level. Thus, after 3 levels of resolution, from a signal of 1,000 samples, we arrive at the 3A approximation and the 3D detail, which each have only 125 samples. Figure 7: Principle of fast wavelet transform or multiresolution analysis. In this study, the investigative ultrasonic signal scalogram will be used to determine and analyze cracks in concrete. The scalogram of the signal  x t can be defined using (5).          2, ,       ,X XS j k d j k j k (5) Figure 9 shows an example of scalogram of a signal representing three cracks in a concrete specimen, one of which (the central crack) is in an advanced state that could lead to an imminent rupture. Figure 8: Example of decomposition of a signal on three levels of resolution using MATLAB. Detecting cracks in concrete using deep neural networks In recent years, artificial intelligence has become a necessity because of its groundbreaking innovations in many areas, including pattern recognition in construction and structural engineering [50]. Deep learning methods, which use consecutive hidden layers of information processing organized in a hierarchical manner, have become essential for representation, learning and classification. Considered today in the Top 10 of the most efficient and flexible deep learning techniques, A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 42 convolutional neural networks (CNNs) are particularly well suited for tasks such as image recognition, image analysis, image segmentation, video analysis or natural language processing [51, 52]. However, this type of machine learning requires the use of sufficiently large input database for training and testing to ensure the highest possible accuracy of the recognition process [53]. A CNN architecture is typically characterized by the presence of multiple convolutional blocks–each consisting of a convolution layer, an activation function and a pooling layer– and a fully connected layer [54]. A convolutional layer, which is a key element of the method, performs a convolution operation on the output of the previous layers using a set of filters, also called kernels, to extract the features that are important for classification, i.e. in this case the “crack” and “non-crack” classes. Figure 9: Example of scalogram of a signal representing three cracks in a concrete specimen. A CNN architecture is typically characterized by the presence of multiple convolutional blocks–each consisting of a convolution layer, an activation function and a pooling layer– and a fully connected layer [54]. A convolutional layer, which is a key element of the method, performs a convolution operation on the output of the previous layers using a set of filters, also called kernels, to extract the features that are important for classification, i.e. in this case the “crack” and “non-crack” classes. There are many CNN architectures, including AlexNet, VGG16, Inception and ResNet, and their performances are regularly compared by many authors [55, 56]. In this article, two different pre-trained CNN models, i.e. AlexNet and VGG16, are experimented. The objective is to demonstrate that the wavelet-based MRA is the key component of the proposed approach which guarantees a very high level of accuracy in the classification, independently of the type of CNN architecture used. Since its introduction in 2012, in the framework of the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012), the AlexNet architecture has already become a very popular CNN architecture and has obtained good results in many applications such as computer vision [51], [57]. AlexNet is not a complex architecture when compared to other major CNN architectures, such as ResNet, that have emerged in recent years [58]. It is also easy to implement with TensorFlow and Keras. As shown in Figure 2, AlexNet consists of five convolutional layers that use kernels to scan the input image by performing convolution operations. The first two convolution layers use a (11 × 11) and (5 × 5) size filter, respectively; the last three layers each use a (3 × 3) kernel. Some of these convolutional layers (i.e., the first two layers and the last layer) are followed by max-pooling (i.e., a subsampling operation usually applied after a convolutional layer, where the maximum values are taken). At this stage, the model is composed of more than 1.7 million parameters. Each convolution layer uses the rectifier linear unit (ReLU) activation function. Unlike the sigmoid activation function, which is frequently used for a binary classification network, ReLU increases the non-linear properties of the decision function and the global network without affecting the receiver fields of the convolution layers. At the output of the convolutional layers, a flattening step is necessary to create a single vector containing the main characteristics of the crack to be identified. Initially intended to classify 1,000 categories, we have modified the AlexNet architecture so it handles only two possible classes, i.e. images with and without cracks. This architecture ends with three fully connected layers (the first two layers are composed of 4,096 A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 43 outputs, and the last layer has only two) with a Softmax classifier, which is reduced in our work to a simple logistic regression, composed of the two possible labels. The three fully connected layers alone account for more than 18.8 million parameters. Therefore, the network, having more than 20 million parameters, can then be trained. The Adam optimizer was used. However, in case of dropout, a neuron is removed from the network with a probability of 0.5. Even if dropout increases the number of iterations by 2, this step is essential to prevent oversizing of AlexNet. VGG16 (the number 16 meaning that the architecture is composed of 16 layers) is a convolutional neural network model proposed by Simonyan and Zisserman [59] to achieve 92.7% accuracy in the famous ImageNet top-5 test, which is a dataset of over 15 million labeled high-resolution images belonging to roughly 22,000 categories. This architecture improves AlexNet by replacing the large kernel filters (11 and 5 in the first and second convolutional layers, respectively) with multiple 3 × 3 kernel filters, one after another. As shown in Figure 2, the image to be classified goes through a stack of convolutional layers, where filters of size 3 × 3. The spatial padding of the input to the convolution layers is such that the spatial resolution is preserved after convolution. The spatial pooling is performed by five max-pooling layers, which follow some of the convolution layers (not all convolution layers are followed by max-pooling). As for the AlexNet architecture, the ReLU activation function is used in the convolution steps. Three fully connected layers follow the convolutional layer stack. The last layer is a softmax layer used to classify each pixel into “crack” or “non-crack” classes. Although the VGG16 architecture is very large and requires nearly eight times more parameters to be trained compared to the AlexNet architecture, it is easy to implement in current open-source software libraries for artificial neural networks. To conduct this study, we used two computers connected in parallel; each of the two being equipped with a 9th generation Intel Core i7 Hexa Core microprocessor. Each computer is equipped with a high-end NVIDIA GeForce RTX 2080 graphics processing unit (GPU) with the following main memory features: GDDR6 type; 8 gigabyte (Gb) capacity; 14 Gb/s speed; 448 Gb/s bandwidth; and a speed of 60 TOPS (tera operations per second) to process the very large number of operations (up to a few billion for each image) required to compute neural networks. As for software tools, the open source machine learning tool TensorFlow, developed by the Google Brain team, was used. This is now an essential tool for machine learning applications, such as neural networks [60]. The implementation of the convolutional neural network algorithms was done with the Keras library, using the Python programming language. Keras, which is used here as an interface for TensorFlow, was chosen for the ease of its implementation of many functions and procedures, its modularity, and its extension capabilities. MAIN RESULTS AND DISCUSSION irst, we tested the methodology on available image datasets of visually or optically observable cracks on the surface of concrete samples. For this purpose, we used a public database containing 4,800 manually labeled images of cracked and non-cracked concrete bridge decks [61]. 80% of these images were allocated to the training phase and 20% for validation. It is noted that regardless of the deep learning architectures implemented, the wavelet-based MRA does not add anything at this stage as the accuracy levels obtained are those found in the literature. We then sought to demonstrate the relevance of wavelet-based multiresolution analysis to identify the initiation of cracks in concrete, i.e. well before the fracture is visible on the surface of the material. For this purpose, a private database of B- scan mappings obtained by wavelet-based MRA was constructed from the 35 concrete specimens we fabricated and aged by the compression tests. For each concrete specimen, this database contains 40 images without cracks, and 100 images representing several stages of aging, i.e. from the initiation of cracks in the core of the material, then to their propagation, to the fracture of the specimen itself. In total, 4,900 images are available, each with dimensions of 120 pixels × 120 pixels × 3 color channels. For each of the two classes, i.e. “crack” and “non-crack”, 80% of the images are assigned to the training phase and 20% to the validation. Before training and validation, the images are normalized by subtracting their mean in order to have centered data. This ensures similar image characteristics to avoid uncontrollable gradients in the loss function with respect to the neural network weights during backpropagation. Figure 10 gives an illustrative example of the impact of wavelet-based multiresolution analysis combined with a simple deep learning architecture to automatically detect cracks in concrete long before they are visible by optical inspection. We have of course tested both architectures in Figure 2 (i.e., AlexNet and VGG16), as well as a more advanced architecture such as ResNet-50 (i.e., composed of 50 layers with over 23 million trainable parameters). With a CNN architecture composed of about 20 million of parameters, and the equipment used, the training phase lasted about 25 hours, or about 30 minutes per epoch (i.e. the number of cycles through the full training dataset). Similar results can be obtained regardless of the deep learning architecture implemented. F A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 44 In Figure 10, the graph on the left shows the smoothed accuracy of the recognition of a crack as a function of the epochs. The graph on the right shows the evolution of the associated cost function. The cross-entropy loss function, also known as the logarithmic loss function, is one of the most commonly used cost functions when adjusting model weights during training. We have used the binary cross-entropy loss function is implemented. It consists of comparing each predicted class probability with the desired 0 or 1 output, identifying “crack” or “non-crack” respectively, for the actual class. A score is then computed, penalizing the probability according to the distance between it and the expected real value. In the case that we have implemented, the penalty function is logarithmic, which gives a high score for large differences close to 1 and a low score for small differences tending towards 0. For both training and validation, the results in Figure 10 show that fewer than 25 epochs are required for the smoothed accuracy and loss function to converge. More precisely, the smoothed accuracy reaches a maximum of 97% during the training phase and 98% during the validation phase, thus confirming the great importance of the wavelet-based multiresolution analysis. The loss function, on the other hand, reaches a minimum lower than 0.1 during both training and validation, which means that the model is adequately fine-tuned. Figure 10: Wavelet-based multiresolution analysis coupled with a simple deep learning architecture for automatic detection of crack formation in concrete: examples of smoothed accuracy and loss function (binary cross-entropy model). CONCLUSIONS his paper reports on an original methodology that was implemented to efficiently detect crack initiation using non- destructive ultrasonic testing of elements of a concrete civil engineering structure. Compared to existing related studies in the literature, our main contributions are as follows:  Proposal of a detection method at an early stage, i.e. well before the concrete fracture is visible on the surface, in order to implement appropriate maintenance actions and thus avoid the failure of the structure.  A key element of this method is the wavelet-based multi-resolution analysis (MRA) of the ultrasonic signal received from a sample or a concrete specimen subjected to several types of solicitation. The received ultrasonic signal is analyzed at each resolution (or scale) by wavelet transformation.  The resulting image is squared to serve as input to an automatic crack type identification system based on deep learning by convolutional neural networks (CNNs). Two architectures, chosen both for their ease of implementation in open-source platforms and libraries dedicated to machine learning and to limit the computational load, were tested. The purpose was not to optimize CNN architectures. If this were the case, then we would have chosen modular structures (e.g. ResNext, Xception, Channel Boosted CNN, etc.) based on auxiliary learners that utilize either spatial or feature map information or input channels to improve classification performance. The objective was to rather show that with a multiresolution analysis based on wavelets, it is possible to detect crack initiations in concrete and that the accuracy of this detection is independent of the chosen CNN architecture. After aging concrete specimens in compression tests, we built a database containing nearly 5,000 B-scan mappings from wavelet-based MRA of specimens with and without crack initiation and propagation. Regardless of the two architectures implemented, the results show that the accuracy is greater than 98%. The loss function reaches values less than 0.1, which T A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 45 means that both models are finely tuned. All these results prove the relevance and efficiency of the approach described in this paper. It would be interesting if the non-destructive methodology proposed in this paper could be implemented on all or part of civil engineering structures, such as suspension bridges, reinforced concrete bridges with central cantilever spans or masonry railway viaducts, that require permanent remote monitoring in order to prevent the occurrence of failures that would jeopardize the safety and performance of the structure itself. Remote monitoring should not in any case replace visual or optical surveillance of structures, which remains the basis of monitoring. However, deep learning algorithms are of undisputable relevance for remote monitoring, especially when many images or videos showcasing structures’ state of health are captured, because any cracks can in this case be detected very quickly and automatically. REFERENCES [1] Mani, M., Bouali, M. F., Kriker, A. and Hima A. (2021). Experimental characterization of a new sustainable sand concrete in an aggressive environment, Frattura ed Integrità Strutturale, 55, pp. 231–251. DOI: 10.3221/IGF-ESIS.55.0. [2] Vasco, M. C., Polydoropoulou, P., Chamos, A. N. and Pantelakis S. G. (2017). Experimental characterization of a new sustainable sand concrete in an aggressive environment, Frattura ed Integrità Strutturale, 42, pp. 9–22. DOI: 10.3221/IGF-ESIS.42.02. [3] Bhaskar, S., Gettu, R., Bharatkumar, B. H. and Neelamegam M. (2011). Studies on chloride induced corrosion of reinforcement steel in cracked concrete, SDHM Structural Durability and Health Monitoring, 7(4), pp. 231–251. DOI: 10.3970/sdhm.2011.007.231. [4] Golewski, G. L. (2019). New principles for implementation and operation of foundations for machines: A review of recent advances. Structural Engineering and Mechanics, 71(3), 317–327. DOI: 10.12989/SEM.2019.71.3.317. [5] Wang, D., Dong, Y., Zhang, D., Wang, W. and Lu X. (2020). Monitoring and analysis of reinforced concrete plate-column structures under room temperature and fire based on acoustic emission, Frattura ed Integrità Strutturale, 53, pp. 9–22. DOI: 10.3221/IGF-ESIS.53.20. [6] Golewski, G. L. (2020). Changes in the Fracture Toughness under Mode II Loading of Low Calcium Fly Ash (LCFA) Concrete Depending on Ages. Materials, 13, 5241. DOI: 10.3390/ma13225241. [7] Golewski, G. L.; Gil, D. M. (2021). Studies of Fracture Toughness in Concretes Containing Fly Ash and Silica Fume in the First 28 Days of Curing. Materials, 14, 319. DOI: 10.3390/ma14020319. [8] Golewski, G. L. (2021). The Beneficial Effect of the Addition of Fly Ash on Reduction of the Size of Microcracks in the ITZ of Concrete Composites under Dynamic Loading. Energies, 14, 668. DOI: 10.3390/en14030668. [9] Lamonaca, F., Sciammarella, P. F., Scuro, C., Carnì, D. L. and Olivito, R. S. (2018). Internet of Things for Structural Health Monitoring. 2018 IEEE International Workshop on Metrology for Industry 4.0 and IoT, Brescia, Italy, 16–18 April, pp. 95–100. DOI: 10.1109/METROI4.2018.8439038. [10] Schabowicz, K. (2019). Non-Destructive Testing of Materials in Civil Engineering, Materials, 12, 3237. DOI: 10.3390/ma12193237. [11] Kogbara, R. B., Iyengar, S. R., Grasley, Z. C., Masad, E. A. and Zollinger, D. G. (2015). Non-destructive evaluation of concrete mixtures for direct LNG containment, Materials & Design, 82, pp. 260-272. DOI: 10.1016/j.matdes.2015.05.084. [12] Wiggenhauser, H., Köpp, C., Timofeev, J. et al. (2018). Controlled Creating of Cracks in Concrete for Non-destructive Testing, Journal of Nondestructive Evaluation, 37(67). DOI: 10.1007/s10921-018-0517-x. [13] Ouahabi, A. (2013). Signal and Image Multiresolution Analysis, Wiley-ISTE Ltd, London (UK) and Hoboken (New Jersey, USA). ISBN: 978-1-118-56866-8. [14] He, W.-Y., Zhu, S. and Chen, Z.-W. (2017). Wavelet-based multi-scale finite element modeling and modal identification for structural damage detection, Advances in Structural Engineering, 20(8), pp. 1185–1195. DOI: 10.1177/1369433216687566. [15] Kim, J. J., Kim, A.-R. and Lee, S.-W. (2020). Artificial Neural Network-Based Automated Crack Detection and Analysis for the Inspection of Concrete Structures, Applied Sciences, 10, 8105. DOI: 10.3390/app10228105. [16] Li, S. and Zhao, X. (2019). Image-Based Concrete Crack Detection Using Convolutional Neural Network and Exhaustive Search Technique, Advances in Civil Engineering, 2019, pp. 1–12. DOI: 10.1155/2019/6520620. [17] Bulatovic, S., Aleksic, V. and Milovic L. (2013). Failure of steam line causes determined by NDT testing in power and heating plants, Frattura ed Integrità Strutturale, 26, pp. 41–48. DOI: 10.3221/IGF-ESIS.26.05. A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 46 [18] Prakash, R. V. and John M. (2019). Post-Impact Fatigue Damage Analysis of Quasi Isotropic CFRP Laminates through Infrared Thermography, 49, pp. 536–546. DOI: 10.3221/IGF-ESIS.49.50. [19] Akgul, F. (2020). Inspection and evaluation of a network of concrete bridges based on multiple NDT techniques, Structure and Infrastructure Engineering. DOI: 10.1080/15732479.2020.1790016. [20] Lee, T., Lee, J. and Choi, H. (2020). Assessment of Strength Development at Hardened Stage on High-Strength Concrete Using NDT, Applied Sciences, 10(18), 6261. DOI: 10.3390/app10186261. [21] Jaskowska-Lemańska, J. and Sagan, J. (2019). Non-destructive testing methods as a main tool supporting effective waste management in construction processes, Archives of Civil Engineering, 65(4), pp. 263–276. DOI: 10.2478/ace-2019-0059. [22] Girault, J.-M., Ossant, F., Ouahabi, A., Kouamé, D. and Patat, F. (1998). Time-varying autoregressive spectral estimation for ultrasound attenuation in tissue characterization, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 45(3), pp. 650–659. DOI: 10.1109/58.677609. [23] Labat, V., Remenieras, J.-P., Matar, O. B., Ouahabi, A. and Patat, F. (2000). Harmonic propagation of finite amplitude sound beams: experimental determination of the nonlinearity parameter B/A, Ultrasonics, 38(1–8), pp. 292–296. DOI: 10.1016/S0041-624X(99)00113-4. [24] Sadowski, Ł., Hoła, J., Czarnecki, S. and Wang, D. (2018). Pull-off adhesion prediction of variable thick overlay to the substrate, Automation in Construction, 85, pp. 10–23. DOI: 10.1016/j.autcon.2017.10.001. [25] Louhi Kasahara, J. Y., Yamashita, A. and Asama, H. (2020). Acoustic Inspection of Concrete Structures Using Active Weak Supervision and Visual Information, Sensors, 20(3), 629. DOI: 10.3390/s20030629. [26] Ouahabi, Y. R., Bensadok, K. and Ouahabi, A. (2021). Optimization of the Biomethane Production Process by Anaerobic Digestion of Wheat Straw Using Chemical Pretreatments Coupled with Ultrasonic Desintegration, Sustainability, 13(13), 7202. DOI: 10.3390/su13137202. [27] Yuan, F., Yu, Y., Li, L. and Tian, G. (2021). Investigation of DC Electromagnetic-Based Motion Induced Eddy Current on NDT for Crack Detection, IEEE Sensors Journal, 21(6), pp. 7449–7457. DOI: 10.1109/JSEN.2021.3049551. [28] Rodrigues, R., Gaboreau, S., Gance, J., Ignatiadis, I. and Betelu, S. (2021). Reinforced concrete structures: A review of corrosion mechanisms and advances in electrical methods for corrosion monitoring, Construction and Building Materials, 269, 121240. DOI: 10.1016/j.conbuildmat.2020.121240. [29] Moosavi, R., Grunwald, M. and Redmer, B. (2020). Crack detection in reinforced concrete, NDT & E International, 109, 102190. DOI: 10.1016/j.ndteint.2019.102190. [30] Levitt, M. (2004). Precast Concrete: Materials, Manufacture, Properties and Usage, Applied Science Publishers, London (UK) and New Jersey (USA). ISBN: 0-203-49057-6. [31] De Moortel, I. and Hood, A. W. (2000). Wavelet analysis and the determination of coronal plasma properties, Astronomy and astrophysics, 363, pp. 269–278. [32] Ramsey, J. B. (2002). Wavelets in Economics and Finance: Past and Future, Studies in Nonlinear Dynamics and Econometrics, 6(3). DOI: 10.2202/1558-3708.1090. [33] Shiralashetti, S. C. and Deshi, A. B. (2017). Numerical solution of differential equations arising in fluid dynamics using Legendre wavelet collocation method, International Journal of Computational Materials Science and Engineering, 6(2), 1750014. DOI: 10.1142/S2047684117500142. [34] Su, T., Li, Z. and Kahn, R. (2020). A new method to retrieve the diurnal variability of planetary boundary layer height from lidar under different thermodynamic stability conditions, Remote Sensing of Environment, 237, 111519. DOI: 10.1016/j.rse.2019.111519. [35] Guetbi, C., Kouamé, D., Ouahabi, A. and Remenieras, J.-P. (1997). New emboli detection methods [Doppler ultrasound], 1997 IEEE Ultrasonics Symposium Proceedings, An International Symposium (Cat. No.97CH36118), Toronto, ON, Canada, 5–8 October, pp. 1119–1122. DOI: 10.1109/ULTSYM.1997.661775. [36] Ouahabi, A. (2010). Multifractal analysis for texture characterization: A new approach based on DWT, Proceedings of the 10th International Conference on Information Science, Signal Processing and their Applications (IEEE/ISSPA), Kuala Lumpur, Malaysia, 10–13 May, pp. 698–703. DOI: 10.1109/ISSPA.2010.5605576. [37] Djeddi, M., Ouahabi, A., Batatia, H., Basarab, A. and Kouamé, D. (2010). Discrete wavelet transform for multifractal texture classification: Application to ultrasound imaging, Proceedings of the 17th IEEE International Conference on Image Processing (ICIP), Hong Kong, China, 26–29 September, pp. 637–640. DOI: 10.1109/ICIP.2010.5650017. [38] Sui, K. and Kim, H.-G. (2019). Research on application of multimedia image processing technology based on wavelet transform, EURASIP Journal on Image and Video Processing, 24. DOI: 10.1186/s13640-018-0396-1. [39] Haneche, H., Ouahabi, A. and Boudraa, B. (2019). New mobile communication system design for Rayleigh environments based on compressed sensing-source coding, IET Communications, 13(15), pp. 2375–2385. A. Arbaoui et alii, Frattura ed Integrità Strutturale, 58 (2021) 33-47; DOI: 10.3221/IGF-ESIS.58.03 47 DOI: 10.1049/iet-com.2018.5348. [40] Haneche, H., Boudraa, B. and Ouahabi, A. (2020). A new way to enhance speech signal based on compressed sensing, Measurement, 151, pp. 107–117. DOI: 10.1016/j.measurement.2019.107117. [41] Ait Aouit, D. and Ouahabi A. (2008). Monitoring crack growth using thermography, Comptes Rendus Mécanique, 336, pp. 677–683. DOI: 10.1016/j.crme.2008.06.001. [42] Girault, J.-M., Kouamé, D. and Ouahabi, A. (2010). Analytical formulation of the fractal dimension of filtered stochastic signal, Signal Processing, 90(9), pp. 2690–2697. DOI: 10.1016/j.sigpro.2010.03.019. [43] Ait Aoui, D. and Ouahabi, A. (2011). Nonlinear Fracture Signal Analysis Using Multifractal Approach Combined with Wavelet, Fractals - Complex Geometry, Patterns, and Scaling in Nature and Society, 19(18), pp. 175–183. DOI: 10.1142/S0218348X11005270. [44] Ouahabi, A. and Femmam, S. (2011). Wavelet-based multifractal analysis of 1-D and 2-D signals: new results, Analog Integrated Circuits and Signal Processing, 69(1), pp. 3–15. DOI: 10.1007/s10470-011-9620-y. [45] Jacques, S., Caldeira, A., Batut, N., Schellmanns, A., Leroy, R. and Gonthier, L. (2012). Lifetime prediction modeling of non-insulated TO-220AB packages with lead-based solder joints during power cycling, Microelectronics Reliability, 52(1), pp. 212–216. DOI: 10.1016/j.microrel.2011.08.017. [46] Calvez, D., Roqueta, F., Jacques, S., Béchou, L., Ousten, Y. and Ducret, S. (2014). Crack Propagation Modeling in Silicon: a Comprehensive Thermo-Mechanical FEM Approach for Power Devices, IEEE Transactions on Components Packaging Manufacturing Technologies, 4(2), pp. 360–366. DOI: 10.1109/TCPMT.2013.2293094. [47] AlZubi, S., Islam, N. and Abbod, M. (2011). Multiresolution Analysis Using Wavelet, Ridgelet, and Curvelet Transforms for Medical Image Segmentation, International Journal of Biomedical Imaging, 2011. DOI: 10.1155/2011/136034. [48] Anila, S., Sivaraju, S.S. and Devarajan, N. (2017). A New Contourlet Based Multiresolution Approximation for MRI Image Noise Removal. National Academy Science Letters, 40, pp. 39–41. DOI: 10.1007/s40009-016-0498-1. [49] Kafieh, R., Rabbani, H. and G. Unal. (2019). Bandlets on Oriented Graphs: Application to Medical Image Enhancement, IEEE Access, 7, pp. 32589–32601. DOI: 10.1109/ACCESS.2019.2903467. [50] Pan, Y. and Zhang, L. (2021). Roles of artificial intelligence in construction engineering and management: A critical review and future trends, Automation in Construction, 122, 103517. DOI: 10.1016/j.autcon.2020.103517. [51] Adjabi, I., Ouahabi, A., Benzaoui, A. and Taleb-Ahmed, A. (2020). Past, Present, and Future of Face Recognition: A Review, Electronics, 9(8), 1188. DOI: 10.3390/electronics9081188. [52] Ouahabi, A. and Taleb-Ahmed, A. (2021). Deep learning for real-time semantic segmentation: Application in ultrasound imaging, Pattern Recognition Letters, 144, pp. 27–34. DOI: 10.1016/j.patrec.2021.01.010. [53] Mimouna, A., Alouani, I., Ben Khalifa, A., El Hillali, Y., Taleb-Ahmed, A., Menhaj, A., Ouahabi, A. and Ben Amara, N. E. (2020). OLIMP: A Heterogeneous Multimodal Dataset for Advanced Environment Perception. Electronics, 9, 560. DOI: 10.3390/electronics9040560. [54] Dung, C. V. and Anh, L. D. (2019). Autonomous concrete crack detection using deep fully convolutional neural network, Automation in Construction, 99, pp. 52–58. DOI: 10.1016/j.autcon.2018.11.028. [55] Adjabi, I., Ouahabi, A., Benzaoui, A. and Jacques, S. (2021). Multi-Block Color-Binarized Statistical Images for Single- Sample Face Recognition, Sensors, 21(3), 728. DOI: 10.3390/s21030728. [56] Wu, X. and Liu, X. (2021). Building crack identification and total quality management method based on deep learning, Pattern Recognition Letters, 145, pp. 225–231. DOI: 10.1016/j.patrec.2021.01.034. [57] Khan, A., Sohail, A., Zahoora, U. and Qureshi, A. S. (2020). A survey of the recent architectures of deep convolutional neural networks, Articial Intelligence Review, 53(8), pp. 5455–5516. DOI: 10.1007/s10462-020-09825-6. [58] Abbas, Q., Ibrahim M. E. A. and Jaffar, M. A. (2019). A comprehensive review of recent advances on deep vision systems, Artificial Intelligence Review, 52(1), pp. 39–76. DOI: 10.1007/s10462-018-9633-3. [59] Simonyan, K. and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7–9. [60] Yuan, L., Qu, Z., Zhao, Y., Zhang, H. and Nian, Q. (2017). A convolutional neural network based on TensorFlow for face recognition. 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China, 25–26 March, pp. 525–529. DOI: 10.1109/IAEAC.2017.8054070. [61] Maguire, M., Dorafshan, S. and Thomas, R. J. (2018). SDNET2018: a concrete crack image dataset for machine learning applications, Utah State University, Logan. DOI: 10.15142/T3TD19.