Robotic Inspection and Characterization of Subsurface Defects on Concrete Structures Using Impact Sounding

Impact-sounding (IS) and impact-echo (IE) are well-developed non-destructive evaluation (NDE) methods that are widely used for inspections of concrete structures to ensure the safety and sustainability. However, it is a tedious work to collect IS and IE data along grid lines covering a large target area for characterization of subsurface defects. On the other hand, data processing is very complicated that requires domain experts to interpret the results. To address the above problems, we present a novel robotic inspection system named as Impact-Rover to automate the data collection process and introduce data analytics software to visualize the inspection result allowing regular non-professional people to understand. The system consists of three modules: 1) a robotic platform with vertical mobility to collect IS and IE data in hard-to-reach locations, 2) vision-based positioning module that fuses the RGB-D camera, IMU and wheel encoder to estimate the 6-DOF pose of the robot, 3) a data analytics software module for processing the IS data to generate defect maps. The Impact-Rover hosts both IE and IS devices on a sliding mechanism and can perform move-stop-sample operations to collect multiple IS and IE data at adjustable spacing. The robot takes samples much faster than the manual data collection method because it automatically takes the multiple measurements along a straight line and records the locations. This paper focuses on reporting experimental results on IS. We calculate features and use unsupervised learning methods for analyzing the data. By combining the pose generated by our vision-based localization module and the position of the head of the sliding mechanism we can generate maps of possible defects. The results on concrete slabs demonstrate that our impact-sounding system can effectively reveal shallow defects.


INTRODUCTION
The civil infrastructure in the United States of America is reaching pass its life expectancy and the cost of repairs is going to tremendously increase over the next years. ASCE 2021 report card scores America's infrastructure with an average C-grade.
According to this report there are more than 617,000 bridges and 42% of them are over 50 years old, with 7.5% of them considered to be in a poor condition or structurally deficient. Condition of bridges is graded with C, a decrease from C+ from the ASCE 2017 report card. It is critically important to increase the inspection frequency so we can detect and repair the defects before any additional deterioration damages these structures beyond the repair point. The inspection of infrastructure is a time-consuming, expensive and potentially a dangerous process. Inspectors use NDT methods like impact-echo [1], acoustic inspection [2], ultrasonic [3], GPR [4], visual inspection [5] etc., to detect defects like cracks, delamination, voids, rebar corrosion etc. GPR [15] [16], is one of the most widely used methods in detecting defects like voids, delamination, buried objects and metallic rebars [6], corrosion [7] etc. Impact-echo method is used to detect delamination, cracks and voids in concrete structures [1]. However, impact-echo although successful still lacks any good result for shallow delamination [8].
The focus of this paper is detection of shallow defects using automated robotic impact-sounding and generation of high-resolution maps. First, an automated robotic solution with wall-climbing capabilities is developed. This automated solution drastically decreases the necessary time to collect data, improves the data consistency so all data points are collected under very similar condition, and finally frees the inspectors from the tedious task of collecting data manually. Second, we use FFT [9] to extract features out of the raw sound files. We show that by using other features other than just frequency peaks, we can use very noisy sound recordings recorded with an affordable microphone, without any physical filter required, and are able to detect voids and delamination in the concrete slab. This paper is potentially impactful, because we use K-Means [10] and Spectral Clustering [26] to analyze the data and we get very good results.

Robotic System and Software
We developed a robotic inspection system with vertical mobility which is called Impact-Rover (see Figure 1). The robot consists of two DC brushless motors for  differential drive locomotion, suction motor for climbing on concrete walls, a solenoid impactor on a sliding mechanism that is controlled by a stepper motor, specially designed embedded system control board, accelerometer with data acquisition module for IE, a microphone for IS, RealSense D435i RGBD camera for localization and Intel NUC on-board computer. Impact-Rover can collect data automatically in a pre-defined area.

Impact-Sounding and Impact-Echo Modules
The impact-sounding module consist of an Olson Solenoid for generating the impact. This solenoid can tap the surface and provide 0.011 to 0.019 Joules per hit. Sound data are recorded using 3.5 mm standard microphone. IS and IE devices both rest on the sliding mechanism to collect data at an adjustable spacing.
The IE module consist of three important parts. An accelerometer from PCB Inc. with sensitivity of 100 mV/g, frequency range (±3 dB) 0.5 to 20 kHz. Digital ICP Signal Conditioner Model 485B39 which connects through USB port and is used to transfer the data to the on-board computer. The servo motor is used to lower the IE transducer up and down to be in contact with the concrete surface for taking measurements.

Localization Module
In order to automate the data collection process, we need to determine the location of the robot with respect to the starting point and displacement of IE and IS sensors on the sliding mechanism. This way we can tag each measurement with position information. We use RealSense RGBD camera D435i to estimate the pose [11] [12][13] [14] and then fuse it with wheel's encoders to improve the localization accuracy of the robot.

IMPACT-SOUNDING BASED INSPECTION Data Processing and Feature Engineering
Due to the nature of the data and from our previous research experience [12][17] [18], we decided to use discrete Fourier Transform [19] or frequency domain analysis. This method decomposes the signal "($), where $ = 1, 2, … , + − 1 into spectral components and provides frequency information (see Equation 1). Thus, inputting a sound signal into Fourier Transform process, as output we get a function in the complex domain. This function represents the amount of each frequency present in the original signal. We perform Fast Fourier Transform (FFT) [20] on all impactsounding data we collected. For each sound signal we obtain the pair {. ! , / ! }, where / ! is the frequency vector and . ! corresponds to the amplitude vector, 2 = 1, 2, … , 3.
Feature extracting is the process of extracting non-redundant, abstract or meaningful information out of the data which better represents an underlying problem to the predictive models. Difference between sound signals collected over defective and nondefective concrete are not obvious in time-domain (see Figure 3 a, b and c). On the other hand, FFT signal shows a distinguishable pattern in energy (see Figure 3 d, e and f) i.e., magnitude of energy is lower over defects because they absorb impact energy. Our feature extraction method is inspired by [21], where we use FFT while they use Wavelet Transform [22]. Feature calculations is defined with the following expressions (see Equations 2 to 7 and Table I). These types of features capture different properties of the sound data. From mathematics we know that moments are quantitative measures related to the shape of the function. For these reasons we use spectral moments as features.

Unsupervised clustering and PCA
In this work we use unsupervised classification to classify non-defective and defective concrete. We use K-means [10], which is one of the most famous unsupervised clustering methods. Given the matrix of features 4 (see Equation 9), this algorithm partitions all observations (5 " , 5 # , … , 5 $ ) into 2 ≤ + sets Given an initial set of centroids 9 ":! " randomly initialized, algorithm [24] alternates between two steps. First, calculates the least squared Euclidean distance for each observation and assigns it to the closest centroid (see Equation 10). Second, algorithm recalculates the centroids for updated clusters (see Equation 11). Algorithm ends when 9 ":! ( ≈ 9 ":! ()" . We use Silhouette Coefficient [23] to determine the number of clusters.
Additionally, we use Principal Component Analysis (PCA) [25] for defect analysis and dimension reduction of feature matrix ;. PCA takes as input the matrix of data and calculates eigenvalues and eigenvectors, it only uses significant eigenvectors to recreate the data. We use this method in two ways. First, we use the filtering properties of this method for data processing that goes as input into K-means clustering. Second, we visualize some of the principal components and use image enhancement techniques [27] to improve the results.

EXPERIMENTS
We collected data in our concrete slab over the area that contains two types of defects, void and delamination. We collected 902 data points (.wav) with a 2cm spacing in x and y directions, over an area 20cm wide and 162cm long. Sounds are recorded using a sample rate of 44.1 kHz and a 16-bit AD. Each data point has an ID, which is tagged with the location information with respect to the starting point.

Feature Visualization
Features contain meaningful information; it helps to visualize the defects. From visualization (see Figure 5) we can clearly see the void box, while delamination circle is visible, but much less. We use enhancement on feature matrix to suppress the noise (Equation 12), leading to much clearer visualization results. For example, first and second moments are enhanced by squaring them, and power feature is enhanced by increasing it to the third power.

PCA Maps
We use PCA decomposition for the feature matrix ; (see Equation 9). For one component (C1) the amount of preserved information is 76.8%, for two components (C1+C2) 96.7% and for three (C1+C2+C3) 98.8%. For visualization we decided to use three components. We visualize all individual components and their linear combination (see Figure 6). They act as three combined predictors with equal weights. From results we can see that using PCA we are able to generate meaningful maps. Void is clearly visible, especially in the enhanced maps. Delamination is weakly detected.

Unsupervised Clustering Maps
Up to this point, we generated few types of data that can go into K-means and Spectral Clustering. First type is the feature matrix 4 (see Equation 8). Second type is enhanced feature matrix, where feature vectors are defined as 5 *+, 0 (see Equation 12). Third type is PCA. We use Silhouette coefficient to determine how good the clusters are separated, closer to 1 is better, -1 is worse. Table II shows the values for all cases when using two (defect/no defect) three clusters (void/delamination/no defect). According to the Table II, best clustering is achieved by Spectral Clustering using two clusters.
C2 C3 Figure 7. Clustering Results. a) Two clusters result matches with the ground truth and clearly finds the defects. b) Three clusters result shows that concrete around artificial defects is deteriorated. c) Scatter plots of PCA components with cluster information.
When we use ; and >@A(;, 3) as input for the K-means, the output is noisy and not very clear (see Figure 7a1, 7a2, 7b1 and 7b2). Feature enhancement improves the results (see Figure 7a1 and 7a3). Spectral Clustering uses the unmodified features ; and outputs very similar results with feature enhancement (see Figure 7a3 and 7a4). On the other hand, having three clusters differentiates the void and delamination (see Figure  7b). This figure shows parts of concrete close by artificial defects as weakened or deteriorated. Finally, 3D scatter plots are generated by using PCA decomposition with cluster information (see Figure 7c).

CONCLUSION
This paper introduces a robotic solution for data collection and an unsupervised learning method for impact-sounding data analysis. The robotic platform is equipped with visual, inertial and encoder fusion for high accuracy localization. It makes data collection much faster and improves data consistency. Data analysis algorithm starts with transforming the sound data into the spectral domain, then we calculate features. We use PCA to generate maps, then use K-means and Spectral Clustering for classification. Results from PCA maps and clustered maps can be used together to reveal defects. Experimental results show the effectiveness of our proposed method.

ACKNOLEDGEMENT
Financial support for this study was provided by the U.S. Department of Transportation under Grant No. 69A3551747126 through INSPIRE UTC. J. Xiao has significant financial interest in InnovBot LLC, a company involved in R&D and commercialization of the technology.
PCA Components Three Clusters

clusters 2 clusters
Two Clusters