Bridge defect detection technology based on machine vision and embedded

In order to improve the degree of intelligence of bridge body maintenance, reduce the consumption of manpower and material resources, and improve inspection efficiency, the technology of machine vision inspection is proposed, and a method of detecting bridge body surface defects and embedded design image transmission and positioning is studied. This paper introduced the existing detection methods and technologies of bridge cracks, and analyzed the detection methods of bridge cracks based on image processing and convolutional neural networks. On the basis of this research, the project collected information and processed and analyzed the cracks on the actual bridge deck, and obtained the image preprocessing results of grayscale, removed background noise, thresholded segmentation, and retained edge information and high-frequency signals. And further, the project used drones to capture image data and used deep learning algorithms to achieve automatic detection of aerial crack images. This scheme can effectively reduce the impact of road background and noise on the detection of bridge surface defects.


Introduction
Bridge inspection technology is an important means to control and inspect bridges. The document requires that during the "14th Five-Year Plan" period, the bridge renovation and reconstruction plan should be carried out [1] .
At present, the inspection methods mainly include bridge inspection vehicles, ultrasonic inspection and optical fiber sensor inspection. Among them, Hou Yuemin and others studied the application of ultrasonic flat surveying method in concrete bridges [2] . Use the propagation of ultrasonic waves in the medium to detect the depth of cracks. Song Pei studied the application of optical fiber sensing in the detection of urban bridges [3] . Its design can only target a single bridge and does not have universal applicability. In recent years, a new technology that uses digital image processing has also achieved great results. Liu Meng et al. used various image algorithms to analyze defects [4] , but from the results, the background noise interference is still relatively Big. Some studies use stationary wavelet algorithm combined with SPCNN algorithm optimization method to identify cracks [5] .The equipment cost involved in the above vision algorithm is high. Aiming at the shortcomings and limitations of the abovementioned existing detection methods, a bridge defect detection technology based on machine vision and embedded is proposed and designed. The recognition of bridge cracks and other defects is realized by combining a microcontroller and a neural network. The image acquisition system uses the Raspberry Pi 4B-based main control board to control it. Because the Raspberry Pi has the characteristics of small size, sufficient computing power, and convenient expansion, it is suitable for information processing and control of UAV load. Its own camera interface is convenient to expand the CMOS camera module. On this basis, the system uses a KS12A884 camera module, which is manufactured using SMT technology and supports up to 1200w high-definition camera. Supporting Ubuntu system makes it compatible with Raspberry Pi. The average number of shooting frames can reach 30 frames. Experimental shooting meets the requirements. With the addition of the zoom module, the requirements for different focal lengths can be achieved. The acquired camera information is saved in frames by algorithm, and the acquired original video image is converted into framed image data.

Image transmission system
As the UAV is far away from the ground when collecting images, and requires a relatively stable transmission route, the image transmission module of HQL-010 is used to transmit the image data, and the transmission distance is 800 meters to 15 kilometers. And it supports TTL serial ports, comes with dual-transmit and dual-receive antennas, and can achieve a maximum transmission rate of 30Mbps. With low power consumption and high sensitivity, it is suitable for UAV image data transmission and can meet the data transmission requirements of experiments. After docking with the Raspberry Pi, you can use the Raspberry Party Video Transmission Module to control it so that it starts video transmission at the specified time.

Positioning and perception system
In order to obtain the real-time information and location data of the defect, the design adopts the Beidou-GPS dual-mode positioning chip combined with the antenna to locate the longitude and latitude position. When the controller combines with the host computer to perform image processing to identify the crack or damage of the bridge body, it will execute an interrupt program. The interrupt program receiving interrupt will return the latitude and longitude data to the host computer. The laser ranging module can be connected through the Raspberry Pi serial port. Realize real-time detection of height information. Use the USB to serial port module CP2102 to connect Beidou-GPS dual-mode positioning with the serial port ttyUSB0 of the Raspberry Pi. The module can be called and tested through Python using the GPS library. The positioning and height perception module can better realize the acquisition of the location information of the bridge deck defects, and realize the detection and sharing of real-time location information on the host computer through the visual programming interface.

Image grayscale and image enhancement
The image collected by the pontic image processing is an RGB color image. The RGB image needs to be processed separately for the three components of RGB during processing. The three-channel image optically adjusts the color and does not reflect the essential characteristics of the image. The key role is to reduce the extraction and processing of irrelevant information by graying the image.
The image acquired by the COMS camera has uneven brightness distribution, which will affect the processing process such as edge detection. In order to make the gray value distribution more uniform, a continuous and uniform gray histogram is obtained by using the obtained mapping relationship of the reorganized histogram.
S T r (1) 1 (2) Among them, S is the image after transformation, and r is the image before transformation. p(r) is the probability density of r, and p(s) is the probability density function of s. Through the principle of image science, the gray-scale transformation in the preprocessing stage will not destroy the pixel distribution of the original image and maintain the number of pixels . The algorithm calculates the proportion of each gray level, uses the proportion relationship and statistical law to calculate the cumulative distribution function of each gray level, and then judges the corresponding gray interval after the change through the cumulative distribution function. The pixel level in the transformed image is evenly distributed, thereby achieving the effect of improving image contrast.

Filtering algorithm
According to the experimental results, the median filter can be compared with the mean filter, which has a better performance in removing salt and pepper noise. The image acquired by the CMOS camera retains salt and pepper noise and part of Gaussian noise. Using the 9×9 movement of the pixel template, the pixel values in the template are rearranged, and the center value is replaced with the median value. After median filtering, Gaussian filter is applied to reduce the effect of full-screen noise.

Laplacian edge enhancement
The salt and pepper noise will be eliminated after the median filtering in the pixels, but there will be a lot of noise on the image. The distribution of these noises is rough and will blur the contour edges, interfere with the shadow contours of the bridge cracks, and bring errors to the subsequent length and width measurement Therefore, it is necessary to further process the edge contour to strengthen the crack contour. This step is of great significance for edge extraction and strengthening. Through the introduction of the Laplacian operator, a high-pass filter based on the Laplacian operator is constructed, which can allow high-energy signals to pass through, while filtering out most of the low-energy noise points to maintain and enhance the effect of edge features.
For image blur and quality defects caused by diffusion phenomenon, the following imaging principles can be used to sharpen: K parameter is a parameter related to the diffusion effect of the image, K is too large, the image sharpening effect is too strong; K is too small, the sharpening effect is not obvious.

Image background subtraction technology
Threshold segmentation is a processing method of searching for connected domains. Through this algorithm, we can segment the object parts we need. Based on the image segmentation and detection method of the connected domain of the target object, it is necessary to manually set the gray threshold of the image, that is, different threshold setting methods will determine the discarding and change of the pixel points of the acquired image. The OTSU threshold method is used to automatically select Threshold to cooperate to improve image display. In order to improve the adaptability of the threshold segmentation algorithm to various research backgrounds and environments, the Trackbar is set to adjust the target image threshold in real time to eliminate the negative impact of changes in the background of different brightness on the accuracy of image threshold division.

Framework design of neural network
Using the AlexNet-based network framework, a total of 96 11×11 convolution kernels are set up. When the 5×5 sampling window is advanced to the next level and the third layer, 27×27 dimensions will be established, a total of 256 convolutions Core, four layers are similar to get 13×13 dimensions, a total of 384 convolution kernels. This neural network framework mainly has five convolutional network layers and three fully-connected network layers. The input target image size is 224×224, and it needs ×3 under three channels. Image feature extraction is performed through related convolution operations. The convolution kernel's sliding depends on the parameter setting of the step length. In the training model, the loss cost loss function is obtained by making the difference between the actual value and the expected value. Through the chain derivation rule, the accumulation of errors will gradually pass. The weight and offset value need to be updated and corrected in the process. The Logistic regression function and its loss function are as follows: y (5) a f y (6) L ylog α 1 log 1 α (7) w ≔ w α (8) The f function is the activation function, and the ReLU function is selected as the activation function. The back propagation process uses the gradient descent method to continuously update the values of W and b. The learning rate α) setting of the network will have a relatively obvious impact on the network gradient descent. If the setting is too small, the model optimization efficiency will not decrease significantly. If the setting is too large, there will be a relatively large deviation between the final optimal solution and the desired optimal solution.

The preparation and training process of bridge data set of convolutional neural network
First prepare your own data set and use the framed pictures taken by the survey drone. The data set is divided into a training set and a validation set. The experiment selected and intercepted about 6000 bridge deck defect images of different bridge decks in different flight attitudes. , 5000 sheets are used as the training set to train the neural network, and the remaining 1000 sheets are used to verify the accuracy of neural network recognition. Generate files such as your own network tags based on the test code. Resize the image into a 256×256 size image, and then generate the corresponding lmdb files.
The training steps of this neural network are: 1. The network initializes the weights, sets the learning rate, batch and epoch values; 2. The forward propagation of the image data through the convolutional layer, the down-sampling layer and the fully connected layer obtains the forward output value; 3. Find the error value between the forward output value of the neural network and the target value through the cost loss function; 4. When it is judged that the error is greater than our expected value, the error is sent back to the network, and the error of the fully connected layer and the convolutional layer is returned.

5.
Return to the second step in the way of weight update, repeat training until the error is equal to or less than the expected value of the system, stop training and output the result.

Algorithm improvement based on Caffe network
The Caffe framework gives the definition process of the model, and constructs the network by optimizing network settings and pre-training weights. By combining Caffe and cuDNN, the AlexNet model [6] was tested on K40 to predict the processing time for each bridge crack image in 1.24ms. The Caffe framework has also adopted its modular settings and optimizations so that the model can be transplanted to other devices. Define your own model by changing the types of layers provided by Caffe, and finally get a network structure that meets your needs.
After many debugging and practice, after adjusting and optimizing the parameters under different working conditions, the prediction effect is better. Overall meet the design requirements, with excellent performance.

Result analysis
By processing the pre-processed image and putting it into the data set, the detection effect and accuracy can be improved. In order to verify the recognition accuracy of the system algorithm and the feasibility of the embedded device, we used the neural network pre-trained model file to actually predict the realtime campus road crack image, and transmitted the image data and location information to the wireless transmission module. Crack defects can still be detected under the condition of more road noise interference. The MAP of the model obtained under 6000 data sets can reach about 91.7%.The upper computer, and finally get the experimental test results on the upper computer as shown below: Figure 6 Neural network crack recognition results

Conclusion
A method based on the combination of digital image processing and neural network is used to detect the defects of the bridge deck mainly for cracks. Through the improved neural network algorithm and the parameter settings after multiple optimizations, the identification of cracks with high accuracy is realized. In this paper, measures such as background subtraction and supplementing the pre-processed image to the data set have improved the detection speed and detection accuracy. It has better adaptability to complex bridge deck environment. It is of great significance to the daily maintenance of the bridge. Through the design of the embedded system, it is possible to realize the wireless transmission of the position information of the bridge deck defect and the image information, realize the frame operation and image preprocessing. After the algorithm is transplanted to the Raspberry Pi, the self-detection can be realized, and the upper computer is responsible for the second Times verification.