TECHNICAL FIELD

The present invention relates to the field of infrastructure inspection. More particularly, the invention relates to a visual recognition artificial intelligence-based system and an associated process that replace visual imagery evaluation and defect detection by human inspectors.

RELATED APPLICATIONS

This application claims priority from U.S. Provisional Patent Application No. 62/862,367 filed Jun. 17, 2019 entitled “ARTIFICIAL INTELLIGENCE-BASED PROCESS AND SYSTEM FOR VISUAL INSPECTION OF INFRASTRUCTURE”. For the purposes of the United States, this application claims the benefit under 35 USC § 119 of United States Provisional Patent Application No. 62/862,367 filed Jun. 17, 2019 entitled “ARTIFICIAL INTELLIGENCE-BASED PROCESS AND SYSTEM FOR VISUAL INSPECTION OF INFRASTRUCTURE”, which is incorporated herein by reference in its entirety.

BACKGROUND

A healthy economy is dependent on well maintained and properly upgraded civil infrastructure, such as transmission lines, pipelines, bridges etc. To maintain this infrastructure, its elements need to be routinely inspected to assess their condition and estimate their long-term viability. Infrastructure inspection entails the evaluation of the physical and functional conditions of infrastructure systems.

Conventional inspection methods rely on visual detection of any changes, such as discolorations, cracks, and corrosion on the surface of the structures over time. Inspections involve trained personnel visiting the site and assessing the structures, hardware and related components, and recording the qualitative aspects of the infrastructure. To ensure that defects are detected before they can cause deficiency or failure in an infrastructure system, inspections need to be carried out at regular intervals.

Traditional inspection processes rely on human inspectors physically visiting sites and conducting the evaluation. This can be time consuming, expensive, and dangerous, and often requires access to hard-to-reach locations and interruption of the infrastructure system's regular operation. As a result, currently available inspection methods limit inspection procedures and practices in terms of time and access, resulting in longer than optimal periods between inspections. This hampers effective decision-making regarding proactive maintenance and upgrades.

Conventional inspection methods are also expensive because they rely on trained professionals' time, often under hazardous working conditions. Further, these inspection methods rely on human judgement and as a result they are inherently error prone. Any judgment made by a human is dependent on prior training and experience, as well as on the current state of attention and information processing capability, resulting in variable performances.

While recent advancement in automation and robotic technology has increased the speed, safety, and efficiency of data collection in the field, no such automation has been done for the data evaluation part of the inspection process. Photographs are still evaluated by human inspectors in order to detect defects and the resulting remediation recommendations are then provided by the human inspectors. While some systems may be able to identify defects, human inspectors are still relied upon to provide the appropriate remediation recommendations. Thus, existing processes are time-consuming, error-prone, and limited by the availability of trained personnel. With the increasing age of the workforce, keeping a sufficient number of inspectors on staff is becoming difficult.

Therefore, there is a need for inspection systems and processes that: (i) can identify defects and provide corresponding remediation recommendations without human involvement; and (ii) are fast, reliable, provide repeatable results, efficient, cost-effective, flexible, and adaptable.

The foregoing examples of the related art and limitations related thereto are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the drawings.

SUMMARY

According to an aspect of the present disclosure, there is provided a method comprising: receiving image data comprising images of infrastructure defects, annotated with classification data; allocating a first portion of the image data as training image data; allocating a second portion of the image data as validation image data; training an artificial intelligence-based image recognizer using the training image data; and validating the image recognizer using the validation image data.

According to an aspect of the present disclosure, the method further comprises: inputting an inspection image into the image recognizer; analyzing, by the image recognizer, the inspection image; identifying, by the image recognizer, a defect in the inspection image; locating, by the image recognizer, a remediation recommendation associated with the defect; and generating an output report based on the defect and the remediation recommendation.

In some embodiments, the output report includes identification and classification of the defect and one or more of: a potential cause of the defect; severity of the defect; and a recommendation on how to address the defect. The classification data may include defect type (e.g. discoloration, fatigue, environmental) and/or severity (e.g. low risk, medium risk, and high risk).

In particular embodiments, a plurality of images of infrastructure is provided, and the plurality of images is filtered to generate a set of inspection images for input to the image recognizer. Filtering the plurality of images may include one or more of: selecting images of high definition (e.g. minimum 1280p×720p resolution), selecting images that show a close-up image of the feature and/or defect, selecting images in which the feature is clearly visible without obstructions; and selecting images captured with good lighting conditions.

Validating the image recognizer may include calculating a recognition accuracy level of the image recognizer and comparing the recognition accuracy level with a threshold confidence level. In particular embodiments, anywhere between 5% to 70% of the image data is initially allocated as the training image data and the remainder of the image data is initially allocated as the validation image data. If the recognition accuracy level is below a threshold confidence level, the allocation of training image data and validation image data may be adjusted to improve the recognition accuracy level.

Another aspect of the present disclosure relates to a system for automatically inspecting infrastructure. The system includes a database management component configured to collect image data including images of infrastructure defects annotated with classification data, allocate a first portion of the image data as training image data, and allocate a second portion of the image data as validation image data. The system also includes an artificial intelligence-based image recognizer configured to receive the image data from the database management component and apply deep learning techniques to develop processes to identify and classify infrastructure defects.

According to particular embodiments, the image recognizer is configured to receive an inspection image, analyze the image, identify a defect in the inspection image, locate in a database a remediation recommendation associated with the defect, and generate an output report based on the defect and the remediation recommendation. The output report may include identification and classification of the defect and one or more of: a potential cause of the defect; severity of the defect; and a recommendation on how to address the defect. The classification data may include defect type (e.g. discoloration, fatigue, environmental) and/or severity (e.g. low risk, medium risk, and high risk).

Some embodiments of the system include an image filtering component configured to receive a plurality of images of infrastructure, and filter the plurality of images to generate a set of inspection images for input to the image recognizer. The image filtering component may be configured to perform one or more of: selecting images of high definition (e.g. minimum 1280p×720p resolution), selecting images that show a close-up image of the feature and/or defect, selecting images in which the feature is clearly visible without obstructions; and selecting images captured with good lighting conditions. Certain embodiments of the system include a validation component configured to receive the output reports generated by the image recognizer using validation data, and compare them to the output reports generated by the image recognizer using the training data, calculate a recognition accuracy level of the image recognizer, and compare the recognition accuracy level with a threshold confidence level.

The database management component may be configured to initially allocate anywhere between 5% to 70% of the image data as the training image data and allocate the remainder of the image data as the validation image data. The validation component may cause the database management component to adjust the allocation of training image data and validation image data to improve the recognition accuracy level if the recognition accuracy level is determined to be below a threshold confidence level.

In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and by study of the following detailed descriptions.

