FIELD

The present invention is related to image processing for road related incidents.

BACKGROUND

Although many technologies exist for surveying the roads, oftentimes the costs and constraints make it less appealing for practical use. Many solutions require modifications to the vehicle, costly attachments, computers and training to use. Usually these technologies specialize in surveying a single component of road condition. To detect the full spectrum of incidents expected to keep roads maintained, the most common and effective practice is still largely manual. Surveyors drive vehicles down roads and stop their vehicle when they find an object of interest. They take a picture, and then note down the incident details to revisit. This is a slow and costly method, oftentimes leaving a backlog of roads that are overdue for reassessment. As municipalities grow, it becomes harder and harder to stay on top of all the incidents in the system.

Municipalities occasionally outsource road assessment to companies which utilize specialized vehicles employing a wide array of sensors, including cameras, thermal, vibration, radar and laser. While providing a greater level of detail, a specialized vehicle is required to complete the task. The vehicle sensors require extensive calibration, and the processing of the data requires specialized systems and knowledge to produce a report. The costs are extensive, and therefore, such assessments are typically done periodically, where the period duration typically ranges from one year to ten years.

While there have been occasional instances of applications aimed at identifying road damage in video or images, such processes are typically done by first acquiring a data set (in the form of video and geo-positioning data), and then uploading the data to a specialized server where it is processed. The process is cumbersome as often the dataset is too large to transmit over cellular networks. Thus, disadvantageously, current systems must rely upon access to the complete image dataset only once the vehicle has returned to the image processing facility.

SUMMARY

It is an object of the present invention to provide an image acquisition system and/or method to obviate or mitigate at least one of the above presented disadvantages.

Current technologies for surveying roads are extremely costly, requiring complex installation of specialized sensors on a vehicle. They require extensive training and experience to operate and are typically limited only to analysis of pavement condition. Due to the high cost and limited capabilities of the available technologies, municipalities are currently relying on manual inspection to cover the detection of all road related incidents. Further, existing on board imaging systems used for navigation of a vehicle are not optimized for acquiring appropriate images containing objects of interest with respect to a road surface. As such, using a separate device containing a camera that is mounted to the vehicle, in addition to any onboard vehicle imaging systems (used for vehicle navigation), is desired.

Municipalities send their maintenance workers to drive the service vehicles on roads where they must visually locate road related concerns while on patrol or a service call. When workers spot a potential concern they are required to pull over and get off the vehicle to closely investigate the issue and manually initiate an incident or a work order using whatever system they have in place. It may be logged by pen and paper or utilizing a computerized tablet or smart phone.

Getting off the vehicle for each detected issue is not only time consuming but also put workers in a dangerous situation. Contrary to the current technologies available and the method that is implemented to detect road related incidents, the system provided here is capable of automated identification and reporting of a wide variety of road related incidents (e.g. various types of pavement damage and degradation, road sign damage, road debris, obstructions and other incidents visible from inside of a vehicle).

The system comprising of: device—a mobile computing device such as smartphone, or embedded computer system with built in or peripheral camera(s) and application utilizing neural network(s) and machine learning work together with server(s) that process and store data received from single or multiple devices, serve as gateway to users via web access and present data to users in a meaningful and intuitive manner.

The device is easy to install, can be easily moved to different vehicles and the user does not require training to operate the device. Once the device is started, the device can operate with minimal human intervention.

The user can have a single device mounted on a vehicle or deploy multiple devices deployed on their fleet to have access to an up-to-date insight of their roads. The data created from the incident detection can be used to automate the opening of work orders which helps streamline the process of resolving incidents.

A first aspect provided is a system for capturing digital images containing objects of interest along a first section of a roadway and for transmitting object data pertaining to the objects of interest over a communications network to a server, the server located remotely from the system, the system comprising: a device having: a camera for obtaining the digital images; at least one sensor for acquiring sensor data pertaining to the objects of interest contained in the digital images; and a memory and processor for executing image processing instructions for processing the digital images in order to generate the object data, the object data including the sensor information and the resultant data representing the objects of interest; and a mounting component for mounting the device to a vehicle, the vehicle for travelling along the roadway during operation of the camera; and a network interface for sending the object data over the communications network to the server during operation of the vehicle on the roadway.

A further aspect provided is a method for capturing digital images containing objects of interest along a first section of a roadway and for transmitting object data pertaining to the objects of interest over a communications network to a server, the server located remotely from the system, the method stored instructions for execution by a processor: obtaining the digital images by a camera mounted by a mounting component to a vehicle, the vehicle for travelling along the roadway during operation of the camera; acquiring sensor data pertaining to the objects of interest contained in the digital images; executing image processing instructions for processing the digital images in order to generate the object data, the object data including the sensor information and the resultant data representing the objects of interest; and sending the object data over the communications network to the server during operation of the vehicle on the roadway.

