In recent years, a vehicle-mounted camera referred to as a video recording type drive recorder has been mounted on an automobile. For example, the vehicle-mounted camera that captures an image of a road condition in front of a vehicle and records the image is mounted on the inner side of a front window in a case where the camera is not adaptable to outdoor use. In this manner, the vehicle-mounted camera mounted on the inner side of the front window captures an image in which a wiper blade intersects a screen in a case where a wiper device is operated.
For example, when the wiper blade is captured as a visual field obstacle that erases a white line to be captured in a case where the white line in the road is detected by image processing and traveling support enabling traveling along a traffic lane is performed, it may be difficult to detect the white line. In a case where an image captured by a guard or the like who goes round, a vehicle, a person, or the like to be noticed is obstructed due to the motion of the wiper blade, which leads to a concern for a reduction in concentration.
Even in a case of a monitoring camera installed outdoors, snow passing a screen during snowfall acts as a visual field obstacle that erases a portion of scenery to be captured, and thus there is a problem in that it is not possible to clearly show a necessary portion of an image. Regarding the snow during snowfall, image data equivalent to one frame is constructed by performing comparison between gray scales of respective pixels of images in a plurality of frames prior to a target frame by using the images of the plurality of frames traced back frontward in a time axis from the target frame to be processed, and collecting pixels ranging from a pixel having the highest grayscale to a pixel having a grayscale at a predetermined position in order (for example, see PTL 1).
According to PTL 1 described above, in an image obtained by capturing a visual field obstacle, a pixel having a grayscale at a predetermined position in order in pixels of a plurality of frames is used as a pixel constituting the visual field obstacle. Thereby, the pixel of the visual field obstacle is not used, and thus it is possible to remove the visual field obstacle. Accordingly, it is considered that image processing disclosed in PTL 1 described above is applied to the reflection of the wiper blade in the vehicle-mounted camera.
However, in a case of the vehicle-mounted camera, scenery, another vehicle, or the like which relatively moves by the traveling of the vehicle serves as a moving subject. The image processing disclosed in PTL 1 described above being performed on an image obtained by the vehicle-mounted camera also affects a moving subject such as scenery or another vehicle, and thus there is a problem of afterimage tear in the moving subject to be captured or partial disappearance.
When a monitoring camera or the like capturing an image of an outdoor subject performs image capture during snowfall, a captured image in which a subject to be monitored is unclear due to a visual field blocked by snow particles is obtained. Consequently, for example, there has been known a technique for removing snowfall noise by performing median filtering in a time-axis direction on a plurality of temporally continuous captured images to thereby sharpen the images (see PTL 2). This technique focuses on the fact that there is a low possibility of pixels having the same coordinates in the plurality of temporally continuous captured images being pixels representing snow particles, and pixel values representing snow particles are excluded by median filtering in which pixels values extracted from the plurality of temporally continuous captured images are sorted in descending order to thereby acquire the median value. Therefore, it is possible to remove the snowfall noise.
There is a technique for detecting an outdoor weather condition such as snowfall regardless of a person's determination, and there has been a known weather display system which is provided with a plurality of sensors for obtaining weather information such as snowfall, rainfall, or the velocity of wind, processes the weather information obtained by the sensors, and displays the obtained weather information on weather display means provided indoors (see PTL 3).
Incidentally, there are some weather conditions, other than the above-described snowfall, in which noise may be generated during image capture performed outdoors. However, for example, in a case of image capture during rainfall, a noise generation condition in a captured image is different from that during snowfall. Accordingly, even when the image processing for removing snowfall noise which is disclosed in PTL 1 described above is applied as it is, it may be difficult to obtain the same noise removal effect as in the case of snowfall. Therefore, in order to more appropriately remove noise caused by a weather condition different from the captured image, it is preferable to apply different image processing depending on a weather condition during image capture (for example, to change a parameter related to the image processing.
Consequently, it is considered that image processing (noise removal) which is suited to each weather condition is performed on the basis of, for example, detection results of a plurality of sensors for detecting weather conditions. However, in this case, it is necessary to provide various sensors for each location where a camera is installed, and thus there is a problem in that time and labor are required for the installation of the sensors and a facility cost is also increased.
Further, there has been known a video camera including an optical or electronic image blur correction mechanism in order to suppress the generation of an uncomfortable video in which a video shown on a screen of a monitor is blurred during watching, for example, in a case where a photographer captures a moving image by using a handheld video camera. In the electronic image blur correction mechanism, image blur is solved by canceling a deviation of a subject by comparing a plurality of captured images with each other.
In order to prevent an image having been subjected to image blur correction from deviating from the screen, an image capture range of a light receiving element may be set to be wider than a range of an image shown on the screen. However, there is a problem in that the cost of the camera is increased due to the use of the high-resolution light receiving element or the size of the camera is increased due to the mounting of the large-size light receiving element. On the other hand, a security guard may perform monitoring by mounting a camera on the security guard's shoulder or head. In this case, there is a demand for promoting a reduction in the size of the camera and suppressing a reduction in resolution. In this case, it is considered that the resolution of a light receiving element and the resolution of an image shown on a screen are made to be substantially the same as each other by using the light receiving element having a small size.
However, in a case where the resolution of the light receiving element and the resolution of the image shown on the screen are made to be substantially the same as each other, the generation of an image in which a deviation of a subject is canceled by image blur correction may result in the occurrence of a deviation of the image with respect to the screen and the generation of a blank region which is not shown on the screen. Thereby, when a boundary between the image and the blank region becomes clear, the contour of the image acts as an edge, and thus there is a problem in that the edge is blurred and flickers in association with image blur. On the other hand, there has been a technique for preventing the blur of an edge from being shown by hiding a blank region together with the edge by using a mask with fixed concentration (see, for example, PTL 4).
There has been a technique for extracting a parallel movement vector and a rotation angle vector that indicate the motion of a camera due to image blur, separating an intended motion component and an unintended motion component from each other, performing filtering on the components, confirming whether or not the amount of correction performed on the image blur exceeds an allowable threshold value, and performing image blur correction so that the amount of correction falls within an allowable range when the amount of correction does not exceed the threshold value (see, for example, PTL 5).
However, in a case where image blur correction is performed by the shift of a captured image such that a subject is positioned in the middle portion of a screen, the mask is displayed in a peripheral portion of the screen like a frame. Accordingly, a display region of the correction image (captured image shown on the screen without being subjected to masking) becomes smaller, and thus there is a problem in that image display efficiency is decreased with respect to a case where an image is displayed on the entire screen. Particularly, in a case where blur in a rotation direction of an image is corrected, a region of a mask for hiding an edge is enlarged, and display efficiency is further decreased.
A blank region including the edge is compensated for by an image of a former frame, and thus it is possible to suppress the enlargement of a non-display region due to the mask for hiding the edge. However, in this case, when a moving image corrected in this manner is reproduced, the image of the former frame which is shown in the blank region is shown in a hardened state due to the occurrence of a delay with respect to the correction image of the present frame, which may result in a feeling of strangeness at the time of viewing the entire screen.
It is possible to cope with a problem in that display efficiency is decreased due to an image region of the screen, by enlarging the image region in the entire screen. However, in a case where an image to be displayed is displayed on the entire screen in an enlarged manner, a basic image is only enlarged. Thus, there is a concern that the definition of pixels of a correction image having been subjected to image blur correction may be decreased. Particularly, for example, there is a scene captured during walking at the time of recording a video for monitoring, which is unsuitable for a case where the definition of an image is required in order to confirm details from the recorded moving image, and an image of a portion processed as a blank region is not kept as a record.
When the amount of hand shake correction (a parallel movement vector or a rotation angle vector) is restricted so as not to exceed an allowable threshold value, image blur correction becomes insufficient with respect to handshake exceeding the threshold value. In this case, a blurred image is shown, which results in an uncomfortable video.
An object of the present disclosure is to be capable of removing noise caused by disturbance during image capture and to suppress afterimage tear in a moving subject other than the noise shown in an image, or partial disappearance.
More specifically, an object of the present disclosure is to be capable of removing a visual field obstacle appearing as disturbance and to suppress afterimage tear in a moving subject other than the visual field obstacle shown in an image, or partial disappearance.
More specifically, an object of the present disclosure is to be capable of determining a weather condition by a simple configuration during image capture performed outdoors and to be capable of acquiring an image from which noise is appropriately removed by image processing based on the determined weather condition.
Further, more specifically, an object of the present disclosure is to hold the resolution of an image corrected by electronic image blur correction and to prevent a video shown on a screen of a monitor from becoming unnatural.