FIELD

The present invention relates generally to image processing and more particularly to analysis of photographed images.

BACKGROUND

Cameras today, even those with good “auto mode” functions, still rely on the camera user to assess a picture's quality, either at the tune of taking the picture, or at a later stage. Certain software, such as Photoshop, offers tools to enhance a picture to an extent. Many users simply take more than one picture, say 5-6 pictures, so they will be able to choose one good one.

Conventional technology constituting background to certain embodiments of the present invention is described in the following publications inter alia:

a. US 20130155474 in which a user of the mobile device is provided with feedback prior to the capturing of at least one image of a paper document which provides the user with at least one instruction for adjusting at least one measured parameter. Parameter values may be combined into a group threshold or total overall quality score, such that the combination of all of the measured parameter values produces an overall quality score which exceeds a defined threshold value before the image can be captured by the camera. According to this publication, “The user can be provided detailed information to assist the user in taking a better quality image of the document. For example, the blurriness may have been the result of motion blur caused by the user moving the camera while taking the image. The test result messages can suggest that the user hold the camera steadier when retaking the image.”

b. WO 2006040761/US20070195174 which describe a system in which the interface module further enables to define the scene dynamics of the captured image and attributes of the captured image scene dynamics include image motion speed and motion speed of the subjects and the interface module further enables setting the captured image attributes relative weight for the compution of the total image grade.

c. US20090278958 which describes inter alia that “The scoring of a current base image may be based on scores which have been given to previously captured base images. In such a manner, redundant calculation may be avoided.”

d. Other systems such as the following patent documents: U.S. Pat. No. 7,362,354; WO2003069559; U.S. Pat. No. 7,688,379; US20090263028; US20060017820; US20060056835; U.S. Pat. No. 8,125,557; WO2011148212; U.S. Pat. No. 7,920,727; US20130155474; U.S. Pat. No. 8,040,382.

The disclosures of all publications and patent documents mentioned in the specification, and of the publications and patent documents cited therein directly or indirectly, are hereby incorporated by reference. Materiality of such publications and patent documents to patentability is not conceded

SUMMARY

Certain embodiments seek to utilize camera hardware such as accelerometers, gyros and autofocus lenses, inter alia, to evaluate pictures taken in real time, and actively assist in obtaining the best picture given the circumstances at hand.

Typically, each available sensor, e.g. accelerometers, gyros, sends its raw data to a processing component400that outputs a quality indicator e.g. as illustrated inFIG. 1. andFIG. 3. Data may also be sent to Device/surrounding objects 4D position computation component230for 4D positioning computation. Raw data may be combined with quality indication from other sensors, such as the camera/lens module200, that can send its own quality indications. For example, some lens modules can estimate the picture focus quality, as it is a part of their auto focus functionality. Some camera/lens modules may have dual cameras or more for achieving 3D data, sophisticated focusing methods like Lytro light field camera, or their own hardware sensors like Google Glass. All this available data can be sent to component400and Device/surrounding objects 4D position computation component230for use of their functions. Various quality indicators, depending on device capabilities and implementation, may be employed.

Prior art has used certain independent quality indicators, each giving a quality of one particular aspect of a picture, for a given picture. Some try to quantify the quality by means of a total quality indicator. Typically this is done as follows:

QItotal=∑i=1n⁢QI⁡(t)i
Where n is the number of quality indicators (QI), and t is the time of test.

Some implementations use a weight function in computation, but the weight is either constant or can be change by the user manually.

QItotal=∑i=1n⁢QI⁡(t)i*wi⁡(t)

However, in this invention, the weight of one indicator will take into account data from other quality indicator/s e.g. their quality indicator value, weight, confidence level (explained forthwith) in them and their previous value, weight and confidence level. For example, if both device shake quality indicator and/or camera focus quality indicator indicate a poor quality, disregard use of aesthetic quality indicator in total computation, even if user indicated that he would like to give it a high priority. This will prevent the device from taking a picture when the picture is blurry, as is the case in prior art. Once shake is low and focus is optimal, aesthetic QI may be used in total quality computations. One quality indicator's importance or relative weight within the total quality indicator may change over time, if some other quality indicator indicates that the first quality indicator becomes less relevant in quantifying the picture quality.

Furthermore, since each QI algorithm has its own assumptions, and each sensor may have measurement errors, the quality indication (QI) value may therefore not be reliable. Also, a QI value that fluctuates widely may not be reliable, and in certain cases, re-assessment of the quality indicator may be called for. Thus, a “confidence level” in the QI is obtained. These indicators may be relayed to a processing component that analyses the current relevance of each indicator, and with suitable configuration, provides a “total” quality value.

This total quality indicator and its detailed list of quality indicators can be used to give the user accurate feedback on the picture quality, whether and how it fits user preferences (configured in151, used in total QI computation), and provide detailed suggestions on how to take a better picture, if possible.

Suggestions may be provided as an oral message, typically pre-recorded, as an icon on a screen, or as text. Suggestions may use the recorded images as a template for the suggestions. This may include marking over exposed areas in red, or marking a problematic element with a circle.

The quality indicators' values and total quality indicator can be shown to the user in real time.

Certain embodiments of the present invention include a method which comprises combining data from multiple sensors and multiple quality indicators, using available GPU and CPU, to compute, in real time, each quality indicator, and to form a combined total indicator to be shown to the user before and after he has pressed the shutter button. The user may get, or choose to get, suggestions from the application on how to improve the next shot, using the quality indicators and their correlation.

Further indication to the user can be made if the image is below a certain total quality indicator threshold.

According to one embodiment, quality indications are used to control the lens module200. For example, focus distance reported may be below hyper focal distance or very short and accelerometer reports movement in direction perpendicular to focus plan. This can be used to find a way to shorten the shutter speed (increase ISO or change aperture for example). Another example may be to change the focus point and possibly focus distance if the object being tracked has moved.

In another embodiment, when the user presses the “shutter” button to take the picture (component105), the device will wait for the total quality indicator to reach a minimum value/threshold, before taking a full resolution picture as shown inFIG. 1. Ideally, this time will be close to the shutter release time. A timeout mechanism can ensure a full resolution picture will be taken, even if the threshold is not reached in a given time after the shutter is released.

A list of suggestions to the users may be for instance “blurred image due to camera shake”, or “blurred subject due to subject movement”, or “Image dynamic range is beyond the sensor dynamic abilities—choose area of interest, or take a high dynamic range (HDR) shot”.

Embodiments include but are not limited to:

Embodiment 1. A method for computing the photo quality of a captured image in a device image acquisition system, said method comprising:computing at least one total quality indicator based on at least two quality indicators computed from said captured image and its previous image frames,to determine whether the photo quality is acceptable; andproviding a corresponding detailed photo quality feedback.
Embodiment 2. A system according to embodiment 1, where automatically activating the capturing apparatus, once a logical criterion based on said quality indicators is met.
Embodiment 3: A system, according to embodiment 1 wherea message provider is operative where at least one appropriate suggestion from a pre-stored table of suggestions on how the digital image can be improved further,where said message is presented as a suggestion to the user.
Embodiment 4: A system, according to embodiment 1 wherepictures are taken and saved at each frame, but only pictures with a said logical criterion are marked as so.
Embodiment 5: A system, according to embodiment 1 where the stream of frames is saved as a video and;quality indicators, user preferences, confidence level, and quality importance are used to quantify video quality.
Embodiment 6: A system, according to embodiment 5, used to give a user suggestions on improvement of the video.
Embodiment 7: A system, according to embodiment 1, where computed QI and their confidence levels are used to change parameters in the lens/sensor module to achieve better total QI.
Embodiment 8: A system and method as in embodiment 1 used for automatic image capturing of at least one picture once a configured total quality indicator meets a minimum threshold, even without the user pressing a shutter button.
Embodiment 9. A system using blur type detection for alerting the user of object movement in a region.
Embodiment 10. A system using blur type detection for alerting the user he is moving too fast in a vehicle to achieve a non-blurred image.
Embodiment 11. A system and method described herein for evaluating video quality.
Embodiment 12. A system with device shake recognition, where a device movement gesture can be used for shutter press indication.
Embodiment 13. A system with device shake recognition, where a device movement gesture can be used for delete picture indication.

Also provided, excluding signals, is a computer program comprising computer program code means for performing any of the methods shown and described herein when said program is run on a computer; and a computer program product, comprising a typically non-transitory computer-usable or-readable medium e.g. non-transitory computer-usable or-readable storage medium, typically tangible, having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement any or all of the methods shown and described herein. It is appreciated that any or all of the computational steps shown and described herein may be computer-implemented. The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes or by a general purpose computer specially configured for the desired purpose by a computer program stored in a typically non-transitory computer readable storage medium. The term “non-transitory” is used herein to exclude transitory, propagating signals or waves, but to otherwise include any volatile or non-volatile computer memory technology suitable to the application.

Any suitable processor, display and input means may be used to process, display e.g. on a computer screen or other computer output device, store, and accept information such as information used by or generated by any of the methods and apparatus shown and described herein; the above processor, display and input means including computer programs, in accordance with some or all of the embodiments of the present invention. Any or all functionalities of the invention shown and described herein, such as but not limited to steps of flowcharts, may be performed by a conventional personal computer processor, workstation or other programmable device or computer or electronic computing device or processor, either general-purpose or specifically constructed, used for processing; a computer display screen and/or printer and/or speaker for displaying; machine-readable memory such as optical disks, CDROMs, DVDs, BluRays, magnetic-optical discs or other discs; RAMs, ROMs, EPROMs, EEPROMs, magnetic or optical or other cards, for storing, and keyboard or mouse for accepting. The term “process” as used above is intended to include any type of computation or manipulation or transformation of data represented as physical, e.g. electronic, phenomena which may occur or reside e.g. within registers and/or memories of a computer or processor. The term processor includes a single processing unit or a plurality of distributed or remote such units.

The above devices may communicate via any conventional wired or wireless digital communication means, e.g. via a wired or cellular telephone network or a computer network such as the Internet.

The apparatus of the present invention may include, according to certain embodiments of the invention, machine readable memory containing or otherwise storing a program of instructions which, when executed by the machine, implements some or all of the apparatus, methods, features and functionalities of the invention shown and described herein. Alternatively or in addition, the apparatus of the present invention may include, according to certain embodiments of the invention, a program as above which may be written in any conventional programming language, and optionally a machine for executing the program such as but not limited to a general purpose computer which may optionally be configured or activated in accordance with the teachings of the present invention. Any of the teachings incorporated herein may whereever suitable operate on signals representative of physical objects or substances.

The embodiments referred to above, and other embodiments, are described in detail in the next section.

Any trademark occurring in the text or drawings is the property of its owner and occurs herein merely to explain or illustrate one example of how an embodiment of the invention may be implemented.

Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions, utilizing terms such as, “processing”, “computing”, “estimating”, “selecting”, “ranking”, “grading”, “calculating”, “determining”, “generating”, “reassessing”, “classifying”, “generating”, “producing”, “stereo-matching”, “registering”, “detecting”, “associating”, “superimposing”, “obtaining” or the like, refer to the action and/or processes of a computer or computing system, or processor or similar electronic computing device, that manipulate and/or transform data represented as physical, such as electronic, quantities within the computing system's registers and/or memories, into other data similarly represented as physical quantities within the computing system's memories, registers or other such information storage, transmission or display devices. The term “computer” should be broadly construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, personal computers, servers, computing system, communication devices, processors (e.g. digital signal processor (DSP), microcontrollers, field programmable gate array (FPGA), application specific integrated circuit (ASIC), etc.) and other electronic computing devices.

The present invention may be described, merely for clarity, in terms of terminology specific to particular programming languages, operating systems, browsers, system versions, individual products, and the like. It will be appreciated that this terminology is intended to convey general principles of operation clearly and briefly, by way of example, and is not intended to limit the scope of the invention to any particular programming language, operating system, browser, system version, or individual product.

Elements separately listed herein need not be distinct components and alternatively may be the same structure.

Any suitable input device, such as but not limited to a sensor, may be used to generate or otherwise provide information received by the apparatus and methods shown and described herein. Any suitable output device or display may be used to display or output information generated by the apparatus and methods shown and described herein. Any suitable processor may be employed to compute or generate information as described herein e.g. by providing one or more modules in the processor to perform functionalities described herein. Any suitable computerized data storage e.g. computer memory may be used to store information received by or generated by the systems shown and described herein. Functionalities shown and described herein may be divided between a server computer and a plurality of client computers. These or any other computerized components shown and described herein may communicate between themselves via a suitable computer network.

