When taking pictures a user of an image capture device such as a camera for example may or may not put care into framing a subject of the image accurately. Hence, in some situations pictures are taken quickly and are generally referred to as ‘snaps’ or ‘snapshots’, whilst in others a long time can be spent carefully framing a subject in order to obtain a desired image composition.
Differences in picture taking behaviour can be the result of a combination of factors. For example, some camera users always take pictures very quickly, perhaps due to lack of training or through not wishing to spend a long time “behind the camera”, whilst other users take time and trouble when the situation permits or a good result is critical, but on other occasions are forced to take a quick snap.
A further distinction in picture taking behaviour relates to the subject itself. In the case of moving objects, a photographer may wait for a desired moment to capture an image. Some examples include: a subject scoring a goal, eye-contact with the user, and the sun emerging from behind a cloud and so forth. In these situations, attention is more focused on capturing a desired moment in time, rather than ensuring that a subject of the captured image is well framed.
Current cameras do not recognize the picture taking behaviour of a user, other than by the direct operation of the camera controls. As such, the image processing functionality of the camera is generally determined only by its settings and the content of the captured image. This precludes the possibility of making image processing responsive to the way the camera is being used.
A particular problem with this rigid approach to image processing is that processing steps such as autocropping are either ‘on’ or ‘off’, as generally set by user choice —the term “autocropping” refers to algorithms which automatically generate one or more crops of a photographic image based on an algorithmic assessment of likely areas of interest, or salient areas, within a captured image. The autocrop location can be selected to include the areas of interest, remove distractions or large boring areas, and produce a pleasing composition for example.
A number of autocropping approaches are known. U.S. patent application Ser. No. U.S. Pat. No. 5,978,519 is an example. It assumes that the “background” region of a captured image has relatively low intensity variance compared with the foreground. A threshold is selected based on the variance of blocks, and a bounding box of the subject is then used for cropping.
A more recent approach may be found in United Kingdom Patent Application No. GB0031423.7 which uses a saliency map determined by region segmentation and colour ‘unusual-ness’ to identify both subject areas and distraction areas. Multiple crops are suggested based on alternative combinations of subjects, in each case excluding distractions and selecting locally optimal crop boundaries.
European Patent Application No. EP1158464A1 also uses a saliency map derived from region segmentation.
Some of the above algorithms can have hints or constraints applied by an external source (typically a person interacting with the algorithm). These provide the algorithm with guidance to ensure that some areas are always preserved, but do not provide an automatic determination of when autocropping may be appropriate.