FIELD

The present application relates generally to computers and computer applications, and more particularly to image visualizations and machine learning to automatically predict image processing procedures that generate desired difference visualization in images.

BACKGROUND

In many processes, insight can be obtained by comparing images that represent the same physical environment or the same phenomena, taken or created under various circumstances. For example, visualizing the uncertainty associated with the output of computer applications is critical in many industries. Visualizing uncertainty can be accomplished through the modification (or annotation) of the images generated by uncertainty-agnostic applications. This principle applies in a situation where the effect of the uncertainty on the images generated by the applications is not predictable. In this case, uncertainty visualization can be achieved by creating an image that results from the composition of a set of realizations, i.e., the images of the result generated by the application for varying values of (an) uncertain parameter(s) or (an) uncertain input data element(s).

As another example, evolution through time of physical phenomena is important in industries such as agriculture, where the variation in time of size and color of crop indicates information about its health. One can then envision a computer application that performs the comparison of images of the crop taken at various times. This comparison can in turn lead to the building of new images, composed from the initial ones that highlight the differences.

As yet another example, comparing a manufactured product with its specification allows identifying manufacturing defects, for example, impurity detection, geometry feature (straightness, curvature, parallelism) verification, abnormal texture area detection and color, brightness feature verification. This can be achieved by comparing and composing images of the “as designed” product—possibly created using a 3D virtual model—and images of the final “as built” product. Still yet, in medical imaging, medical imagery informs doctors of the evolution of diseases through the comparison of, for example, an anomaly such as a tumor through time.

While existing computer vision, histogram methods, and other image processing approaches may analyze and highlight differences among images, no systematic methods exist currently that capture the description of the composition, the image comparison procedure itself. A composition is therefore not easily reproducible, as it cannot be documented, it cannot be taught, and it cannot be automatically re-applied from one problem to the next.

BRIEF SUMMARY

An automatic method and system of determining an image composition procedure that generates a new image visualization based on aggregations and variations of input images are provided. The method may be executed by a hardware processor, and may include receiving a set of input images. The method may also include extracting visual features from the input images. The method may further include receiving context associated with input images. The method may include also learning based on the extracted visual features and the context associated with the input images, a composition procedure comprising a set of image operations to apply on the set of input images. The method may further include determining one or more image operations in the composition procedure to present to a user. The method may also include generating a difference visualization image associated with the input images by executing the one or more image operations.

An automatic system of determining an image composition procedure that generates a new image visualization based on aggregations and variations of input images, in one aspect, may include at least one hardware processor and a display device coupled to the at least one hardware processor. The at least one hardware processor may be operable to receiving a set of input images. The at least one hardware processor may be operable to extract visual features from the input images. The at least one hardware processor may be operable to receive context associated with input images. The at least one hardware processor may be operable to learn based on the extracted visual features and the context associated with the input images, a composition procedure comprising a set of image operations to apply on the set of input images. The at least one hardware processor may be operable to determine one or more image operations in the composition procedure to present to a user on the display device. The at least one hardware processor may be operable to generate a difference visualization image associated with the input images by executing the one or more image operations.

A computer readable storage medium storing a program of instructions executable by a machine to perform one or more methods described herein also may be provided.

Further features as well as the structure and operation of various embodiments are described in detail below with reference to the accompanying drawings. In the drawings, like reference numbers indicate identical or functionally similar elements.

