Generalized structure tensor

In image analysis, the generalized structure tensor (GST) is an extension of the Cartesian structure tensor to curvilinear coordinates. It is mainly used to detect and to represent the "direction" parameters of curves, just as the Cartesian structure tensor detects and represents the direction in Cartesian coordinates. Curve families generated by pairs of locally orthogonal functions have been the best studied.
It is a widely known method in applications of image and video processing including computer vision, such as biometric identification by fingerprints, and studies of human tissue sections.
GST in 2D and locally orthogonal bases.
Let the term image represent a function
formula_1
where formula_2 are real variables and
formula_3, and formula_4, are real valued functions.
GST represents the direction along which the image formula_4 can undergo an infinitesimal translation with minimal (total least squares) error, along the "lines" fulfilling the following conditions:
1. The "lines" are ordinary lines in the curvilinear coordinate basis formula_6
which are curves in Cartesian coordinates as depicted by the equation above. The error is measured in the formula_8 sense and the minimality of the error refers thereby to L2 norm.
2. The functions formula_9 constitute a harmonic pair, i.e. they fulfill Cauchy–Riemann equations,
Accordingly, such curvilinear coordinates formula_6 are locally orthogonal.
Then GST consists in 
where formula_13 are errors of (infinitesimal) translation in the best direction (designated by the angle formula_14) and the worst direction (designated by formula_15). The function formula_16 is the window function defining the "outer scale" wherein the detection of formula_17 will be carried out, which can be omitted if it is already included in formula_4 or if formula_4 is the full image (rather than local). The matrix formula_20 is the identity matrix. Using the chain rule, it can be shown that the integration above can be implemented as convolutions in Cartesian coordinates applied to the ordinary structure tensor when formula_6 pair the real and imaginary parts of an analytic function formula_22, 
where formula_24. 
Examples of analytic functions include formula_25, as well as monomials formula_26, formula_27, where formula_28 is an arbitrary positive or negative integer. The monomials formula_29 are also referred to as Harmonic functions in Computer Vision, and Image Processing. 
Thereby, Cartesian Structure tensor is a special case of GST where formula_30, and formula_31, i.e. the harmonic function is simply formula_32. Thus by choosing a harmonic function formula_33, one can detect all curves that are linear combinations of its real and imaginary parts by convolutions on (rectangular) image grids only, even if formula_6 are non-Cartesian. Furthermore, the convolution computations can be done by using complex filters applied to the complex version of the structure tensor. Thus, GST implementations have frequently been done using complex version of the structure tensor, rather than using the (1,1) tensor.
Complex version of GST.
As there is a complex version of the ordinary [Structure tensor], there is also a complex version of the GST
which is identical to its cousin with the difference that formula_36 is a complex filter. It should be recalled that, the ordinary structure tensor formula_36 is a real filter, usually defined by a sampled and scaled Gaussian to delineate the neighborhood, also known as the outer scale. This simplicity is a reason for why GST implementations have predominantly used the complex version above. For curve families formula_6 defined by analytic functions formula_33, it can be shown that, the neighborhood defining function is complex valued, 
a so called symmetry derivative of a Gaussian. Thus, the orientation wise variation of the pattern to be looked for is directly incorporated into the neighborhood defining function, and the detection occurs in the space of the (ordinary) structure tensor.
Basic concept for its use in image processing and computer vision.
Efficient detection of formula_17 in images is possible by image processing for a pair formula_42, formula_43. Complex convolutions (or the corresponding matrix operations) and point-wise non-linear mappings are the basic computational elements of GST implementations. A total least square error estimation of formula_44 is then obtained along with the two errors, formula_45 and formula_46. In analogy with the Cartesian Structure tensor, the estimated angle is in double angle representation, i.e. formula_44 is delivered by computations, and can be used as a shape feature whereas formula_48 alone or in combination with 
formula_49 can be used as a quality (confidence, certainty) measure for the angle estimation.
Logarithmic spirals, including circles, can for instance be detected by (complex) convolutions and non-linear mappings. The spirals can be in gray (valued) images or in a binary image, i.e. locations of edge elements of the concerned patterns, such as contours of circles or spirals, must not be known or marked otherwise.
Generalized structure tensor can be used as an alternative to Hough transform in image processing and computer vision to detect patterns whose local orientations can be modelled, for example junction points. The main differences comprise:
Physical and mathemical interpretation.
The curvilinear coordinates of GST can explain physical processes applied to images. A well known pair of processes consist in rotation, and zooming. These are related to the coordinate transformation formula_50 and formula_51.
If an image formula_4 consists in iso-curves that can be explained by only $\xi$ i.e. its iso-curves consist in circles formula_53, where formula_54 is any real valued differentiable function defined on 1D, the image is invariant to rotations (around the origin).
Zooming (comprising unzooming) operation is modeled similarly. If the image has iso-curves that look like a "star" or bicycle spokes, i.e. formula_55 for some differentiable 1D function formula_33 then, the image formula_4 is invariant to scaling (w.r.t. the origin).
In combination,
formula_58
is invariant to a certain amount of rotation combined with scaling, where the amount is precised by the parameter formula_17.
Analogously, the Cartesian structure tensor is a representation of a translation too. Here the physical process consists in an ordinary translation of a certain amount along formula_60 combined with translation along formula_61,
where the amount is specified by the parameter formula_17. Evidently formula_17 here represents the direction of the line.
Generally, the estimated formula_17 represents the direction (in formula_6 coordinates) along which infinitesimal translations leave the image invariant, in practice least variant.
With every curvilinear coordinate basis pair, there is thus a pair of infinitesimal translators, a linear combination of which is a Differential operator. The latter are related to Lie algebra.
Miscellaneous.
"Image" in the context of the GST can mean both an ordinary image and an image neighborhood thereof (local image), depending on context. For example, a photograph is an image as is any neighborhood of it.