We introduce one-shot texture segmentation: the task of segmenting an input image containing multiple textures given a patch of a reference texture. This task is designed to turn the problem of texture-based perceptual grouping into an objective benchmark. We show that it is straight-forward to generate large synthetic data sets for this task from a relatively small number of natural textures. In particular, this task can be cast as a self-supervised problem thereby alleviating the need for massive amounts of manually annotated data necessary for traditional segmentation tasks. In this paper we introduce and study two concrete data sets: a dense collage of textures (CollTex) and a cluttered texturized Omniglot data set. We show that a baseline model trained on these synthesized data is able to generalize to natural images and videos without further fine-tuning, suggesting that the learned image representations are useful for higher-level vision tasks.
Textures are well-suited for studying human perception as they are diverse, non-trivial stimuli. There is evidence that certain areas of the human brain are building texture representations to perform low-level visual inference @cite_32 @cite_7 . Within the substantial body of work on texture modelling @cite_43 , experimental settings very similar to ours have been introduced. For instance, in @cite_11 @cite_33 human subjects have to perform texture segmentation of images consisting of multiple textures, relying on local image statistics in a cluttered scene. In @cite_1 a biologically motivated model of texture perception has been used to recognize objects. Finally, textures have also been used to study the difference between human perception and computational models of vision @cite_38 .
@cite_38 @cite_33 @cite_7 @cite_1 @cite_32 @cite_43 @cite_11
2056908494 2144982973 2138585914 2565377392 1958479627
A real-time visual processing theory is used to analyze and explain a wide variety of perceptual grouping and segmentation phenomena, including the grouping of textured images, randomly defined images, and images built up from periodic scenic elements. The theory explains how “local” feature processing and “emergent” features work together to segment a scene, how segmentations may arise across image regions that do not contain any luminance differences, how segmentations may override local image properties in favor of global statistical factors, and why segmentations that powerfully influence object recognition may be barely visible or totally invisible. Network interactions within a Boundary Contour (BC) System, a Feature Contour (FC) System, and an Object Recognition (OR) System are used to explain these phenomena. The BC System is defined by a hierarchy of orientationally tuned interactions, which can be divided into two successive subsystems called the OC filter and the CC loop. The OC filter contains two successive stages of oriented receptive fields which are sensitive to different properties of image contrasts. The OC filter generates inputs to the CC loop, which contains successive stages of spatially short-range competitive interactions and spatially long-range cooperative interactions. Feedback between the competitive and cooperative stages synthesizes a global context-sensitive segmentation from among the many possible groupings of local featural elements. The properties of the BC System provide a unified explanation of several ostensibly different Gestalt rules. The BC System also suggests explanations and predictions concerning the architecture of the striate and prestriate visual cortices. The BC System embodies new ideas concerning the founda-tions of geometry, on-line statistical decision theory, and the resolution of uncertainty in quan-tum measurement systems. Computer simulations establish the formal competence of the BC System as a perceptual grouping system. The properties of the BC System are compared with probabilistic and artificial intelligence models of segmentation. The total network suggests a new approach to the design of computer vision systems, and promises to provide a universal set of rules for perceptual grouping of scenic edges, textures, and smoothly shaded regions. We introduce a new general framework for the recognition of complex visual scenes, which is motivated by biology: We describe a hierarchical system that closely follows the organization of visual cortex and builds an increasingly complex and invariant feature representation by alternating between a template matching and a maximum pooling operation. We demonstrate the strength of the approach on a range of recognition tasks: From invariant single object recognition in clutter to multiclass categorization problems and complex scene understanding tasks that rely on the recognition of both shape-based as well as texture-based objects. Given the biological constraints that the system had to satisfy, the approach performs surprisingly well: It has the capability of learning from only a few training examples and competes with state-of-the-art systems. We also discuss the existence of a universal, redundant dictionary of features that could handle the recognition of most object categories. In addition to its relevance for computer vision, the success of this approach suggests a plausibility proof for a class of feedforward models of object recognition in cortex Using the visual system as a model, we recently showed that the efficient coding principle accounted for the allocation of computational resources in central sensory processing: when sampling an image is the main limitation, resources are devoted to compute the statistical features that are the most variable, and therefore the most informative (eLife 2014;3:e03722. DOI: 10.7554 eLife.03722 , 2014). Building on these results, we use single-unit recordings in the macaque monkey to determine where these computations—sensitivity to specific multipoint correlations—occur. We find that these computations take place in visual area V2, primarily in its supragranular layers. The demonstration that V2 neurons are sensitive to the multipoint correlations that are informative about natural images provides a common computational underpinning for diverse but well-recognized aspects of neural processing in V2, including its sensitivity to corners, junctions, illusory contours, figure ground, and ‘naturalness.’ DOI: http: dx.doi.org 10.7554 eLife.06604.001 Visual textures are a class of stimuli with properties that make them well suited for addressing general questions about visual function at the levels of behavior and neural mechanism. They have structure across multiple spatial scales, they put the focus on the inferential nature of visual processing, and they help bridge the gap between stimuli that are analytically convenient and the complex, naturalistic stimuli that have the greatest biological relevance. Key questions that are well suited for analysis via visual textures include the nature and structure of perceptual spaces, modulation of early visual processing by task, and the transformation of sensory stimuli into patterns of population activity that are relevant to perception. Local image statistics are important for visual analysis of textures, surfaces, and form. There are many kinds of local statistics, including those that capture luminance distributions, spatial contrast, oriented segments, and corners. While sensitivity to each of these kinds of statistics have been well-studied, much less is known about visual processing when multiple kinds of statistics are relevant, in large part because the dimensionality of the problem is high and different kinds of statistics interact. To approach this problem, we focused on binary images on a square lattice – a reduced set of stimuli which nevertheless taps many kinds of local statistics. In this 10-parameter space, we determined psychophysical thresholds to each kind of statistic (16 observers) and all of their pairwise combinations (4 observers). Sensitivities and isodiscrimination contours were consistent across observers. Isodiscrimination contours were elliptical, implying a quadratic interaction rule, which in turn determined ellipsoidal isodiscrimination surfaces in the full 10-dimensional space, and made predictions for sensitivities to complex combinations of statistics. These predictions, including the prediction of a combination of statistics that was metameric to random, were verified experimentally. Finally, check size had only a mild effect on sensitivities over the range from 2.8 to 14 min, but sensitivities to second- and higher-order statistics was substantially lower at 1.4 min. In sum, local image statistics form a perceptual space that is highly stereotyped across observers, in which different kinds of statistics interact according to simple rules.