This paper presents a "learning to learn" approach to figure-ground image segmentation. By exploring webly-abundant images of specific visual effects, our method can effectively learn the visual-effect internal representations in an unsupervised manner and uses this knowledge to differentiate the figure from the ground in an image. Specifically, we formulate the meta-learning process as a compositional image editing task that learns to imitate a certain visual effect and derive the corresponding internal representation. Such a generative process can help instantiate the underlying figure-ground notion and enables the system to accomplish the intended image segmentation. Whereas existing generative methods are mostly tailored to image synthesis or style transfer, our approach offers a flexible learning mechanism to model a general concept of figure-ground segmentation from unorganized images that have no explicit pixel-level annotations. We validate our approach via extensive experiments on six datasets to demonstrate that the proposed model can be end-to-end trained without ground-truth pixel labeling yet outperforms the existing methods of unsupervised segmentation tasks.
To reduce the burden of @math , Denton al @cite_34 use a pyramid structure and Karras al @cite_28 consider a progressive training methodology. Both of them divide the task into smaller sequential steps. In our case, we alleviate the burden of @math by incorporating some well-defined image processing operations into the network model, , converting background color into grayscale to simulate the visual effect of color selectivo , or blurring the background to create the Bokeh effect.
@cite_28 @cite_34
2766527293 2951523806
Conventional unsupervised image segmentation methods use color and geometric information and apply clustering algorithms over pixels. They preserve object boundaries well but often suffer from over-segmentation due to noise and artifacts in the images. In this paper, we contribute on a preprocessing step for image smoothing, which alleviates the burden of conventional unsupervised image segmentation and enhance their performance. Our approach relies on a convolutional autoencoder (CAE) with the total variation loss (TVL) for unsupervised learning. We show that, after our CAE-TVL preprocessing step, the over-segmentation effect is significantly reduced using the same unsupervised image segmentation methods. We evaluate our approach using the BSDS500 image segmentation benchmark dataset and show the performance enhancement introduced by our approach in terms of both increased segmentation accuracy and reduced computation time. We examine the robustness of the trained CAE and show that it is directly applicable to other natural scene images. We present an image processing algorithm for automatic detection of cortex layers and cells from optical microscopy images for Nissl-stained mouse brain. For every layer of cortex we automatically detect a shape and localization of following cortex cells type: neurons of molecular layer, pyramidal neurons, stellate neurons, and astrocytes. The algorithm includes the steps of: preprocessing, neurons and astrocytes localization, neurons classification, refined cortex layer detection, neurons reclassification. For preprocessing we use converting to gray image, Gaussian blurring, converting to black-white image after background removing, and rough estimate of layers. We use morphological operations with variation radius of structure element for neurons localization and neurons classification. In video see-through augmented reality (AR), virtual objects are overlaid over digital video images. One particular problem of this image mixing process is that the visual appearance of the computer graphics differs strongly from the real background image. The reason for this is that typical AR systems use fast but simple real-time rendering techniques for displaying virtual objects. In this paper, methods for reducing the impact of three effects which make virtual and real objects easily distinguishable are presented. The first effect is camera image noise, which is contained in the data delivered by the image sensor used for capturing the real scene. The second effect considered is edge aliasing, which makes distinguishing virtual objects from real objects simple. Finally, we consider motion blur, which is caused by the temporal integration of color intensities in the image sensor during fast movements of the camera or observed objects. In this paper, we present a system for generating a realistic simulation of image noise based on a new camera calibration step. Additionally, a rendering algorithm is introduced, which performs a smooth blending between the camera image and virtual objects at their boundary in order to reduce aliasing. Lastly, a rendering method is presented, which produces motion blur according to the current camera movement. The implementation of the new rendering techniques utilizes the programmability of modern graphics processing units (GPUs) and delivers real-time frame rates. Deep ConvNets have been shown to be effective for the task of human pose estimation from single images. However, several challenging issues arise in the video-based case such as self-occlusion, motion blur, and uncommon poses with few or no examples in the training data. Temporal information can provide additional cues about the location of body joints and help to alleviate these issues. In this paper, we propose a deep structured model to estimate a sequence of human poses in unconstrained videos. This model can be efficiently trained in an end-to-end manner and is capable of representing the appearance of body joints and their spatio-temporal relationships simultaneously. Domain knowledge about the human body is explicitly incorporated into the network providing effective priors to regularize the skelet al structure and to enforce temporal consistency. The proposed end-to-end architecture is evaluated on two widely used benchmarks for video-based pose estimation (Penn Action and JHMDB datasets). Our approach outperforms several state-of-the-art methods. State-of-the-art semantic image segmentation methods are mostly based on training deep convolutional neural networks (CNNs). In this work, we proffer to improve semantic segmentation with the use of contextual information. In particular, we explore patch-patch' context and patch-background' context in deep CNNs. We formulate deep structured models by combining CNNs and Conditional Random Fields (CRFs) for learning the patch-patch context between image regions. Specifically, we formulate CNN-based pairwise potential functions to capture semantic correlations between neighboring patches. Efficient piecewise training of the proposed deep structured model is then applied in order to avoid repeated expensive CRF inference during the course of back propagation. For capturing the patch-background context, we show that a network design with traditional multi-scale image inputs and sliding pyramid pooling is very effective for improving performance. We perform comprehensive evaluation of the proposed method. We achieve new state-of-the-art performance on a number of challenging semantic segmentation datasets including @math , @math - @math , @math , @math - @math , @math - @math , @math - @math , and @math datasets. Particularly, we report an intersection-over-union score of @math on the @math - @math dataset. We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a Convolutional Neural Network (CNN). Rather than using hand-defined rules, the network propagates user edits by fusing low-level cues along with high-level semantic information, learned from large-scale data. We train on a million images, with simulated user inputs. To guide the user towards efficient input selection, the system recommends likely colors based on the input image and current user inputs. The colorization is performed in a single feed-forward pass, enabling real-time use. Even with randomly simulated user inputs, we show that the proposed system helps novice users quickly create realistic colorizations, and offers large improvements in colorization quality with just a minute of use. In addition, we demonstrate that the framework can incorporate other user "hints" to the desired colorization, showing an application to color histogram transfer. Our code and models are available at this https URL Deep ConvNets have been shown to be effective for the task of human pose estimation from single images. However, several challenging issues arise in the video-based case such as self-occlusion, motion blur, and uncommon poses with few or no examples in training data sets. Temporal information can provide additional cues about the location of body joints and help to alleviate these issues. In this paper, we propose a deep structured model to estimate a sequence of human poses in unconstrained videos. This model can be efficiently trained in an end-to-end manner and is capable of representing appearance of body joints and their spatio-temporal relationships simultaneously. Domain knowledge about the human body is explicitly incorporated into the network providing effective priors to regularize the skelet al structure and to enforce temporal consistency. The proposed end-to-end architecture is evaluated on two widely used benchmarks (Penn Action dataset and JHMDB dataset) for video-based pose estimation. Our approach significantly outperforms the existing state-of-the-art methods. Mirrors are indispensable objects in our lives. The capability of simulating a mirror on a computer display, augmented with virtual scenes and objects, opens the door to many interesting and useful applications from fashion design to medical interventions. Realistic simulation of a mirror is challenging as it requires accurate viewpoint tracking and rendering, wide-angle viewing of the environment, as well as real-time performance to provide immediate visual feedback. In this paper, we propose a virtual mirror rendering system using a network of commodity structured-light RGB-D cameras. The depth information provided by the RGB-D cameras can be used to track the viewpoint and render the scene from different prospectives. Missing and erroneous depth measurements are common problems with structured-light cameras. A novel depth denoising and completion algorithm is proposed in which the noise removal and interpolation procedures are guided by the foreground background label at each pixel. The foreground background label is estimated using a probabilistic graphical model that considers color, depth, background modeling, depth noise modeling, and spatial constraints. The wide viewing angle of the mirror system is realized by combining the dynamic scene, captured by the static camera network with a 3-D background model created off-line, using a color-depth sequence captured by a movable RGB-D camera. To ensure a real-time response, a scalable client-and-server architecture is used with the 3-D point cloud processing, the viewpoint estimate, and the mirror image rendering are all done on the client side. The mirror image and the viewpoint estimate are then sent to the server for final mirror view synthesis and viewpoint refinement. Experimental results are presented to show the accuracy and effectiveness of each component and the entire system. Skin detection plays an important role in a wide range of image processing applications ranging from face detection, face tracking, gesture analysis, content-based image retrieval systems and to various human computer interaction domains. Recently, skin detection methodologies based on skin-color information as a cue has gained much attention as skin-color provides computationally effective yet, robust information against rotations, scaling and partial occlusions. Skin detection using color information can be a challenging task as the skin appearance in images is affected by various factors such as illumination, background, camera characteristics, and ethnicity. Numerous techniques are presented in literature for skin detection using color. In this paper, we provide a critical up-to-date review of the various skin modeling and classification strategies based on color information in the visual spectrum. The review is divided into three different categories: first, we present the various color spaces used for skin modeling and detection. Second, we present different skin modeling and classification approaches. However, many of these works are limited in performance due to real-world conditions such as illumination and viewing conditions. To cope up with the rapidly changing illumination conditions, illumination adaptation techniques are applied along with skin-color detection. Third, we present various approaches that use skin-color constancy and dynamic adaptation techniques to improve the skin detection performance in dynamically changing illumination and environmental conditions. Wherever available, we also indicate the various factors under which the skin detection techniques perform well. In image recognition it is often assumed the method used to convert color images to grayscale has little impact on recognition performance. We compare thirteen different grayscale algorithms with four types of image descriptors and demonstrate that this assumption is wrong: not all color-to-grayscale algorithms work equally well, even when using descriptors that are robust to changes in illumination. These methods are tested using a modern descriptor-based image recognition framework, on face, object, and texture datasets, with relatively few training instances. We identify a simple method that generally works best for face and object recognition, and two that work well for recognizing textures. Let @math be a graph with a threshold function @math such that @math for every vertex @math of @math , where @math is the degree of @math in @math . Suppose we are given a target set @math . This paper considers the following repetitive process on @math . At time step @math the vertices of @math are colored black and the other vertices are colored white. After that, at each time step @math , the colors of white vertices (if any) are updated according to the following rule. All white vertices @math that have at least @math black neighbors at the time step @math are colored black, and the colors of the other vertices do not change. The process runs until no more white vertices can update colors from white to black. The following optimization problem is called Target Set Selection: Find a target set @math of smallest possible size such that all vertices in @math are black at the end of the process. Such an @math is called an optimal target set for @math under the th... This paper presents a robust text detection approach based on color-enhanced contrasting extremal region (CER) and neural networks. Given a color natural scene image, six component-trees are built from its grayscale image, hue and saturation channel images in a perception-based illumination invariant color space, and their inverted images, respectively. From each component-tree, color-enhanced CERs are extracted as character candidates. By using a "divide-and-conquer" strategy, each candidate image patch is labeled reliably by rules as one of five types, namely, Long, Thin, Fill, Square-large and Square-small, and classified as text or non-text by a corresponding neural network, which is trained by an ambiguity-free learning strategy. After pruning unambiguous non-text components, repeating components in each component-tree are pruned further. Remaining components are then grouped into candidate text-lines and verified by another set of neural networks. Finally, results from six component-trees are combined, and a post-processing step is used to recover lost characters. Our proposed method achieves superior performance on both ICDAR-2011 and ICDAR-2013 "Reading Text in Scene Images" test sets. Several open problems in this topic are discussed and we present our solution.Color-enhanced CERs are effective to be candidate-text-connected-components.Neural networks work very well for the challenging text non-text classification.The "ambiguity-free learning" strategy addresses the ambiguity problem properly.The "divide-and-conquer" strategy solves the size normalization problem well. We study the graph coloring problem over random graphs of finite average connectivity c. Given a number q of available colors, we find that graphs with low connectivity admit almost always a proper coloring whereas graphs with high connectivity are uncolorable. Depending on q, we find with a one-step replica-symmetry breaking approximation the precise value of the critical average connectivity @math Moreover, we show that below @math there exists a clustering phase @math in which ground states spontaneously divide into an exponential number of clusters. Furthermore, we extended our considerations to the case of single instances showing consistent results. This leads us to propose a different algorithm that is able to color in polynomial time random graphs in the hard but colorable region, i.e., when @math Searching for prior-art patents is an essential step for the patent examiner to validate or invalidate a patent application. In this paper, we consider the whole patent as the query, which reduces the burden on the user, and also makes many more potential search features available. We explore how to automatically transform the query patent into an effective search query, especially focusing on the effect of different patent fields. Experiments show that the background summary of a patent is the most useful source of terms for generating a query, even though most previous work used the patent claims. The concept of nearsightedeness of electronic matter (NEM) was introduced by W. Kohn in 1996 as the physical principal underlining Yang's electronic structure alghoritm of divide and conquer. It describes the fact that, for fixed chemical potential, local electronic properties at a point @math , like the density @math , depend significantly on the external potential @math only at nearby points. Changes @math of that potential, no matter how large , beyond a distance @math , have limited effects on local electronic properties, which tend to zero as function of @math . This remains true even if the changes in the external potential completely surrounds the point @math . NEM can be quantitatively characterized by the nearsightedness range, @math , defined as the smallest distance from @math , beyond which any change of the external potential produces a density change, at @math , smaller than a given @math . The present paper gives a detailed analysis of NEM for periodic met als and insulators in 1D and includes sharp, explicit estimates of the nearsightedness range. Since NEM involves arbitrary changes of the external potential, strong, even qualitative changes can occur in the system, such as the discretization of energy bands or the complete filling of the insulating gap of an insulator with continuum spectrum. In spite of such drastic changes, we show that @math has only a limited effect on the density, which can be quantified in terms of simple parameters of the unperturbed system. Recent progress on many imaging and vision tasks has been driven by the use of deep feed-forward neural networks, which are trained by propagating gradients of a loss defined on the final output, back through the network up to the first layer that operates directly on the image. We propose back-propagating one step further---to learn camera sensor designs jointly with networks that carry out inference on the images they capture. In this paper, we specifically consider the design and inference problems in a typical color camera---where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image. We learn the camera sensor's color multiplexing pattern by encoding it as layer whose learnable weights determine which color channel, from among a fixed set, will be measured at each location. These weights are jointly trained with those of a reconstruction network that operates on the corresponding sensor measurements to produce a full color image. Our network achieves significant improvements in accuracy over the traditional Bayer pattern used in most color cameras. It automatically learns to employ a sparse color measurement approach similar to that of a recent design, and moreover, improves upon that design by learning an optimal layout for these measurements. Much effort has been made in integrating different information in a variational framework to segment images. Recent works on curve propagation were able to incorporate stochastic information (see Paragios, N. and Deriche, R., J. Visual Commun. and Image Representation, 2002; Zhu, S. and Yuille, A., 1996) and prior knowledge on shapes (see Cremers, D. et al, 2002; Rousson M. and Paragios, N., 2002). The information inserted in these studies is most of the time extracted offline. Meanwhile, other approaches have proposed to extract region information during the segmentation process itself (see Chan, T. et al, 2000; Jehan-Besson, S. et al, 2002; Yezzi, A. et al, 1999). Following these new approaches and extending the work of Paragios and Deriche to vector-valued images, we propose an entirely variational framework to approach the segmentation problem. Both the image partition and the statistical parameters for each region are unknown. After a brief reminder on recent segmenting methods, we present a variational formulation obtained from a Bayesian model. After that, we show two different differentiations driving to the same evolution equations. Detailed studies on gray and color images of the 2-phase case follow. We finish with an application to tracking which shows the benefits of our dynamic framework. Purpose Development of a MRI acquisition and reconstruction strategy to depict fet al cardiac anatomy in the presence of maternal and fet al motion. Methods The proposed strategy involves i) acquisition and reconstruction of highly accelerated dynamic MRI, followed by image-based ii) cardiac synchronization, iii) motion correction, iv) outlier rejection, and finally v) cardiac cine reconstruction. Postprocessing entirely was automated, aside from a user-defined region of interest delineating the fet al heart. The method was evaluated in 30 mid- to late gestational age singleton pregnancies scanned without maternal breath-hold. Results The combination of complementary acquisition reconstruction and correction rejection steps in the pipeline served to improve the quality of the reconstructed 2D cine images, resulting in increased visibility of small, dynamic anatomical features. Artifact-free cine images successfully were produced in 36 of 39 acquired data sets; prolonged general fet al movements precluded processing of the remaining three data sets. Conclusions The proposed method shows promise as a motion-tolerant framework to enable further detail in MRI studies of the fet al heart and great vessels. Processing data in image-space allowed for spatial and temporal operations to be applied to the fet al heart in isolation, separate from extraneous changes elsewhere in the field of view. Magn Reson Med 79:327–338, 2018. © 2017 The Authors Magnetic Resonance in Medicine published by Wiley Periodicals, Inc. on behalf of International Society for Magnetic Resonance in Medicine. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. Abstract A method of robust feature-detection is proposed for visual tracking with a pan-tilt head. Even with good foreground models, the tracking process is liable to be disrupted by strong features in the background. Previous researchers have shown that the disruption can be somewhat suppressed by the use of image-subtraction. Building on this idea, a more powerful statistical model of background intensity is proposed in which a Gaussian mixture distribution is fitted to each of the pixels on a ‘virtual’ image plane. A fitting algorithm of the ‘Expectation-Maximisation’ type proves to be particularly effective here. Practical tests with contour tracking show marked improvement over image subtraction methods. Since the burden of computation is off-line, the online tracking process can run in real-time, at video field-rate. Fingerprint recognition plays an important role in many commercial applications and is used by millions of people every day, e.g. for unlocking mobile phones. Fingerprint image segmentation is typically the first processing step of most fingerprint algorithms and it divides an image into foreground, the region of interest, and background. Two types of error can occur during this step which both have a negative impact on the recognition performance: ‘true’ foreground can be labeled as background and features like minutiae can be lost, or conversely ‘true’ background can be misclassified as foreground and spurious features can be introduced. The contribution of this paper is threefold: firstly, we propose a novel factorized directional bandpass (FDB) segmentation method for texture extraction based on the directional Hilbert transform of a Butterworth bandpass (DHBB) filter interwoven with soft-thresholding. Secondly, we provide a manually marked ground truth segmentation for 10560 images as an evaluation benchmark. Thirdly, we conduct a systematic performance comparison between the FDB method and four of the most often cited fingerprint segmentation algorithms showing that the FDB segmentation method clearly outperforms these four widely used methods. The benchmark and the implementation of the FDB method are made publicly available.