This paper aims at developing an integrated system of clothing co-parsing, in order to jointly parse a set of clothing images (unsegmented but annotated with tags) into semantic configurations. We propose a data-driven framework consisting of two phases of inference. The first phase, referred as "image co-segmentation", iterates to extract consistent regions on images and jointly refines the regions over all images by employing the exemplar-SVM (ESVM) technique [23]. In the second phase (i.e. "region colabeling"), we construct a multi-image graphical model by taking the segmented regions as vertices, and incorporate several contexts of clothing configuration (e.g., item location and mutual interactions). The joint label assignment can be solved using the efficient Graph Cuts algorithm. In addition to evaluate our framework on the Fashionista dataset [30], we construct a dataset called CCP consisting of 2098 high-resolution street fashion photos to demonstrate the performance of our system. We achieve 90.29 88.23 segmentation accuracy and 65.52 63.89 recognition rate on the Fashionista and the CCP datasets, respectively, which are superior compared with state-of-the-art methods.
In literature, existing efforts on clothing human segmentation and recognition mainly focused on constructing expressive models to address various clothing styles and appearances @cite_0 @cite_1 @cite_14 @cite_34 @cite_32 @cite_3 @cite_19 . One classic work @cite_0 proposed a composite And-Or graph template for modeling and parsing clothing configurations. Later works studied on blocking models to segment clothes for highly occluded group images @cite_29 , or deformable spatial priors modeling for improving performance of clothing segmentation @cite_9 . Recent approaches incorporated shape-based human model @cite_16 , or pose estimation and supervised region labeling @cite_22 , and achieved impressive results. Despite acknowledged successes, these works have not yet been extended to the problem of clothing co-parsing, and they often require much labeling workload.
@cite_14 @cite_22 @cite_29 @cite_9 @cite_1 @cite_32 @cite_3 @cite_16 @cite_0 @cite_19 @cite_34
2074621908 2094311777 2045570199 1998827347 2108890589
In this paper we demonstrate an effective method for parsing clothing in fashion photographs, an extremely challenging problem due to the large number of possible garment items, variations in configuration, garment appearance, layering, and occlusion. In addition, we provide a large novel dataset and tools for labeling garment items, to enable future research on clothing estimation. Finally, we present intriguing initial results on using clothing estimates to improve pose identification, and demonstrate a prototype application for pose-independent visual garment retrieval. Clothing is one of the most informative cues of human appearance. In this paper, we propose a novel multi-person clothing segmentation algorithm for highly occluded images. The key idea is combining blocking models to address the person-wise occlusions. In contrary to the traditional layered model that tries to solve the full layer ranking problem, the proposed blocking model partitions the problem into a series of pair-wise ones and then determines the local blocking relationship based on individual and contextual information. Thus, it is capable of dealing with cases with a large number of people. Additionally, we propose a layout model formulated as Markov Network which incorporates the blocking relationship to pursue an approximately optimal clothing layout for group people. Experiments demonstrated on a group images dataset show the effectiveness of our algorithm. We present a method for segmenting the parts of multiple instances of a known object category exhibiting large variations in projected shape and colour. The method builds on an existing MRF formulation incorporating a prior shape model and colour distributions for the constituent parts. We propose a novel shape model consisting of a deformable spatial prior probability for the part-label at each pixel. We also make a simple extension to the MRF formulation to deal simultaneously with multiple objects within a global optimisation. Finally, we evaluate the method for the task of segmenting individual items of clothing in images depicting groups of people, and demonstrate improved performance against the state of the art for this task. We describe a simple model for parsing pedestrians based on shape. Our model assembles candidate parts from an oversegmentation of the image and matches them to a library of exemplars. Our matching uses a hierarchical decomposition into a variable number of parts and computes scores on partial matchings in order to prune the search space of candidate segment. Simple constraints enforce consistent layout of parts. Because our model is shape-based, it generalizes well. We use exemplars from a controlled dataset of poses but achieve good test performance on unconstrained images of pedestrians in street scenes. We demonstrate results of parsing detections returned from a standard scanning-window pedestrian detector and use the resulting parse to perform viewpoint prediction and detection re-scoring. Cloth modeling and recognition is an important and challenging problem in both vision and graphics tasks, such as dressed human recognition and tracking, human sketch and portrait. In this paper, we present a context sensitive grammar in an And-Or graph representation which will produce a large set of composite graphical templates to account for the wide variabilities of cloth configurations, such as T-shirts, jackets, etc. In a supervised learning phase, we ask an artist to draw sketches on a set of dressed people, and we decompose the sketches into categories of cloth and body components: collars, shoulders, cuff, hands, pants, shoes etc. Each component has a number of distinct subtemplates (sub-graphs). These sub-templates serve as leafnodes in a big And-Or graph where an And-node represents a decomposition of the graph into sub-configurations with Markov relations for context and constraints (soft or hard), and an Or-node is a switch for choosing one out of a set of alternative And-nodes (sub-configurations) - similar to a node in stochastic context free grammar (SCFG). This representation integrates the SCFG for structural variability and the Markov (graphical) model for context. An algorithm which integrates the bottom-up proposals and the topdown information is proposed to infer the composite cloth template from the image.