Many real world tasks such as reasoning and physical interaction require identification and manipulation of conceptual entities. A first step towards solving these tasks is the automated discovery of distributed symbol-like representations. In this paper, we explicitly formalize this problem as inference in a spatial mixture model where each component is parametrized by a neural network. Based on the Expectation Maximization framework we then derive a differentiable clustering method that simultaneously learns how to group and represent individual entities. We evaluate our method on the (sequential) perceptual grouping task and find that it is able to accurately recover the constituent objects. We demonstrate that the learned representations are useful for next-step prediction.
The method most closely related to our approach is Tagger @cite_20 , which similarly learns perceptual grouping in an unsupervised fashion using @math copies of a neural network that work together by reconstructing different parts of the input. Unlike in case of N-EM, these copies additionally learn to output the grouping, which gives Tagger more direct control over the segmentation and supports its use on complex texture segmentation tasks. Our work maintains a close connection to EM and relies on the posterior inference of the E-Step as a grouping mechanism. This facilitates theoretical analysis and simplifies the task for the resulting networks, which we find can be markedly smaller than in Tagger. Furthermore, Tagger does not include any recurrent connections on the level of the hidden states, precluding it from next step prediction on sequential tasks. RTagger @cite_6 : a recurrent extension of Tagger that does support sequential data was developed concurrent to this work.
@cite_6 @cite_20
2740777550 2463993963
This paper presents the IMS contribution to the CoNLL 2017 Shared Task. In the preprocessing step we employed a CRFPOS morphological tagger and a neural tagger predicting supertags. On some languages, we also applied word segmenta- tion with the CRF tagger and sentence segmentation with a perceptron-based parser. For parsing we took an ensemble approach by blending multiple instances of three parsers with very different architectures. Our system achieved the third place overall and the second place for the surprise languages We apply computer vision with deep learning -- in the form of a convolutional neural network (CNN) -- to build a highly effective boosted top tagger. Previous work (the "DeepTop" tagger of ) has shown that a CNN-based top tagger can achieve comparable performance to state-of-the-art conventional top taggers based on high-level inputs. Here, we introduce a number of improvements to the DeepTop tagger, including architecture, training, image preprocessing, sample size and color pixels. Our final CNN top tagger outperforms BDTs based on high-level inputs by a factor of @math --3 or more in background rejection, over a wide range of tagging efficiencies and fiducial jet selections. As reference points, we achieve a QCD background rejection factor of 500 (60) at 50 top tagging efficiency for fully-merged (non-merged) top jets with @math in the 800--900 GeV (350--450 GeV) range. Our CNN can also be straightforwardly extended to the classification of other types of jets, and the lessons learned here may be useful to others designing their own deep NNs for LHC applications.