We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.
Multiple views or perturbations of the data can provide useful signal for semi-supervised learning. In the co-training framework @cite_9 , different views of the data are used to learn two distinct classifiers that are then used to train one another over unlabeled data. Reed al @cite_1 use a reconstruction consistency term for training classification and detection models. Bachman al @cite_18 employ the pseudo-ensemble regularization term to train models robust on input perturbations. Sajjadi al @cite_30 enforce consistency between outputs computed for different transformations of input examples. Simon al @cite_27 utilize multi-view geometry to generate hand keypoint labels from multiple cameras and retrain the detector. In an auto-encoder scenario, Hinton al @cite_44 propose to use multiple capsules'' to model multiple geometric transformations. Our method is also based on multiple geometric transformations, but it does not require to modify network structures or impose consistency by adding any extra loss terms.
@cite_30 @cite_18 @cite_9 @cite_1 @cite_44 @cite_27
2431080869 2949299379 2048679005 2962762541 2609211631
We formalize the notion of a pseudo-ensemble, a (possibly infinite) collection of child models spawned from a parent model by perturbing it according to some noise process. E.g., dropout (Hinton et. al, 2012) in a deep neural network trains a pseudo-ensemble of child subnetworks generated by randomly masking nodes in the parent network. We present a novel regularizer based on making the behavior of a pseudo-ensemble robust with respect to the noise process generating it. In the fully-supervised setting, our regularizer matches the performance of dropout. But, unlike dropout, our regularizer naturally extends to the semi-supervised setting, where it produces state-of-the-art results. We provide a case study in which we transform the Recursive Neural Tensor Network of (Socher et. al, 2013) into a pseudo-ensemble, which significantly improves its performance on a real-world sentiment analysis benchmark. A citing sentence is one that appears in a scientific article and cites previous work. Citing sentences have been studied and used in many applications. For example, they have been used in scientific paper summarization, automatic survey generation, paraphrase identification, and citation function classification. Citing sentences that cite multiple papers are common in scientific writing. This observation should be taken into consideration when using citing sentences in applications. For instance, when a citing sentence is used in a summary of a scientific paper, only the fragments of the sentence that are relevant to the summarized paper should be included in the summary. In this paper, we present and compare three different approaches for identifying the fragments of a citing sentence that are related to a given target reference. Our methods are: word classification, sequence labeling, and segment classification. Our experiments show that segment classification achieves the best results. Purpose Development of a MRI acquisition and reconstruction strategy to depict fet al cardiac anatomy in the presence of maternal and fet al motion. Methods The proposed strategy involves i) acquisition and reconstruction of highly accelerated dynamic MRI, followed by image-based ii) cardiac synchronization, iii) motion correction, iv) outlier rejection, and finally v) cardiac cine reconstruction. Postprocessing entirely was automated, aside from a user-defined region of interest delineating the fet al heart. The method was evaluated in 30 mid- to late gestational age singleton pregnancies scanned without maternal breath-hold. Results The combination of complementary acquisition reconstruction and correction rejection steps in the pipeline served to improve the quality of the reconstructed 2D cine images, resulting in increased visibility of small, dynamic anatomical features. Artifact-free cine images successfully were produced in 36 of 39 acquired data sets; prolonged general fet al movements precluded processing of the remaining three data sets. Conclusions The proposed method shows promise as a motion-tolerant framework to enable further detail in MRI studies of the fet al heart and great vessels. Processing data in image-space allowed for spatial and temporal operations to be applied to the fet al heart in isolation, separate from extraneous changes elsewhere in the field of view. Magn Reson Med 79:327338, 2018.  2017 The Authors Magnetic Resonance in Medicine published by Wiley Periodicals, Inc. on behalf of International Society for Magnetic Resonance in Medicine. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, dis It is expensive to label images with 3D structure or precise camera pose. Yet, this is precisely the kind of annotation required to train single-view 3D reconstruction models. In contrast, unlabeled images or images with just category labels are easy to acquire, but few current models can use this weak supervision. We present a unified framework that can combine both types of supervision: a small amount of camera pose annotations are used to enforce pose-invariance and view-point consistency, and unlabeled images combined with an adversarial loss are used to enforce the realism of rendered, generated models. We use this unified framework to measure the impact of each form of supervision in three paradigms: semi-supervised, multi-task, and transfer learning. We show that with a combination of these ideas, we can train single-view reconstruction models that improve up to 7 points in performance (AP) when using only 1 pose annotated training data. Abstract Documents in computer-readable form can be used to provide information about other documents, i.e. those they cite. To do this efficiently requires procedures for computer recognition of citing statements. This is not easy, especially for multi-sentence citing statements. Computer recognition procedures have been developed which are accurate to the following extent: 73 of the words in statements selected by computer procedures as being citing statements are words which are correctly attributable to the corresponding documents. The retrieval effectiveness of computer-recognized citing statements was tested in the following way. First, for eight retrieval requests in inorganiic chemistry, average recall by search of Chemical Abstracts Service indexing and Chemical Abstracts abstract text words was found to be 50 . Words from citing statements referring to the papers to be retrieved were then added to the index terms and abstract words as additional access points, and searching was repeated. Average recall increased to 70 . Only words from citing statements published within a year of the cited papers were used. The retrieval effect of citing statement words alone (published within a year) without index or abstract terms was the following: average recall was 40 . When just the words of the titles of the cited papers were added to those citing statement words, average recall increased to 50 . In a document network such as a citation network of scientific documents, web-logs, etc., the content produced by authors exhibits their interest in certain topics. In addition some authors influence other authors' interests. In this work, we propose to model the influence of cited authors along with the interests of citing authors. Moreover, we hypothesize that apart from the citations present in documents, the context surrounding the citation mention provides extra topical information about the cited authors. However, associating terms in the context to the cited authors remains an open problem. We propose novel document generation schemes that incorporate the context while simultaneously modeling the interests of citing authors and influence of the cited authors. Our experiments show significant improvements over baseline models for various evaluation criteria such as link prediction between document and cited author, and quantitatively explaining unseen text.