Data collection and annotation are time-consuming in machine learning, expecially for large scale problem. A common approach for this problem is to transfer knowledge from a related labeled domain to a target one. There are two popular ways to achieve this goal: adversarial learning and self training. In this article, we first analyze the training unstablity problem and the mistaken confusion issue in adversarial learning process. Then, inspired by domain confusion and self-ensembling methods, we propose a combined model to learn feature and class jointly invariant representation, namely Domain Confusion with Self Ensembling (DCSE). The experiments verified that our proposed approach can offer better performance than empirical art in a variety of unsupervised domain adaptation benchmarks.
On the contrary, there is a Self-Ensembling (SE) method @cite_36 which is compeletly diffrent from the previous. It is derived from mean-teacher @cite_30 and used in unsupervised domain adaptation problem. The model is formed of a student network and a teacher network. The student is trained using the cross entropy loss as usual and the teacher's weights are equal to the exponential moving average weights of the student. Meanwhile, when the maximum predicted probability of a sample is greater than a predefined confidence threshold of 0.968, the teacher gives the self-ensembling predicted probability as the consistency labels of the student. Their approach achieved some state-of-the-art results in a variety of benchmarks.
@cite_36 @cite_30
2767722847 2790921521
This paper explores the use of self-ensembling for visual domain adaptation problems. Our technique is derived from the mean teacher variant (, 2017) of temporal ensembling (;, 2017), a technique that achieved state of the art results in the area of semi-supervised learning. We introduce a number of modifications to their approach for challenging domain adaptation scenarios and evaluate its effectiveness. Our approach achieves state of the art results in a variety of benchmarks, including our winning entry in the VISDA-2017 visual domain adaptation challenge. In small image benchmarks, our algorithm not only outperforms prior art, but can also achieve accuracy that is close to that of a classifier trained in a supervised fashion. Abstract In this paper, we propose a semi-supervised facial landmark detection algorithm (SEMI) based on convolutional neural network (CNN), which can detect facial components and landmarks simultaneously. Unlike previous coarse-to-fine algorithms, our model does not need extra input such as initial landmark prediction. It also solves the occlusion problem of large area by detecting the visible facial components while existing face detectors failed to detect faces. Semi-supervised learning algorithm is also an effective data augmentation method. In our experiment, each image has two types of ground truth, one is bounding-box related (classification and coordinates) and the other is landmark coordinates inside the bounding-box. The supervised data have both two types of ground truth while the semi-supervised data only have the bounding-box. Our model was trained by the merge of two parts of data. Extensive evaluations on Helen, LFPW and 300-W show that our algorithm is able to complete the landmark task and performs better than many state-of-the-art facial landmark detecting algorithms.