Tracking Facial Points in unconstrained videos is challenging due to the non-rigid deformation that changes over time. In this paper, we propose to exploit incremental learning for person-specific alignment in wild conditions. Our approach takes advantage of part-based representation and cascade regression for robust and efficient alignment on each frame. Unlike existing methods that usually rely on models trained offline, we incrementally update the representation subspace and the cascade of regressors in a unified framework to achieve personalized modeling on the fly. To alleviate the drifting issue, the fitting results are evaluated using a deep neural network, where well-aligned faces are picked out to incrementally update the representation and fitting models. Both image and video datasets are employed to valid the proposed method. The results demonstrate the superior performance of our approach compared with existing approaches in terms of fitting accuracy and efficiency.
The active appearance model (AAM) is a well-known model that can represent a non-rigid object like the face effectively. However, the AAM often fails to converge correctly when the illumination conditions of face images change largely because it uses a set of fixed appearance basis vectors that are usually obtained in a training phase. To overcome this problem, we propose an adaptive AAM that updates the appearance basis vectors with the current face image by the incremental principal component analysis (PCA). However, the update of the appearance basis vectors with ill-fitted face images can worsen the AAM fitting to the forthcoming face images. To avoid this situation, we devise a conditional update method that updates the appearance basis vectors when the AAM fitting is good and the AAM reconstruction error is large. We evaluate the goodness of AAM fitting in terms of the number of outliers. When the AAM fitting is good we update the online appearance model (OAM) parameters, where the OAM is taken to keep the variation of input face image continuously, and also evaluate the goodness of the appearance basis vectors in terms of the magnitude of AAM reconstruction error. When the appearance basis vectors of the current AAM produces a large AAM reconstruction error, we update the appearance basis vectors using the incremental PCA. The proposed conditional update of the appearance basis vectors stabilizes the AAM fitting and improves the face tracking performance especially when the illumination condition changes very dynamically. Experimental results show that the adaptive AAM is superior to the conventional AAM in terms of the occurrence rate of fitting error and the fitting accuracy.  The development of facial databases with an abundance of annotated facial data captured under unconstrained 'in-the-wild' conditions have made discriminative facial deformable models the de facto choice for generic facial landmark localization. Even though very good performance for the facial landmark localization has been shown by many recently proposed discriminative techniques, when it comes to the applications that require excellent accuracy, such as facial behaviour analysis and facial motion capture, the semi-automatic person-specific or even tedious manual tracking is still the preferred choice. One way to construct a person-specific model automatically is through incremental updating of the generic model. This paper deals with the problem of updating a discriminative facial deformable model, a problem that has not been thoroughly studied in the literature. In particular, we study for the first time, to the best of our knowledge, the strategies to update a discriminative model that is trained by a cascade of regressors. We propose very efficient strategies to update the model and we show that is possible to automatically construct robust discriminative person and imaging condition specific models 'in-the-wild' that outperform state-of-the-art generic face alignment strategies.