Abstract Recently, the progress of learning-by-synthesis has proposed a training model for synthetic images, which can effectively reduce the cost of human and material resources. However, due to the different distribution of synthetic images compared to real images, the desired performance cannot still be achieved. Real images consist of multiple forms of light orientation, while synthetic images consist of a uniform light orientation. These features are considered to be characteristic of outdoor and indoor scenes, respectively. To solve this problem, the previous method learned a model to improve the realism of the synthetic image. Different from the previous methods, this paper takes the first step to purify real images. Through the style transfer task, the distribution of outdoor real images is converted into indoor synthetic images, thereby reducing the influence of light. Therefore, this paper proposes a real-time style transfer network that preserves image content information (e.g., gaze direction, pupil center position) of an input image (real image) while inferring style information (e.g., image color structure, semantic features) of style image (synthetic image). In addition, the network accelerates the convergence speed of the model and adapts to multi-scale images. Experiments were performed using mixed studies (qualitative and quantitative) methods to demonstrate the possibility of purifying real images in complex directions. Qualitatively, it compares the proposed method with the available methods in a series of indoor and outdoor scenarios of the LPW dataset. In quantitative terms, it evaluates the purified image by training a gaze estimation model on the cross data set. The results show a significant improvement over the baseline method compared to the raw real image.
Recent studies aim to better represent the appearance, and @cite_28 proposed a low-dimensional feature extraction method. It divides the eye area into three columns and five columns and calculates the gray value and the percentage of each area. Therefore, a 15-dimensional feature vector is defined. However, this feature does not apply to eye images under free head movement. @cite_35 introduced a deep feature extracted from convolutional neural networks. The deep feature has sparse characters and provides a effective solution for gaze estimation.
@cite_28 @cite_35
2492303557 2281723898 2139196511 1946259682
Conventional appearance-based gaze estimation methods employ local or global features as eye gaze appearance descriptor. But these methods don't work well under natural light with free head movement. To solve this problem, we present an appearance-based gaze estimation method using deep feature representation and feature forest regression. The deep feature is learned through hierarchical extraction of deep Convolutional Neural Network (CNN). And random forest regression with cluster-to-classify node splitting rules is used to take advantage of data distribution in sparse feature space. Experimental results demonstrate that the deep feature has a better performance than local features on calibrated gaze regression. The combination of deep features and random forest regression provides an effective solution for gaze estimation in a natural environment. A key issue regarding feature extraction is the capability of a technique to extract distinctive features to represent facial expressions while requiring a low computational complexity. In this study, the authors propose a novel approach for appearance-based facial feature extraction to perform the task of facial expression recognition on video sequences. The proposed spatiotemporal texture map (STTM) is capable of capturing subtle spatial and temporal variations of facial expressions with low computational complexity. First, face is detected using Viola–Jones face detector and frames are cropped to remove unnecessary background. Facial features are then modelled with the proposed STTM, which uses the spatiotemporal information extracted from three-dimensional Harris corner function. A block-based method is adopted to extract the dynamic features and represent the features in the form of histograms. The features are then classified into classes of emotion by the support vector machine classifier. The experimental results demonstrate that the proposed approach shows superior performance compared with the state-of-the-art approaches with an average recognition rate of 95.37, 98.56, and 84.52 on datasets containing posed expressions, spontaneous micro-expressions, and close-to-real-world expressions, respectively. They also show that the proposed algorithm requires low computational cost. The problem of estimating human gaze from eye appearance is regarded as mapping high-dimensional features to low-dimensional target space. Conventional methods require densely obtained training samples on the eye appearance manifold, which results in a tedious calibration stage. In this paper, we introduce an adaptive linear regression (ALR) method for accurate mapping via sparsely collected training samples. The key idea is to adaptively find the subset of training samples where the test sample is most linearly representable. We solve the problem via l1-optimization and thoroughly study the key issues to seek for the best solution for regression. The proposed gaze estimation approach based on ALR is naturally sparse and low-dimensional, giving the ability to infer human gaze from variant resolution eye images using much fewer training samples than existing methods. Especially, the optimization procedure in ALR is extended to solve the subpixel alignment problem simultaneously for low resolution test eye images. Performance of the proposed method is evaluated by extensive experiments against various factors such as number of training samples, feature dimensionality and eye image resolution to verify its effectiveness. We present a method for estimating eye gaze direction, which represents a departure from conventional eye gaze estimation methods, the majority of which are based on tracking specific optical phenomena like corneal reflection and the Purkinje images. We employ an appearance manifold model, but instead of using a densely sampled spline to perform the nearest manifold point query, we retain the original set of sparse appearance samples and use linear interpolation among a small subset of samples to approximate the nearest manifold point. The advantage of this approach is that since we are only storing a sparse set of samples, each sample can be a high dimensional vector that retains more representational accuracy than short vectors produced with dimensionality reduction methods. The algorithm was tested with a set of eye images labelled with ground truth point-of-regard coordinates. We have found that the algorithm is capable of estimating eye gaze with a mean angular error of 0.38 degrees, which is comparable to that obtained by commercially available eye trackers.