5156351
Model-based bundle adjustment with application to face modeling
We present a new model-based bundle adjustment algorithm to recover the 3D model of a scene/object from a sequence of images with unknown motions. Instead of representing scene/object by a collection of isolated 3D features (usually points), our algorithm uses a surface controlled by a small set of parameters. Compared with previous model-based approaches, our approach has the following advantages. First instead of using the model space as a regularizer we directly use it as our search space, thus resulting in a more elegant formulation with fewer unknowns and fewer equations. Second, our algorithm automatically associates tracked points with their correct locations on the surfaces, thereby eliminating the need for a prior 2D-to-3D association. Third, regarding face modeling, we use a very small set of face metrics (meaningful deformations) to parameterize the face geometry, resulting in a smaller search space and a better posed system. Experiments with both synthetic and real data show that this new algorithm is faster, more accurate and more stable than existing ones.
3137419
Subtle Facial Expression Modeling with Vector Field Decomposition
Facial expression means not only the feature motions but also the subtle appearance changes in the face. The details are important to facial expression synthesis. In this paper, a novel method based on vector field decomposition is presented to model subtle facial expressions. The intensity change of facial expression image is represented by Helmholtz-Hodge vector field decomposition. Based on this method, subtle facial expression mapping is carried out much faster and better. Experiment results show that our method is robust and convincible to model the subtle facial expression.
1313086
Image-based surface detail transfer
Changing an object's appearance by adding geometric details is desirable in many real-world applications. Bump mapping has acted as an alternative to adding geometrical details, to an otherwise smooth object. But constructing visually interesting bump maps requires practice and artistic skills. Computer vision techniques have helped for modeling real-world objects and their surface details. In our approach, for cases where we only want to transfer geometrical details from one object to another, we might not need to explicitly compute 3D structures. In particular, our novel technique captures the geometrical details of an object from a single image in a way that is independent of the object's reflectance property. We can then transfer geometrical details to another surface, producing the appearance of a new surface with added geometrical details while preserving the object's reflectance property. Our method's advantages are that it's simple to implement, requires only a single image for each object, and produces effective results.
6930232
Expressive expression mapping with ratio images
Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. In this paper, we present a novel technique for facial expression mapping. We capture the illumination change of one person's expression in what we call an expression ratio image (ERI). Together with geometric warping, we map an ERI to any other person's face image to generate more expressive facial expressions.
15783652
Low bit-rate video streaming for face-to-face teleconference
Face-to-face video teleconferencing is very important for real time communication. Current teleconferencing applications use standard video codecs, such as MPEG 1/2/4, for the compression of face video. Either a high bandwidth is required for high quality video transmission, or the transmitted face video is blurred at low bitrates. We present a system for real-time coding of face video at low bit-rate. There are two main contributions. First, we improve the technique of long term memory prediction by selecting frames into the database in an optimal way. A new frame is selected into the database only when it is significantly different from those frames which are already in the database. In this way, the database can cover a wider range of images. Second, we incorporate prior knowledge about faces into the long term memory prediction framework. The prior knowledge includes: (1) facial motions are repetitive such that most of them can be reconstructed from multiple reference frames; (2) different components of the face and the background can tolerate different levels of error because of different perceptual importance. Experiments show that, at similar PSNR, the proposed system works much faster and achieves better visual quality than the standard H.264/JVT codec.
5771284
Face relighting with radiance environment maps
A radiance environment map pre-integrates a constant surface reflectance with the lighting environment. It has been used to generate photo-realistic rendering at interactive speed. However, one of its limitations is that each radiance environment map can only render the object, which has the same surface reflectance as what it integrates. We present a ratio-image based technique to use a radiance environment map to render diffuse objects with different surface reflectance properties. This method has the advantage that it does not require the separation of illumination from reflectance, and it is simple to implement and runs at interactive speed. In order to use this technique for human face relighting, we have developed a technique that uses spherical harmonics to approximate the radiance environment map for any given image of a face. Thus we are able to relight face images when the lighting environment rotates. Another benefit of the radiance environment map is that we can interactively modify lighting by changing the coefficients of the spherical harmonics basis. Finally we can modify the lighting condition of one person's face so that it matches the new lighting condition of a different person's face image assuming the two faces have similar skin albedos.
2023739
Joint video scene segmentation and classification based on hidden Markov model
Video classification and segmentation are fundamental steps for efficient accessing, retrieval and browsing of large amounts of video data. We have developed a scene classification scheme using a hidden Markov model (HMM) based classifier. By utilizing the temporal behaviors of different scene classes, the HMM classifier can effectively classify video segments into one of the pre-defined scene classes. In this paper, we describe two approaches for joint video classification and segmentation based on a HMM, which works by searching for the most likely class transition path utilizing the dynamic programming technique.
16274304
Face detection and tracking in video using dynamic programming
Face detection and tracking are important in video content analysis since the most important objects in most video are human beings. This paper proposes a new approach for combined face detection and tracking in video. The face detection algorithm is a fast template matching procedure using iterative dynamic programming (DP). Although the face detection algorithm is designed for frontal face, the same mechanism can also be applied to track non-frontal faces with online adapted face models. Due to the essence of template matching, the algorithm is capable of comparing the similarity among different faces, which makes it suitable for tracking the same face that occur at disjointed temporal locations in video. While the proposed face detection method provides comparable accuracy as the neural network based approach, it is much faster.
3166137
Integrating Temporal Logics
In this paper, we study the predicative semantics of different temporal logics and the relationships between them. We use a notation called generic composition to simplify the manipulation of predicates. The modalities of possibility and necessity become generic composition and its inverse of converse respectively. The relationships between different temporal logics are also characterised as such modalities. Formal reasoning is carried out at the level of predicative semantics and supported by the higher-level laws of generic composition and its inverse. Various temporal domains are unified under a notion called resource cumulation. Temporal logics based on these temporal domains can be readily defined, and their axioms identified. The formalism provides a framework in which human experience about system development can be formalised as refinement laws. The approach is demonstrated in the transformation from Duration Calculus to Temporal Logic of Actions. A number of common design patterns are studied. The refinement laws identified are then applied to the case study of water pump controlling.
42940477
Hamiltonian iterated line graphs
The n-iterated line graph of a graph G is Ln(G) = L(Ln-1(G)), where L1(G) denotes the line graph L(G) of G, and Ln-1(G) is assumed to be nonempty. Harary and Nash-Williams characterized those graphs G for which L(G) is hamiltonian. In this paper, we will give a characterization of those graphs G for which Ln(G) is hamiltonian, for each n ≥ 2. This is not a simple consequence of Harary and Nash-Williams' result. As an application, we show two methods for determining the hamiltonian index of a graph and enhance various results on the hamiltonian index known earlier.
15316832
Multimedia search and retrieval: new concepts, system implementation, and application
We first present new concepts applicable to the design of multimedia search and retrieval schemes in general, and to MPEG-7 in particular, the multimedia description standard in progress. Raw multimedia data is assumed to exist in the form of programs that typically consist of a combination of media types such as visual, audio, and text. We partition each such media stream into smaller units based on actual physical events. These physical events within each media stream can then be effectively indexed for retrieval. The concept of logical events is introduced next; we define logical events as those that can provide different "views" of the content as may be desired by a user. Such events usually result from either the correlation of events that cross different media types, of by merging recursively chosen events from a lower level within each media type. We then address the related issue of how to develop a practical multimedia information retrieval system that exploits the aforementioned concepts of physical and logical events as well as other aspects such as storage, representation and indexing to enable efficient search, retrieval, and browsing. Finally, we implement the proposed concepts and solutions within a multimedia system that addresses a real application, effective browsing of broadcast news, and evaluate its performance.