8291379
Video Flashlights: Real Time Rendering of Multiple Videosfor Immersive Model Visualization
Videos and 3D models have traditionally existed in separate worlds and as distinct representations. Although texture maps for 3D models have been traditionally derived from multiple still images, real-time mapping of live videos as textures on 3D models has not been attempted. This paper presents a system for rendering multiple live videos in real-time over a 3D model as a novel and demonstrative application of the power of commodity graphics hardware. The system, metaphorically called the Video Flashlight system, "illuminates" a static 3D model with live video textures from static and moving cameras in the same way as a flashlight (torch) illuminates an environment. The Video Flashlight system is also an augmented reality solution for security and monitoring systems that deploy numerous cameras to monitor a large scale campus or an urban site. Current video monitoring systems are highly limited in providing global awareness since they typically display numerous camera videos on a grid of 2D displays. In contrast, the Video Flashlight system exploits the real-time rendering capabilities of current graphics hardware and renders live videos from various parts of an environment co-registered with the model. The user gets a global view of the model and is also able to visualize the dynamic videos simultaneously in the context of the model. In particular, the location of pixels and objects seen in the videos are precisely overlaid on the model while the user navigates through the model. The paper presents an overview of the system, details of the real-time rendering and demonstrates the efficacy of the augmented reality application.
31836660
A Global Matching Framework for Stereo Computation
This paper presents a new global matching framework f o r stereo computation. In this framework, the second view is first predicted front the reference view using the depth information. A global match measure is then defined as the similarity function between the predicted image and the actual image. Stereo coniputcction is converted into a search problem where the goal is to find the depth rnup that maximizes the global match measure. The major advantage of this framework is that the global visibility constraint is inherently enforced in the computation. This paper explores several key components of this framework including ( I ) three color segmentation based depth representations, (2) an incremental warping algorithm that dramatically reduces the computational complexity, and (3) scene constraints such as the smoothness constraint and the color sinzilarity constraint. Experimental results using different types of depth representations are presented. The quality of the computed depth maps is demonstrated through image-based rendering from new viewpoints.
25332181
A Heterogeneous Feature-based Image Alignment Method
In this paper, we propose a robust heterogeneous feature based image alignment method that utilizes points, lines and regions in a unified framework. The image motion is decomposed into progressively complex components, i.e., translation, similarity, affine, and projective motion models, and alignment is obtained with deliberatively selected suitable feature types and associated descriptors. Large convergence range is obtained by gradually constraining the search range of features in each stage. Notably, point and line features are jointly used and formulated in a RANSAC (random sample consensus) framework for robust estimation of a homography between low textured images. Further improvement is obtained with region based direct method. Experiments demonstrate superior alignment results of our approach to both gradient-based direct method and tradition point feature based alignment method
13502244
Object Tracking with Bayesian Estimation of Dynamic Layer Representations
Decomposing video frames into coherent 2D motion layers is a powerful method for representing videos. Such a representation provides an intermediate description that enables applications such as object tracking, video summarization and visualization, video insertion, and sprite-based video compression. Previous work on motion layer analysis has largely concentrated on two-frame or multi-frame batch formulations. The temporal coherency of motion layers and the domain constraints on shapes have not been exploited. This paper introduces a complete dynamic motion layer representation in which spatial and temporal constraints on shape, motion and layer appearance are modeled and estimated in a maximum a-posteriori (MAP) framework using the generalized expectation-maximization (EM) algorithm. In order to limit the computational complexity of tracking arbitrarily shaped layer ownership, we propose a shape prior that parameterizes the representation of shape and prevents motion layers from evolving into arbitrary shapes. In this work, a Gaussian shape prior is chosen to specifically develop a near-real-time tracker for vehicle tracking in aerial videos. However, the general idea of using a parametric shape representation as part of the state of a tracker is a powerful one that can be extended to other domains as well. Based on the dynamic layer representation, an iterative algorithm is developed for continuous object tracking over time. The proposed method has been successfully applied in an airborne vehicle tracking system. Its performance is compared with that of a correlation-based tracker and a motion change-based tracker to demonstrate the advantages of the new method. Examples of tracking when the backgrounds are cluttered and the vehicles undergo various rigid motions and complex interactions such as passing, turning, and stop-and-go demonstrate the strength of the complete dynamic layer representation.
6283962
Multi-view 3D analysis with applications for augmented reality and enhanced video visualization
The article presents methods for 3D scene geometry recovery/refinement and pose estimation from motion imagery in two representative scenarios. First, we present a method for pose estimation and scene geometry recovery from extended sequences without prior knowledge of the scheme. Second, we discuss how to recover camera poses when a rough scene model is provided. We show how to extend and refine the scene model using the recovered poses. Finally, we present applications of the above techniques for 3D imagery manipulation such as enhanced visualization for video and 3D insertion of synthetic objects in the imagery.
5884426
Validation of an optical flow method for tag displacement estimation
Presents a validation study of an optical-flow method for the rapid estimation of myocardial displacement in magnetic resonance tagged cardiac images. This registration and change visualization (RCV) software uses a hierarchical estimation technique to compute the flow field that describes the warping of an image of one cardiac phase into alignment with the next. This method overcomes the requirement of constant pixel intensity in standard optical-flow methods by preprocessing the input images to reduce any intensity bias which results from the reduction in stripe contrast throughout the cardiac cycle. To validate the method, SPAMM-tagged images were acquired of a silicon gel phantom with simulated rotational motion. The pixel displacement was estimated with the RCV method and the error in pixel tracking was <4% 1000 ms after application of the tags, and after 30/spl deg/ of rotation. An additional study was performed using a SPAMM-tagged multiphase slice of a canine left ventricle. The true displacement was determined using a previously validated active contour model (snakes). The error between methods was 6.7% at end systole. The RCV method has the advantage of tracking all pixels in the image in a substantially shorter period than the snakes method.
65135986
Model dependent inference of three-dimensional information from a sequence of two-dimensional images
In order to autonomously navigate through a complex environment, a mobile robot requires sensory feedback. This feedback will typically include the 3D motion and location of the robot and the 3D structure and motion of obstacles and other environmental features. The general problem considered in this thesis is how this 3D information may be obtained from a sequence of images generated by a camera mounted on a mobile robot. 
The first set of algorithms developed in this thesis are for robust determination of the 3D pose of the mobile robot from a matched set of model and image landmark features. Least-squares techniques for point and line tokens, which minimize both rotation and translation simultaneously are developed and shown to be far superior to the earlier techniques which solved for rotation first and then translation. However, least-squares techniques fail catastrophically when outliers (or gross errors) are present in the match data. Outliers arise frequently due to incorrect correspondences or gross errors in the 3D model. Robust techniques for pose determination are developed to handle data contaminated by fewer than 50.0% outliers. 
To make the model based approach widely applicable, it is necessary to be able to automatically build the landmark models. The approach adopted in this thesis is one of model extension and refinement. A partial model of the environment is assumed to exist and this model is extended over a sequence of frames. As will be shown in the experiments, the prior knowledge of the small partial model greatly enhances the robustness of the 3D structure computations. The initial 3D model may have errors and these too are refined over the sequence of frames. 
Finally, the sensitivity of pose determination and model extension to incorrect estimates of camera parameters is analyzed. It is shown that for small field of view systems, offsets in the image center do not significantly affect the location of the camera and the location of new 3D points in a world coordinate system. Errors in the focal length significantly affect only the component of translation along the optical axis in the pose computation.
44489135
Software Energy Optimization of Real Time Preemptive Tasks by Minimizing Cache-Related Preemption Costs
An attempt has been made to optimize the software energy of real time preemptive tasks by minimizing the cache related preemption costs which are primarily incurred due to the inter-task interference in the cache. We have presented an algorithm that outputs an "optimum" task layout which reduces the overall inter-task interference in cache and thus reduces the preemption costs of each task of a given task set. We have compared the result of our estimated layout with that of the random layout generated for benchmark examples for demonstrating the performance of our algorithm.
8733679
Stochastic Fluid Theory for P2P Streaming Systems
We develop a simple stochastic fluid model that seeks to expose the fundamental characteristics and limitations of P2P streaming systems. This model accounts for many of the essential features of a P2P streaming system, including the peers' realtime demand for content, peer churn (peers joining and leaving), peers with heterogeneous upload capacity, limited infrastructure capacity, and peer buffering and playback delay. The model is tractable, providing closed-form expressions which can be used to shed insight on the fundamental behavior of P2P streaming systems. The model shows that performance is largely determined by a critical value. When the system is of moderate-to-large size, if a certain ratio of traffic loads exceeds the critical value, the system performs well; otherwise, the system performs poorly. Furthermore, large systems have better performance than small systems since they are more resilient to bandwidth fluctuations caused by peer churn. Finally, buffering can dramatically improve performance in the critical region, for both small and large systems. In particular, buffering can bring more improvement than can additional infrastructure bandwidth.
2683699
Towards scalable reliability frameworks for error prone CMPs
As technology scales and the energy of computation continually approaches thermal equilibrium [1,2], parameter variations and noise levels will lead to larger error rates at various levels of the computation stack. The error rates would be especially high for post-CMOS and nanoelectronic systems as well as for probabilistic [3] and stochastic architectures [4]. N-modular redundancy (NMR) at the core-level has been proposed as a way to attain system reliability goals for multicore architectures. While core-level DMR and TMR have been shown to be effective when errors are rare, a large amount of core-level redundancy will be required for attaining system reliability goals in face of high error rates. This makes voting latency and bandwidth significant performance bottlenecks for such systems. In this paper, we present a scalable NMR framework for error prone chip multiprocessors(CMPs). The framework supports in-network fault tolerance where voting logic is integrated into routers to allow for truly distributed voting. The in-network fault tolerance router utilizes the expected redundancy in vote messages, to reduce some of the blocking overhead incurred at the leader, and also provide a mechanism to trade-off network bandwidth with latency. Our framework also supports proactive checkpoint deallocation which allows cores participating in voting to continue on with execution instead of waiting on notification from the voting logic. Finally, the framework supports dynamic constitution that allows an arbitrary core on this chip to be a part of an NMR group. This allows bypassing faulty cores as well as scheduling for performance. Our experiments show significant performance/bandwidth benefits from these optimizations.
14817913
The FastTrack overlay: A measurement study
Both in terms of number of participating users and in traffic volume, FastTrack is one of the most important applications in the Internet today. Nevertheless, because FastTrack is proprietary and uses encryption, little is understood about FastTrack's overlay structure and dynamics, its messaging protocol, and its index management. We have built two measurement apparatus--the FastTrack Sniffing Platform and the FastTrack Probing Tool--to unravel many of the mysteries behind FastTrack. We deploy the apparatus to study FastTrack's overlay structure and dynamics, its neighbor selection, its use of dynamic port numbers to circumvent firewalls, and its index management. Although this study does not fully solve the FastTrack puzzle, it nevertheless leads to a coherent description of FastTrack and its overlay. Furthermore, we leverage the measurement results to set forth a number of key principles for the design of a successful unstructured P2P overlay. The measurement results and resulting design principles in this paper should be useful for future architects of P2P overlay networks as well as for engineers managing ISPs.