16039054
Feature selection for ranking using boosted trees
Modern search engines have to be fast to satisfy users, so there are hard back-end latency requirements. The set of features useful for search ranking functions, though, continues to grow, making feature computation a latency bottleneck. As a result, not all available features can be used for ranking, and in fact, much of the time, only a small percentage of these features can be used. Thus, it is crucial to have a feature selection mechanism that can find a subset of features that both meets latency requirements and achieves high relevance. To this end, we explore different feature selection methods using boosted regression trees, including both greedy approaches (selecting the features with highest relative importance as computed by boosted trees; discounting importance by feature similarity and a randomized approach. We evaluate and compare these approaches using data from a commercial search engine. The experimental results show that the proposed randomized feature selection with feature-importance-based backward elimination outperforms greedy approaches and achieves a comparable relevance with 30 features to a full-feature model trained with 419 features and the same modeling parameters.
11110859
A locally-adaptive algorithm for measuring blocking artifacts in images and videos
Block transform coding is the most popular approach for image and video compression. The objective measurement of blocking artifacts plays an important role in the design, optimization, and assessment of image and video coding systems. This paper presents a new algorithm for measuring blocking artifacts in images and videos. It exhibits unique and useful features: 1) it examines the blocks individually so that it can measure the severity of blocking artifacts locally; 2) it is a one-pass algorithm in the sense that the image needs to be accessed only once; 3) it takes into account the blocking artifacts for high bit rate images and the flatness for the very low bit rate images; 4) the blocking artifacts measure is well-defined in the range of 0-10. Experiments on various still images and videos show that this blockiness measure is very efficient in terms of computational complexity and memory usage, and can produce consistent blocking artifacts measurement.
981043
Learning Event Durations from Event Descriptions
We have constructed a corpus of news articles in which events are annotated for estimated bounds on their duration. Here we describe a method for measuring inter-annotator agreement for these event duration distributions. We then show that machine learning techniques applied to this data yield coarse-grained event duration information, considerably outperforming a baseline and approaching human performance.
9260921
Complexity adaptive quantization for intra-frames in very low bit rate video coding
Conventional rate control schemes focus on the problem of finding an optimal quantization value for P- and B-frames. No rate control is available for the encoding of I-frames. This could pose big problems due to the large number of bits an I-frame could generate, and due to the fact that the number of bits varies drastically from sequence to sequence, depending on their image complexity. This problem becomes severe especially at very low bit rate. Therefore, a mechanism to allocate data bits to an I-frame according to its complexity is indispensable in order to have constant coding quality. This work presents a mechanism to establish for I-frames a generic relationship between the quantization value, data bits, and their image complexity. Experimental results show that this generic relationship provides a fairly accurate estimation of quantization value for an I-frame at given data bits and image complexity, and is very useful in controlling the data bits generated by an I-frame.
39269573
An adaptive edge-preserving artifacts removal filter for video post-processing
We propose an adaptive edge-preserving artifacts removal filter (EPARF) for post-processing video compressed using block-based motion compensation so as to improve visual quality. The proposed filter is based on half-quadratic regularization, and it can adaptively remove both blocking and ringing artifacts while preserving object edges. The proposed algorithm has been applied to video sequences compressed by the MSRA scalable video coding (SVC) codec with block-based motion-compensated temporal filtering (MCTF). Experimental results demonstrate that EPARF can significantly improve both subjective and objective quality.
8588212
Temporal Aggregates in OWL-Time
In this paper, we describe our approach for representing temporal aggregates in OWL-Time, and a systematic way of mapping iCalendar recurrence sets to OWL-Time temporal sequences. We present several examples of natural language expressions for temporal aggregates and how they would be represented in OWL-Time; some examples can be described in both iCalendar and OWL-Time, while others can only be represented in OWL-Time. We also illustrate how we can translate from a natural language sentence to our representation.
268487
Mining non-redundant high order correlations in binary data
Many approaches have been proposed to find correlations in binary data. Usually, these methods focus on pair-wise correlations. In biology applications, it is important to find correlations that involve more than just two features. Moreover, a set of strongly correlated features should be non-redundant in the sense that the correlation is strong only when all the interacting features are considered together. Removing any feature will greatly reduce the correlation.In this paper, we explore the problem of finding non-redundant high order correlations in binary data. The high order correlations are formalized using multi-information, a generalization of pairwise mutual information. To reduce the redundancy, we require any subset of a strongly correlated feature subset to be weakly correlated. Such feature subsets are referred to as Non-redundant Interacting Feature Subsets (NIFS). Finding all NIFSs is computationally challenging, because in addition to enumerating feature combinations, we also need to check all their subsets for redundancy. We study several properties of NIFSs and show that these properties are useful in developing efficient algorithms. We further develop two sets of upper and lower bounds on the correlations, which can be incorporated in the algorithm to prune the search space. A simple and effective pruning strategy based on pair-wise mutual information is also developed to further prune the search space. The efficiency and effectiveness of our approach are demonstrated through extensive experiments on synthetic and real-life datasets.
3069611
A General Framework for Fast Co-clustering on Large Datasets Using Matrix Decomposition
Simultaneously clustering columns and rows (co- clustering) of large data matrix is an important problem with wide applications, such as document mining, microarray analysis, and recommendation systems. Several co-clustering algorithms have been shown effective in discovering hidden clustering structures in the data matrix. For a data matrix of m rows and n columns, the time complexity of these methods is usually in the order of mtimesn (if not higher). This limits their applicability to data matrices involving a large number of columns and rows. Moreover, an implicit assumption made by existing co-clustering methods is that the whole data matrix needs to be held in the main memory. In this paper, we propose a general framework, CRD, for co-clustering large datasets utilizing recently developed sampling- based matrix decomposition methods. The time complexity of our approach is linear in m and n. And it does not require the whole data matrix be in the main memory. Experimental results show that CRD achieves competitive accuracy to existing co-clustering methods but with much less computational cost.
108674
Sample Selection for Maximal Diversity
The problem of selecting a sample subset sufficient to preserve diversity arises in many applications. One example is in the design of recombinant inbred lines (RIL) for genetic association studies. In this context, genetic diversity is measured by how many alleles are retained in the resulting inbred strains. RIL panels that are derived from more than two parental strains, such as the collaborative cross (Churchill et al., 2004), present a particular challenge with regard to which of the many existing lab mouse strains should be included in the initial breeding funnel in order to maximize allele retention. A similar problem occurs in the study of customer reviews when selecting a subset of products with a maximal diversity in reviews. Diversity in this case implies the presence of a set of products having both positive and negative ranks for each customer. In this paper, we demonstrate that selecting an optimal diversity subset is an NP-complete problem via reduction to set cover. This reduction is sufficiently tight that greedy approximations to the set cover problem directly apply to maximizing diversity. We then suggest a slightly modified subset selection problem in which an initial greedy diversity solution is used to effectively prune an exhaustive search for all diversity subsets bounded from below by a specified coverage threshold. Extensive experiments on real datasets are performed to demonstrate the effectiveness and efficiency of our approach.
41075889
Efficient Compression of Multi-view Video Using Hierarchical B Pictures
3D and free viewpoint video are new types of natural video media that expand the user's sensation far beyond what is offered by traditional media. As video compression is a key issue in traditional video communications and the use of multi video streams makes it even more important for such applications. In this paper, we presented an investigation of the multi- view video coding structures with focus on exploiting hierarchical B pictures. We demonstrate hierarchical B pictures prediction structures, and present the advantage of several multi-view video coding prediction schemes. This demonstration will show the benefits of using hierarchical B pictures based on H.264/AVC. The simulation results demonstrate that the objective quality can be improved with the advanced MVC prediction structures in comparison to the simulcast coding structures.
2768791
Finding representative set from massive data
In the information age, data is pervasive. In some applications, data explosion is a significant phenomenon. The massive data volume poses challenges to both human users and computers. In this project, we propose a new model for identifying representative set from a large database. A representative set is a special subset of the original dataset, which has three main characteristics: It is significantly smaller in size compared to the original dataset. It captures the most information from the original dataset compared to other subsets of the same size. It has low redundancy among the representatives it contains. We use information-theoretic measures such as mutual information and relative entropy to measure the representativeness of the representative set. We first design a greedy algorithm and then present a heuristic algorithm that delivers much better performance. We run experiments on two real datasets and evaluate the effectiveness of our representative set in terms of coverage and accuracy. The experiments show that our representative set attains expected characteristics and captures information more efficiently.