108674
Sample Selection for Maximal Diversity
The problem of selecting a sample subset sufficient to preserve diversity arises in many applications. One example is in the design of recombinant inbred lines (RIL) for genetic association studies. In this context, genetic diversity is measured by how many alleles are retained in the resulting inbred strains. RIL panels that are derived from more than two parental strains, such as the collaborative cross (Churchill et al., 2004), present a particular challenge with regard to which of the many existing lab mouse strains should be included in the initial breeding funnel in order to maximize allele retention. A similar problem occurs in the study of customer reviews when selecting a subset of products with a maximal diversity in reviews. Diversity in this case implies the presence of a set of products having both positive and negative ranks for each customer. In this paper, we demonstrate that selecting an optimal diversity subset is an NP-complete problem via reduction to set cover. This reduction is sufficiently tight that greedy approximations to the set cover problem directly apply to maximizing diversity. We then suggest a slightly modified subset selection problem in which an initial greedy diversity solution is used to effectively prune an exhaustive search for all diversity subsets bounded from below by a specified coverage threshold. Extensive experiments on real datasets are performed to demonstrate the effectiveness and efficiency of our approach.
9042204
CRD: fast co-clustering on large datasets utilizing sampling-based matrix decomposition
The problem of simultaneously clustering columns and rows (co-clustering) arises in important applications, such as text data mining, microarray analysis, and recommendation system analysis. Compared with the classical clustering algorithms, co-clustering algorithms have been shown to be more effective in discovering hidden clustering structures in the data matrix. The complexity of previous co-clustering algorithms is usually O(m X n), where m and n are the numbers of rows and columns in the data matrix respectively. This limits their applicability to data matrices involving a large number of columns and rows. Moreover, some huge datasets can not be entirely held in main memory during co-clustering which violates the assumption made by the previous algorithms. In this paper, we propose a general framework for fast co-clustering large datasets, CRD. By utilizing recently developed sampling-based matrix decomposition methods, CRD achieves an execution time linear in m and n. Also, CRD does not require the whole data matrix be in the main memory. We conducted extensive experiments on both real and synthetic data. Compared with previous co-clustering algorithms, CRD achieves competitive accuracy but with much less computational cost.
9189430
Exploring the energy-time tradeoff in MPI programs on a power-scalable cluster
Recently, energy has become an important issue in high-performance computing. For example, supercomputers that have energy in mind, such as BlueGene/L, have been built; the idea is to improve the energy efficiency of nodes. Our approach, which uses off-the-shelf, high-performance cluster nodes that are frequency scalable, allows energy saving by scaling down the CPU. This paper investigates the energy consumption and execution time of applications from a standard benchmark suite (NAS) on a power-scalable cluster. We study via direct measurement and simulation both intra-node and inter-node effects of memory and communication bottlenecks, respectively. Additionally, we compare energy consumption and execution time across different numbers of nodes. Our results show that a power-scalable cluster has the potential to save energy by scaling the processor down to lower energy levels. Furthermore, we found that for some programs, it is possible to both consume less energy and execute in less time when using a larger number of nodes, each at reduced energy. Additionally, we developed and validated a model that enables us to predict the energy-time tradeoff of larger clusters.
765245
CARE: Finding Local Linear Correlations in High Dimensional Data
Finding latent patterns in high dimensional data is an important research problem with numerous applications. Existing approaches can be summarized into 3 categories: feature selection, feature transformation (or feature projection) and projected clustering. Being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. In many emerging biomedical applications, however, scientists are interested in the local latent patterns held by feature subsets, which may be invisible via any global transformation. In this paper, we investigate the problem of finding local linear correlations in high dimensional data. Our goal is to find the latent pattern structures that may exist only in some subspaces. We formalize this problem as finding strongly correlated feature subsets which are supported by a large portion of the data points. Due to the combinatorial nature of the problem and lack of monotonicity of the correlation measurement, it is prohibitively expensive to exhaustively explore the whole search space. In our algorithm, CARE, we utilize spectrum properties and effective heuristic to prune the search space. Extensive experimental results show that our approach is effective in finding local linear correlations that may not be identified by existing methods.
11841640
Accelerating Profile Queries in Elevation Maps
Elevation maps are a widely used spatial data representation in geographical information systems (GIS). Paths on elevation maps can be characterized by profiles, which describe relative elevation as a function of distance. In this research, we address the inverse of this mapping - given a profile, how to efficiently find paths that could have generated it. This is called the profile query problem. Profiles have a wide variety of uses that include registering tracking information, or even other maps, to a given map. We describe a probabilistic model to characterize the maximal likelihood that a point lying on a path matches the query profile. Propagation of such probabilities to neighboring points can effectively prune the search space. This model enables us to efficiently answer queries of arbitrary profiles with user-specified error tolerances. When compared to existing spatial index methods, our approach supports more flexible queries with orders of magnitude speedup.
268487
Mining non-redundant high order correlations in binary data
Many approaches have been proposed to find correlations in binary data. Usually, these methods focus on pair-wise correlations. In biology applications, it is important to find correlations that involve more than just two features. Moreover, a set of strongly correlated features should be non-redundant in the sense that the correlation is strong only when all the interacting features are considered together. Removing any feature will greatly reduce the correlation.In this paper, we explore the problem of finding non-redundant high order correlations in binary data. The high order correlations are formalized using multi-information, a generalization of pairwise mutual information. To reduce the redundancy, we require any subset of a strongly correlated feature subset to be weakly correlated. Such feature subsets are referred to as Non-redundant Interacting Feature Subsets (NIFS). Finding all NIFSs is computationally challenging, because in addition to enumerating feature combinations, we also need to check all their subsets for redundancy. We study several properties of NIFSs and show that these properties are useful in developing efficient algorithms. We further develop two sets of upper and lower bounds on the correlations, which can be incorporated in the algorithm to prune the search space. A simple and effective pruning strategy based on pair-wise mutual information is also developed to further prune the search space. The efficiency and effectiveness of our approach are demonstrated through extensive experiments on synthetic and real-life datasets.
2085735
A router based unequal error control scheme for video over the Internet
This paper presents a router based unequal error protection scheme for video over the Internet. The proposed scheme classifies the whole bitstream into two priorities according to their importance and packetizes them into packets with two priorities. To reduce the loss ratio of video packets with higher priority, an active queue management scheme is designed at each router to provide more chance for these packets to transmit over the Internet. Compared to an end-to-end based approach, in which some redundancy packets are generated to protect the packets with higher priority, our proposed scheme utilizes the whole bandwidth for the source coding. Therefore, the final picture quality is improved.
18984298
Block Standstill and Homogeneity Based Fast Motion Estimation Algorithm for H.264 Video Coding
The paper proposed a fast motion estimation algorithm based on the spatial homogeneity and temporal standstill of video sequences. It classified a frame into different regions such as standstill, stationary or dynamic region. For MBs lying in the standstill region, motion estimation is skipped; for MBs lying in the stationary region, motion estimation is applied once to an MB in the region, and the resulting motion vector will be used by other MBs in the same region. Normal motion estimation is only carried out in the dynamic region. The new algorithm can be used either by itself alone, or by combining with other existing fast motion estimation methods. Experimental results show that the new algorithm can significantly improve the time efficiency of the H.264 encoder, and is able to achieve an average reduction of 27% in encoding, with an average PSNR loss of only 0.09 dB and 0.42% bit rate increase, compared with the original H.264 reference software.
31618528
Efficient contact state graph generation for assembly applications
An important aspect in the design of many automated assembly strategies is the ability to automatically generate the set of contact states that may occur during an assembly task. In this paper, we present an efficient means of constructing the set of all geometrically feasible contact states that may occur within a bounded set of misalignments (bounds determined by robot inaccuracy). This set is stored as a graph, referred to as an Assembly Contact State Graph (ACSG), which indicates neighbor relationships between feasible states. An ACSG is constructed without user intervention in two stages. In the first stage, all hypothetical primitive principle contacts (PPCs; all contact states allowing 5 degrees of freedom) are evaluated for geometric feasibility with respect to part-imposed and robot-imposed restrictions on relative positioning (evaluated using optimization). In the second stage, the feasibility of each of the various combinations of PPCs is efficiently evaluated, first using topological existence and uniqueness criteria, then using part-imposed and robot-imposed geometric criteria.
16312564
A new hybrid model for time-series prediction
This paper proposed a new hybrid model in order to increase time series prediction accuracy. This hybrid model considers the routine time prediction technique like AR, ANN or any others as atomic building block. A linear hybrid technique is used to combine their forecast result into the final result. The hybrid algorithm was tested against three different kinds of time series data. Experiments results showed the effectiveness of the proposed hybrid model.
6961558
Minimizing a stochastic maximum-reliability path
We consider a stochastic network interdiction problem in which the goal is to detect an evader, who selects a maximum-reliability path. Subject to a resource constraint, the interdictor installs sensors on a subset of the network’s arcs to minimize the value of the evader’s maximum-reliability path, i.e., to maximize the detection probability. When this decision is made, the evader’s origin–destination pair is known to the interdictor only through a probability distribution. Our model is framed as a stochastic mixed-integer program and solved by an enhanced L-shaped decomposition method. Our primary enhancement is via a valid inequality, which we call a step inequality. In earlier work [Morton et al., IIE Trans 39 (2007), 3–14], we developed step inequalities for the special case in which the evader encounters at most one sensor on an origin–destination path. Here, we generalize the step inequality to the case where the evader encounters multiple sensors. In this more general setting, the step inequality is tightly coupled to the decomposition scheme. An efficient separation algorithm identifies violated step inequalities and strengthens the linear programming relaxation of the L-shaped method’s master program. We apply this solution procedure with further computational enhancements to a collection of test problems. © 2008 Wiley Periodicals, Inc. NETWORKS, Vol. 52(3), 111–119 2008