12021051
COE: A General Approach for Efficient Genome-Wide Two-Locus Epistasis Test in Disease Association Study
The availability of high density single nucleotide polymorphisms (SNPs) data has made genome-wide association study computationally challenging. Two-locus epistasis (gene-gene interaction) detection has attracted great research interest as a promising method for genetic analysis of complex diseases. In this paper, we propose a general approach, COE, for efficient large scale gene-gene interaction analysis, which supports a wide range of tests. In particular, we show that many commonly used statistics are convex functions. From the observed values of the events in two-locus association test, we can develop an upper bound of the test value. Such an upper bound only depends on single-locus test and the genotype of the SNP-pair. We thus group and index SNP-pairs by their genotypes. This indexing structure can benefit the computation of all convex statistics. Utilizing the upper bound and the indexing structure, we can prune most of the SNP-pairs without compromising the optimality of the result. Our approach is especially efficient for large permutation test. Extensive experiments demonstrate that our approach provides orders of magnitude performance improvement over the brute force approach.
9042204
CRD: fast co-clustering on large datasets utilizing sampling-based matrix decomposition
The problem of simultaneously clustering columns and rows (co-clustering) arises in important applications, such as text data mining, microarray analysis, and recommendation system analysis. Compared with the classical clustering algorithms, co-clustering algorithms have been shown to be more effective in discovering hidden clustering structures in the data matrix. The complexity of previous co-clustering algorithms is usually O(m X n), where m and n are the numbers of rows and columns in the data matrix respectively. This limits their applicability to data matrices involving a large number of columns and rows. Moreover, some huge datasets can not be entirely held in main memory during co-clustering which violates the assumption made by the previous algorithms. In this paper, we propose a general framework for fast co-clustering large datasets, CRD. By utilizing recently developed sampling-based matrix decomposition methods, CRD achieves an execution time linear in m and n. Also, CRD does not require the whole data matrix be in the main memory. We conducted extensive experiments on both real and synthetic data. Compared with previous co-clustering algorithms, CRD achieves competitive accuracy but with much less computational cost.
268487
Mining non-redundant high order correlations in binary data
Many approaches have been proposed to find correlations in binary data. Usually, these methods focus on pair-wise correlations. In biology applications, it is important to find correlations that involve more than just two features. Moreover, a set of strongly correlated features should be non-redundant in the sense that the correlation is strong only when all the interacting features are considered together. Removing any feature will greatly reduce the correlation.In this paper, we explore the problem of finding non-redundant high order correlations in binary data. The high order correlations are formalized using multi-information, a generalization of pairwise mutual information. To reduce the redundancy, we require any subset of a strongly correlated feature subset to be weakly correlated. Such feature subsets are referred to as Non-redundant Interacting Feature Subsets (NIFS). Finding all NIFSs is computationally challenging, because in addition to enumerating feature combinations, we also need to check all their subsets for redundancy. We study several properties of NIFSs and show that these properties are useful in developing efficient algorithms. We further develop two sets of upper and lower bounds on the correlations, which can be incorporated in the algorithm to prune the search space. A simple and effective pruning strategy based on pair-wise mutual information is also developed to further prune the search space. The efficiency and effectiveness of our approach are demonstrated through extensive experiments on synthetic and real-life datasets.
2768791
Finding representative set from massive data
In the information age, data is pervasive. In some applications, data explosion is a significant phenomenon. The massive data volume poses challenges to both human users and computers. In this project, we propose a new model for identifying representative set from a large database. A representative set is a special subset of the original dataset, which has three main characteristics: It is significantly smaller in size compared to the original dataset. It captures the most information from the original dataset compared to other subsets of the same size. It has low redundancy among the representatives it contains. We use information-theoretic measures such as mutual information and relative entropy to measure the representativeness of the representative set. We first design a greedy algorithm and then present a heuristic algorithm that delivers much better performance. We run experiments on two real datasets and evaluate the effectiveness of our representative set in terms of coverage and accuracy. The experiments show that our representative set attains expected characteristics and captures information more efficiently.
11841640
Accelerating Profile Queries in Elevation Maps
Elevation maps are a widely used spatial data representation in geographical information systems (GIS). Paths on elevation maps can be characterized by profiles, which describe relative elevation as a function of distance. In this research, we address the inverse of this mapping - given a profile, how to efficiently find paths that could have generated it. This is called the profile query problem. Profiles have a wide variety of uses that include registering tracking information, or even other maps, to a given map. We describe a probabilistic model to characterize the maximal likelihood that a point lying on a path matches the query profile. Propagation of such probabilities to neighboring points can effectively prune the search space. This model enables us to efficiently answer queries of arbitrary profiles with user-specified error tolerances. When compared to existing spatial index methods, our approach supports more flexible queries with orders of magnitude speedup.
108674
Sample Selection for Maximal Diversity
The problem of selecting a sample subset sufficient to preserve diversity arises in many applications. One example is in the design of recombinant inbred lines (RIL) for genetic association studies. In this context, genetic diversity is measured by how many alleles are retained in the resulting inbred strains. RIL panels that are derived from more than two parental strains, such as the collaborative cross (Churchill et al., 2004), present a particular challenge with regard to which of the many existing lab mouse strains should be included in the initial breeding funnel in order to maximize allele retention. A similar problem occurs in the study of customer reviews when selecting a subset of products with a maximal diversity in reviews. Diversity in this case implies the presence of a set of products having both positive and negative ranks for each customer. In this paper, we demonstrate that selecting an optimal diversity subset is an NP-complete problem via reduction to set cover. This reduction is sufficiently tight that greedy approximations to the set cover problem directly apply to maximizing diversity. We then suggest a slightly modified subset selection problem in which an initial greedy diversity solution is used to effectively prune an exhaustive search for all diversity subsets bounded from below by a specified coverage threshold. Extensive experiments on real datasets are performed to demonstrate the effectiveness and efficiency of our approach.
44456878
A 2-Stage Partial Distortion Search Algorithm for Block Motion Estimation
In this paper, we propose a novel 2-Stage Partial Distortion Search (2S-PDS) algorithm to reduce the computational complexity in block motion estimation algorithms. In this algorithm, an early-rejection stage is introduced where the partial distortion of a decimated pixel block is calculated and compared with its local minimum. A block is rejected without calculating the full distortion of the entire block if the partial distortion is larger. In order to reduce the probability of false rejection, the local minimum is amplified by a pre-defined threshold before the comparison. Experimental results show that the proposed algorithm can reduce the complexity of block motion estimation algorithm significantly with only marginal performance penalty. The proposed algorithm can be used in combination with full-search or other fast search algorithms.
18214781
Fast mode decision algorithm for intraprediction in H.264/AVC video coding
The H.264/AVC video coding standard aims to enable significantly improved compression performance compared to all existing video coding standards. In order to achieve this, a robust rate-distortion optimization (RDO) technique is employed to select the best coding mode and reference frame for each macroblock. As a result, the complexity and computation load increase drastically. This paper presents a fast mode decision algorithm for H.264/AVC intraprediction based on local edge information. Prior to intraprediction, an edge map is created and a local edge direction histogram is then established for each subblock. Based on the distribution of the edge direction histogram, only a small part of intraprediction modes are chosen for RDO calculation. Experimental results show that the fast intraprediction mode decision scheme increases the speed of intracoding significantly with negligible loss of peak signal-to-noise ratio.
24069960
Study of Improved Hierarchy Genetic Algorithm Based on Adaptive Niches
Canonical genetic algorithms have the defects of pre-maturity and stagnation when applied in optimizing problems. In order to avoid the shortcomings, an adaptive niche hierarchy genetic algorithm (ANHGA) is proposed. The algorithm is based on the adaptive mutation operator and crossover operator to adjust the crossover rate and probability of mutation of each individual, whose mutation values are decided using individual gradient. This approach is applied in Percy and Shubert function optimization. Comparisons of niche genetic algorithm (NGA), hierarchy genetic algorithm (HGA) and ANHGA have been done by establishing a simulation model and the results of mathematics model and actual industrial model show that ANHGA is feasible and efficient in the design of multi-extremum.
11110859
A locally-adaptive algorithm for measuring blocking artifacts in images and videos
Block transform coding is the most popular approach for image and video compression. The objective measurement of blocking artifacts plays an important role in the design, optimization, and assessment of image and video coding systems. This paper presents a new algorithm for measuring blocking artifacts in images and videos. It exhibits unique and useful features: 1) it examines the blocks individually so that it can measure the severity of blocking artifacts locally; 2) it is a one-pass algorithm in the sense that the image needs to be accessed only once; 3) it takes into account the blocking artifacts for high bit rate images and the flatness for the very low bit rate images; 4) the blocking artifacts measure is well-defined in the range of 0-10. Experiments on various still images and videos show that this blockiness measure is very efficient in terms of computational complexity and memory usage, and can produce consistent blocking artifacts measurement.
18783473
A multi-objective-based non-stationary UAV assignment model for constraints handling using PSO
An unmanned aerial vehicle (UAV) assignment requires allocating vehicles to destinations to complete various jobs. It is a complex assignment problem with hard constraints, and potential dimensional explosion when the scenarios become more complicated and the size of problems increases. Moreover, the non-stationary UAV assignment problem, studied in the paper, is more difficult, since dynamic scenarios are introduced, e.g. change of the number, or different task requirements of targets and vehicle, etc. In this paper, a "Constraint-First-Objective- Next" model is proposed for the non-stationary problem. The proposed model can effectively handle constraints as an additional objective, including constraints expressed by nature language, and is flexible enough to be combined with kinds of intelligent computation algorithms. A local version of PSO is cooperated with the proposed model to solve non-stationary UAV assignment problems. Numerical experimental results illustrate that it can efficiently achieve the optima and demonstrate the effectiveness.