15076082
Simultaneous Gene Clustering and Subset Selection for Sample Classification Via MDL
MOTIVATION
The microarray technology allows for the simultaneous monitoring of thousands of genes for each sample. The high-dimensional gene expression data can be used to study similarities of gene expression profiles across different samples to form a gene clustering. The clusters may be indicative of genetic pathways. Parallel to gene clustering is the important application of sample classification based on all or selected gene expressions. The gene clustering and sample classification are often undertaken separately, or in a directional manner (one as an aid for the other). However, such separation of these two tasks may occlude informative structure in the data. Here we present an algorithm for the simultaneous clustering of genes and subset selection of gene clusters for sample classification. We develop a new model selection criterion based on Rissanen's MDL (minimum description length) principle. For the first time, an MDL code length is given for both explanatory variables (genes) and response variables (sample class labels). The final output of the proposed algorithm is a sparse and interpretable classification rule based on cluster centroids or the closest genes to the centroids.


RESULTS
Our algorithm for simultaneous gene clustering and subset selection for classification is applied to three publicly available data sets. For all three data sets, we obtain sparse and interpretable classification models based on centroids of clusters. At the same time, these models give competitive test error rates as the best reported methods. Compared with classification models based on single gene selections, our rules are stable in the sense that the number of clusters has a small variability and the centroids of the clusters are well correlated (or consistent) across different cross validation samples. We also discuss models where the centroids of clusters are replaced with the genes closest to the centroids. These models show comparable test error rates to models based on single gene selection, but are more sparse as well as more stable. Moreover, we comment on how the inclusion of a classification criterion affects the gene clustering, bringing out class informative structure in the data.


AVAILABILITY
The methods presented in this paper have been implemented in the R language. The source code is available from the first author.
12165073
On the Convergence of Boosting Procedures
A boosting algorithm seeks to minimize empirically a loss function in a greedy fashion. The resulted estimator takes an additive function form and is built iteratively by applying a base estimator (or learner) to updated samples depending on the previous iterations. This paper studies convergence of boosting when it is carried out over the linear span of a family of basis functions. For general loss functions, we prove the convergence of boosting's greedy optimization to the infinimum of the loss function over the linear span. As a side product, these results reveal the importance of restricting the greedy search step sizes, as known in practice through the works of Friedman and others.
6762008
Perceptual audio coding using adaptive pre- and post-filters and lossless compression
This paper proposes a versatile perceptual audio coding method that achieves high compression ratios and is capable of low encoding/decoding delay. It accommodates a variety of source signals (including both music and speech) with different sampling rates. It is based on separating irrelevance and redundancy reductions into independent functional units. This contrasts traditional audio coding where both are integrated within the same subband decomposition. The separation allows for the independent optimization of the irrelevance and redundancy reduction units. For both reductions, we rely on adaptive filtering and predictive coding as much as possible to minimize the delay. A psycho-acoustically controlled adaptive linear filter is used for the irrelevance reduction, and the redundancy reduction is carried out by a predictive lossless coding scheme, which is termed weighted cascaded least mean squared (WCLMS) method. Experiments are carried out on a database of moderate size which contains mono-signals of different sampling rates and varying nature (music, speech, or mixed). They show that the proposed WCLMS lossless coder outperforms other competing lossless coders in terms of compression ratios and delay, as applied to the pre-filtered signal. Moreover, a subjective listening test of the combined pre-filter/lossless coder and a state-of-the-art perceptual audio coder (PAC) shows that the new method achieves a comparable compression ratio and audio quality with a lower delay.
116839195
Some results on empirical processes and stochastic complexity
This thesis contains two parts. In Part I, an intuitive and simple approach is used to obtain uniform convergence and Central Limit Theorems for empirical processes indexed by families of functions, for stationary mixing sequences with not too slow a $\beta$-mixing (or $\phi$-mixing) rate. The conditions imposed are in terms of random metric entropies, and the $\beta$-mixing (or $\phi$-mixing) rate. In the case of a Vapnik-Cervonenkis class, the uniform convergence result holds for any strictly stationary sequence with $\beta$-mixing (or $\phi$-mixing) rate $\beta\sb{n}$ (or $\phi\sb{n}$) = $o$($n\sp{-d}$) for some d $>$ 0, while the Central Limit Theorem requires the rate $\beta\sb{n}$ (or $\phi\sb{n}$ = $o$($n\sp{-d}$) for some d $>$ 3. 
In Part II, we use a shortest code length criterion to select the bin width of a histo-gram from a string of i.i.d. observations. Under smoothness conditions on the underlying density on (0, 1), this criterion is shown to be optimal as a code selection criterion in reaching the smallest redundancy within the class of codes under consideration. Furthermore, the criterion consistently selects the underlying density when it is uniform on (0, 1). 
Minimax lower bounds for the redundancy over many smooth families of underlying densities are also derived. In a special case, the lower bound is shown to be achieved almost surely, and in expectation, by a predictive code based on a sequence of histograms. In the case of Sobolev balls, the minimax optimal code is described and a heuristic proof is provided.
2105049
Model Selection in Gaussian Graphical Models: High-Dimensional Consistency of l1-regularized MLE
We consider the problem of estimating the graph structure associated with a Gaussian Markov random field (GMRF) from i.i.d. samples. We study the performance of study the performance of thel1-regularized maximum likelihood estimator in the high-dimensional setting, where the number of nodes in the graph p, the number of edges in the graph s and the maximum node degree d, are allowed to grow as a function of the number of samples n. Our main result provides sufficient conditions on (n, p, d) for thel1-regularized MLE estimator to recover all the edges of the graph with high probability. Under some conditions on the model covariance, we show that model selection can be achieved for sample sizes n = Ω(d log(p)), with the error decaying as O(exp(−c log(p))) for some constant c. We illustrate our theoretical results via simulations and show good correspondences between the theoretical predictions and behavior in simulations.
828878
Image denoising via lossy compression and wavelet thresholding
Some past work has proposed to use lossy compression to remove noise, based on the rationale that a reasonable compression method retains the dominant signal features more than the randomness of the noise. Building on this theme, we explain why compression (via coefficient quantization) is appropriate for filtering noise from signal by making the connection that quantization of transform coefficients approximates the operation of wavelet thresholding for denoising. That is, denoising is mainly due to the zero-zone and that the full precision of the thresholded coefficients is of secondary importance. The method of quantization is facilitated by a criterion similar to Rissanen's minimum description length principle. An important issue is the threshold value of the zero-zone (and of wavelet thresholding). For a natural image, it has been observed that its subband coefficients can be well modeled by a Laplacian distribution. With this assumption, we derive a threshold which is easy to compute and is intuitive. Experiments show that the proposed threshold performs close to optimal thresholding.
15309050
Prediction of Transmembrane Helical Segments in Membrane Proteins Using Back Propagation Neural Network with Wavelet Analysis
Transmembrane proteins possess important physiological function. The increasing transmembrane protein sequences from genome projects raise the needs for theoretical methods to predict their structures, especially transmembrane helical segments (TMHs). wnnTM, a new method of BP neural network based on wavelet multiresolution analysis (MRA) is initially developed to predict the number and location of TMHs in membrane proteins. 80 proteins with known 3D-structure randomly selected from Mptopo database are used as test set to evaluate the prediction accuracy of the method. The final results indicate that the proposed method is more effective than BP neural network model only.
1248870
An incentive mechanism for message relaying in unstructured peer-to-peer systems
Distributed message relaying is an important function of a peer-to-peer system to discover service providers. Existing search protocols in unstructured peer-to-peer systems either create huge burden on communications or cause long response time. Moreover, these systems are also vulnerable to the free riding problem. In this paper we present an incentive mechanism that not only mitigates the free riding problem, but also achieves good system efficiency in message relaying for peer discovery. In this mechanism promised rewards are passed along the message propagation process. A peer is rewarded if a service provider is found via a relaying path that includes this peer. We provide some analytic insights to the symmetric Nash equilibrium strategies of this game, and an approximate approach to calculate this equilibrium. Experiments show that this incentive mechanism brings a system utility generally higher than breadth-first search and random walks, based on both the estimated utility from our approximate equilibrium and the utility generated from learning in the incentive mechanism.
41743230
Automatically wrapping legacy software into services: A grid case study
The computational grid is rapidly evolving into a service-oriented computing infrastructure that facilitates resource sharing for solving large-scale data and computationally intensive problems. Peer-to-peer (P2P) systems have emerged as an infrastructure enabling technologies for enhanced scalability and reliability in file sharing and content distribution. It is envisioned that P2P enabled service-oriented grid systems would virtualize various resources as services with high scalability and reliability. Many legacy software resources exist nowadays, but making them grid aware services for effective resource sharing has become an issue of vital importance. This paper presents GSLab, a toolkit for automatically wrapping legacy software into services that can be published, discovered and reused in grid environments. GSLab employs Sun Grid Engine (SGE) to enhance its performance in execution of wrapped services. Using GSLab, we have automatically wrapped a legacy computer animation rendering code written in C as a service that can be discovered and accessed in a SGE environment. The evaluation results show that the performance of GSLab improves with an increase in the number of computing nodes involved.
205431881
An integrated approach to high-level information fusion
In today's fast paced military operational environment, vast amounts of information must be sorted out and fused not only to allow commanders to make situation assessments, but also to support the generation of hypotheses about enemy force disposition and enemy intent. Current information fusion technology has the following two limitations. First, current approaches do not consider the battlefield context as a first class entity. In contrast, we consider situational context in terms of terrain analysis and inference. Second, there are no integrated and implemented models of the high-level fusion process. This paper describes the HiLIFE (High-Level Information Fusion Environment) computational framework for seamless integration of high levels of fusion (levels 2, 3 and 4). The crucial components of HiLIFE that we present in this paper are: (1) multi-sensor fusion algorithms and their performance results that operate in heterogeneous sensor networks to determine not only single targets but also force aggregates, (2) computational approaches for terrain-based analysis and inference that automatically combine low-level terrain features (such as forested areas, rivers, etc.) and additional information, such as weather, and transforms them into high-level militarily relevant abstractions, such as NO-GO, SLOW-GO areas, avenues of approach, and engagement areas, (3) a model for inferring adversary intent by mapping sensor readings of opponent forces to possible opponent goals and actions, and (4) sensor management for positioning intelligence collection assets for further data acquisition. The HiLIFE framework closes the loop on information fusion by specifying how the different components can computationally work together in a coherent system. Furthermore, the framework is inspired by a military process, the Intelligence Preparation of the Battlefield, that grounds the framework in practice. HiLIFE is integrated with a distributed military simulation system, OTBSAF, and the RETSINA multi-agent infrastructure to provide agile and sophisticated reasoning. In addition, the paper presents validation results of the automated terrain analysis that were obtained through experiments using military intelligence Subject Matter Experts (SMEs).
41609314
A robust and fast skew detection algorithm for generic documents
Abstract A robust and fast skew detection algorithm based on hierarchical Hough transform is proposed. It is capable of detecting the skew angle for various document images, including technical articles, postal labels, handwritten text, forms, drawings and bar codes. The algorithm is robust even when black margins introduced by photocopying are present in the image and when the document is scanned at a low resolution of 50 dpi. The algorithm consists of two steps. In the first step we quickly extract the centroids of connected components using a graph data structure. Then, a hierarchical Hough transform (at two different angular resolutions) is applied to the selected centroids. The skew angle corresponds to the location of the highest peak in the Hough space. The performance of the algorithm is shown on a number of document images collected from various application domains. The algorithm is not very sensitive to algorithmic parameters. For an A4 size document image scanned at 50 dpi (typically 413 × 575 pixels), our algorithm is able to detect the skew angle with an accuracy of 0.1° in 0.4s of CPU time on a SunSparc 20 workstation.