19636079
Information In The Non-Stationary Case
Information estimates such as the ``direct method'' of Strong et al. (1998) sidestep the difficult problem of estimating the joint distribution of response and stimulus by instead estimating the difference between the marginal and conditional entropies of the response. While this is an effective estimation strategy, it tempts the practitioner to ignore the role of the stimulus and the meaning of mutual information. We show here that, as the number of trials increases indefinitely, the direct (or ``plug-in'') estimate of marginal entropy converges (with probability 1) to the entropy of the time-averaged conditional distribution of the response, and the direct estimate of the conditional entropy converges to the time-averaged entropy of the conditional distribution of the response. Under joint stationarity and ergodicity of the response and stimulus, the difference of these quantities converges to the mutual information. When the stimulus is deterministic or non-stationary the direct estimate of information no longer estimates mutual information, which is no longer meaningful, but it remains a measure of variability of the response distribution across time.
34993492
Microarray image compression: SLOCO and the effect of information loss
Microarray image technology is a powerful tool for monitoring the expression of thousands of genes simultaneously. Each microarray experiment produces immense amounts of image data, and efficient storage and transmission require compression that takes advantage of microarray image structure. In this paper we develop a compression scheme for microarray images which can be either lossless or lossy with successive refinements. Existing measures of distortion such as mean squared pixel-wise error and visual fidelity are not appropriate for microarray images. We introduce a new measure of distortion for lossy compression: the sensitivity of microarray information extraction to compression loss. Furthermore, our scheme has a coded data structure that allows fast decoding and reprocessing of image sub-blocks, and includes summary statistics and image segmentation information. The average lossless compression ratio is 1.83:1 for our cDNA test images and 2.43:1 for our inkjet test images, comparable or better than state-of-the-art lossless schemas, yet with additional structure and information. At an average lossy compression ratio of 8:1 for cDNA microarrays, we find that our scheme minimizes the effects of compression loss compared to other algorithms. We show that the variability in differential gene expression levels extracted from lossily vs. losslessly compressed microarray images is less than both the variability between different arrays and the variability between different extraction algorithms. In fact, lossy compression can improve the estimation of gene expression levels for cDNA images.
35646601
Iterated logarithmic expansions of the pathwise code lengths for exponential families
Rissanen's (1978, 1986, 1989, 1996) minimum description length (MDL) principle is a statistical modeling principle motivated by coding theory. For exponential families we obtain pathwise expansions, to the constant order, of the predictive and mixture codelengths used in MDL. The results are useful for understanding different MDL forms.
2105049
Model Selection in Gaussian Graphical Models: High-Dimensional Consistency of l1-regularized MLE
We consider the problem of estimating the graph structure associated with a Gaussian Markov random field (GMRF) from i.i.d. samples. We study the performance of study the performance of thel1-regularized maximum likelihood estimator in the high-dimensional setting, where the number of nodes in the graph p, the number of edges in the graph s and the maximum node degree d, are allowed to grow as a function of the number of samples n. Our main result provides sufficient conditions on (n, p, d) for thel1-regularized MLE estimator to recover all the edges of the graph with high probability. Under some conditions on the model covariance, we show that model selection can be achieved for sample sizes n = Ω(d log(p)), with the error decaying as O(exp(−c log(p))) for some constant c. We illustrate our theoretical results via simulations and show good correspondences between the theoretical predictions and behavior in simulations.
17350574
Nonparametric sparse hierarchical models describe V1 fMRI responses to natural images
We propose a novel hierarchical, nonlinear model that predicts brain activity in area V1 evoked by natural images. In the study reported here brain activity was measured by means of functional magnetic resonance imaging (fMRI), a noninvasive technique that provides an indirect measure of neural activity pooled over a small volume (≈ 2mm cube) of brain tissue. Our model, which we call the V-SPAM model, is based on the reasonable assumption that fMRI measurements reflect the (possibly nonlinearly) pooled, rectified output of a large population of simple and complex cells in V1. It has a hierarchical filtering stage that consists of three layers: model simple cells, model complex cells, and a third layer in which the complex cells are linearly pooled (called "pooled-complex" cells). The pooling stage then obtains the measured fMRI signals as a sparse additive model (SpAM) in which a sparse nonparametric (nonlinear) combination of model complex cell and model pooled-complex cell outputs are summed. Our results show that the V-SPAM model predicts fMRI responses evoked by natural images better than a benchmark model that only provides linear pooling of model complex cells. Furthermore, the spatial receptive fields, frequency tuning and orientation tuning curves of the V-SPAM model estimated for each voxel appears to be consistent with the known properties of V1, and with previous analyses of this data set. A visualization procedure applied to the V-SPAM model shows that most of the nonlinear pooling consists of simple compressive or saturating nonlinearities.
116839195
Some results on empirical processes and stochastic complexity
This thesis contains two parts. In Part I, an intuitive and simple approach is used to obtain uniform convergence and Central Limit Theorems for empirical processes indexed by families of functions, for stationary mixing sequences with not too slow a $\beta$-mixing (or $\phi$-mixing) rate. The conditions imposed are in terms of random metric entropies, and the $\beta$-mixing (or $\phi$-mixing) rate. In the case of a Vapnik-Cervonenkis class, the uniform convergence result holds for any strictly stationary sequence with $\beta$-mixing (or $\phi$-mixing) rate $\beta\sb{n}$ (or $\phi\sb{n}$) = $o$($n\sp{-d}$) for some d $>$ 0, while the Central Limit Theorem requires the rate $\beta\sb{n}$ (or $\phi\sb{n}$ = $o$($n\sp{-d}$) for some d $>$ 3. 
In Part II, we use a shortest code length criterion to select the bin width of a histo-gram from a string of i.i.d. observations. Under smoothness conditions on the underlying density on (0, 1), this criterion is shown to be optimal as a code selection criterion in reaching the smallest redundancy within the class of codes under consideration. Furthermore, the criterion consistently selects the underlying density when it is uniform on (0, 1). 
Minimax lower bounds for the redundancy over many smooth families of underlying densities are also derived. In a special case, the lower bound is shown to be achieved almost surely, and in expectation, by a predictive code based on a sequence of histograms. In the case of Sobolev balls, the minimax optimal code is described and a heuristic proof is provided.
5803741
Incentive Mechanisms for Peer-to-Peer Systems
Most of the existing research in peer-to-peer systems focuses on protocol design and doesn’t consider the rationality of each peer. One phenomenon that should not be ignored is free riding. Some peers simply consume system resources but contribute nothing to the system. In this paper we present an agent-based peer-to-peer system, in which each peer is a software agent and the agents cooperate to search the whole system through referrals. We present a static and a dynamic pricing mechanism to motivate each agent to behave rationally while still achieving good overall system performance. We study the behavior of the agents under two pricing mechanisms and evaluate the impact of free riding using simulations.
15008871
Large Sample Properties of Weighted Monte Carlo Estimators
A general approach to improving simulation accuracy uses information about auxiliary control variables with known expected values to improve the estimation of unknown quantities. We analyze weighted Monte Carlo estimators that implement this idea by applying weights to independent replications. The weights are chosen to constrain the weighted averages of the control variables. We distinguish two cases (unbiased and biased), depending on whether the weighted averages of the controls are constrained to equal their expected values or some other values. In both cases, the number of constraints is usually smaller than the number of replications, so there may be many feasible weights. We select maximally uniform weights by minimizing a separable convex function of the weights subject to the control variable constraints. Estimators of this form arise (sometimes implicitly) in several settings, including at least two in finance: calibrating a model to market data (as in the work of Avellaneda et al. 2001) and calculating conditional expectations to price American options. We analyze properties of these estimators as the number of replications increases. We show that in the unbiased case, weighted Monte Carlo reduces asymptotic variance, and that all convex objective functions within a large class produce estimators that are very close to each other in a strong sense. In contrast, in the biased case the choice of objective function does matter. We show explicitly how the choice of objective determines the limit to which the estimator converges.
15046055
Transform from Models to Service Description Based on MDA
Within MDA framework, system models can be built up on an abstract level that is independent from the platform. Later on, this abstract model can be transmitted into service description, which is related to specific platform on the implementation level. In Web services applications, this transition has a great advantage. Because, while the model of implementation changes from one case to another to fit the different platforms, the model of whole system, on a higher abstract level, stays the same. Thus, the efficiency is promoted. This paper firstly discusses the extension of UML for Web services, and then it presents the modeling process of WSDL. Finally, it gives the rules and arithmetic of transition from models to WSDL documents in detail
20441735
Web Services Discovery with Rough Sets
Web services are emerging as a major technology for building service-oriented distributed systems. Potentially, various resources on the Internet can be virtualized as Web services for a wider use by their communities. Service discovery becomes an issue of vital importance for Web services applications. This article presents ROSSE, a Rough Sets based Search Engine for Web service discovery. One salient feature of ROSSE lies in its capability to deal with uncertainty of service properties when matching services. A use case is presented to demonstrate the use of ROSSE for discovery of car services. ROSSE is evaluated in terms of its accuracy and efficiency in service discovery.
10885412
Managing the pedigree and quality of information in dynamic information sharing environments
The quality of information is crucial for decision making in many mission-critical applications such as battlefield operations and intelligence analysis. However, as the system becomes larger and more diverse, it is becoming increasingly difficult to assess the quality of information from various operators or data sources. In this paper we propose an agent-based approach to managing the quality of information, e.g., its trustworthiness, in network centric information sharing environments, where software agents collaborate with each other to automatically represent and assess the trustworthiness of information from its pedigree within the framework of Dempster-Shafer theory.