A detailed algorithmic explanation is required for how a network of chemical reactions can generate the sophisticated behavior displayed by living cells. Though several previous works have shown that reaction networks are computationally universal and can in principle implement any algorithm, there is scope for constructions that map well onto biological reality, make efficient use of the computational potential of the native dynamics of reaction networks, and make contact with statistical mechanics. We describe a new reaction network scheme for solving a large class of statistical problems. Specifically we show how reaction networks can implement information projection, and consequently a generalized Expectation-Maximization algorithm, to solve maximum likelihood estimation problems in partially-observed exponential families on categorical data. Our scheme can be thought of as an algorithmic interpretation of E. T. Jaynes’s vision of statistical mechanics as statistical inference.
Similarities in the mathematical structure of statistics, statistical mechanics, and information theory have been noticed and remarked upon several times @cite_8 @cite_27 @cite_5 @cite_35 , and have led to multiple contributions @cite_12 @cite_0 @cite_14 @cite_28 @cite_17 @cite_6 with the goal of presenting some or all of these topics from the point of view from which the subject appears in the greatest simplicity,'' to borrow a prescient quote from J. W. Gibbs. Our EM reaction network scheme employs a statistical mechanical system to minimize an information theoretic quantity in the service of solving a statistical problem. It is a concrete illustration of the connections between these three disciplines, and of the opportunities that lie at their intersection.
@cite_35 @cite_14 @cite_8 @cite_28 @cite_6 @cite_0 @cite_27 @cite_5 @cite_12 @cite_17
2164998314 1545211435 1995294201 2117093731
To comprehend the multipartite organization of large-scale biological and social systems, we introduce an information theoretic approach that reveals community structure in weighted and directed networks. We use the probability flow of random walks on a network as a proxy for information flows in the real system and decompose the network into modules by compressing a description of the probability flow. The result is a map that both simplifies and highlights the regularities in the structure and their relationships. We illustrate the method by making a map of scientific communication as captured in the citation patterns of >6,000 journals. We discover a multicentric organization with fields that vary dramatically in size and degree of integration into the network of science. Along the backbone of the network—including physics, chemistry, molecular biology, and medicine—information flows bidirectionally, but the map reveals a directional pattern of citation from the applied fields to the basic sciences. * Recasts topics in random fields by following a completely new way of handling both geometry and probability * Significant exposition of the work of others in the field * Presentation is clear and pedagogical * Excellent reference work as well as excellent work for self study @PARASPLIT This monograph is devoted to a completely new approach to geometric problems arising in the study of random fields. The groundbreaking material in Part III, for which the background is carefully prepared in Parts I and II, is of both theoretical and practical importance, and striking in the way in which problems arising in geometry and probability are beautifully intertwined. @PARASPLIT The three parts to the monograph are quite distinct. Part I presents a user-friendly yet comprehensive background to the general theory of Gaussian random fields, treating classical topics such as continuity and boundedness, entropy and majorizing measures, Borell and Slepian inequalities. Part II gives a quick review of geometry, both integral and Riemannian, to provide the reader with the material needed for Part III, and to give some new results and new proofs of known results along the way. Topics such as Crofton formulae, curvature measures for stratified manifolds, critical point theory, and tube formulae are covered. In fact, this is the only concise, self-contained treatment of all of the above topics, which are necessary for the study of random fields. The new approach in Part III is devoted to the geometry of excursion sets of random fields and the related Euler characteristic approach to extremal probabilities. @PARASPLIT "Random Fields and Geometry" will be useful for probabilists and statisticians, and for theoretical and applied mathematicians who wish to learn about new relationships between geometry and probability. It will be helpful for graduate students in a classroom setting, or for self-study. Finally, this text will serve as a basic reference for all those interested in the companion volume of the applications of the theory. These applications, to appear in a forthcoming volume, will cover areas as widespread as brain imaging, physical oceanography, and astrophysics. While the use of statistical physics methods to analyze large corpora has been useful to unveil many patterns in texts, no comprehensive investigation has been performed on the interdependence between syntactic and semantic factors. In this study we propose a framework for determining whether a text (e.g., written in an unknown alphabet) is compatible with a natural language and to which language it could belong. The approach is based on three types of statistical measurements, i.e. obtained from first-order statistics of word properties in a text, from the topology of complex networks representing texts, and from intermittency concepts where text is treated as a time series. Comparative experiments were performed with the New Testament in 15 different languages and with distinct books in English and Portuguese in order to quantify the dependency of the different measurements on the language and on the story being told in the book. The metrics found to be informative in distinguishing real texts from their shuffled versions include assortativity, degree and selectivity of words. As an illustration, we analyze an undeciphered medieval manuscript known as the Voynich Manuscript. We show that it is mostly compatible with natural languages and incompatible with random texts. We also obtain candidates for keywords of the Voynich Manuscript which could be helpful in the effort of deciphering it. Because we were able to identify statistical measurements that are more dependent on the syntax than on the semantics, the framework may also serve for text analysis in language-dependent applications. We consider the problem of signal estimation (denoising) from a statistical mechanical perspective, using a relationship between the minimum mean square error (MMSE), of estimating a signal, and the mutual information between this signal and its noisy version. The paper consists of essentially two parts. In the first, we derive several statistical-mechanical relationships between a few important quantities in this problem area, such as the MMSE, the differential entropy, the Fisher information, the free energy, and a generalized notion of temperature. We also draw analogies and differences between certain relations pertaining to the estimation problem and the parallel relations in thermodynamics and statistical physics. In the second part of the paper, we provide several application examples, where we demonstrate how certain analysis tools that are customary in statistical physics, prove useful in the analysis of the MMSE. In most of these examples, the corresponding statistical-mechanical systems turn out to consist of strong interactions that cause phase transitions, which in turn are reflected as irregularities and discontinuities (similar to threshold effects) in the behavior of the MMSE.