The relation between Granger causality and directed information theory: a review <s> It is the aim of this report to review the conceptual and theoretical links between Granger causality and directed information theory. <s> An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared information due to common history and input signals. In our new approach, these influences are excluded by appropriate conditioning of transition probabilities. The resulting transfer entropy is able to distinguish effectively driving and responding elements and to detect asymmetry in the interaction of subsystems. <s> BIB001 </s> The relation between Granger causality and directed information theory: a review <s> It is the aim of this report to review the conceptual and theoretical links between Granger causality and directed information theory. <s> In this paper, we will present and review the most usual methods to detect linear and nonlinear causality between signals: linear Granger causality test (Geweke in J Am Stat Assoc 77:304–313, 1982) extended to direct causality in multivariate case (LGC), directed coherence (DCOH, Saito and Harashima in Recent advances in EEG and EMG data processing, Elsevier, Amsterdam, 1981), partial directed coherence (PDC, Sameshima and Baccala 1999) and nonlinear Granger causality test of Baek and Brock (in Working Paper University of Iowa, 1992) extended to direct causality in multivariate case (partial nonlinear Granger causality, PNGC). All these methods are tested and compared on several ARX, Poisson and nonlinear models, and on neurophysiological data (depth EEG). The results show that LGC, DCOH and PDC are not very robust in relation to nonlinear linkages but they seem to correctly find linear linkages if only the autoregressive parts are nonlinear. PNGC is extremely dependent on the choice of parameters. Moreover, LGC and PNGC may give misleading results in the case of causality on a spectral band, which is illustrated by our neurophysiological database. <s> BIB002 </s> The relation between Granger causality and directed information theory: a review <s> It is the aim of this report to review the conceptual and theoretical links between Granger causality and directed information theory. <s> Synchronization, a basic nonlinear phenomenon, is widely observed in diverse complex systems studied in physical, biological and other natural sciences, as well as in social sciences, economy and finance. While studying such complex systems, it is important not only to detect synchronized states, but also to identify causal relationships (i.e. who drives whom) between concerned (sub) systems. The knowledge of information-theoretic measures (i.e. mutual information, conditional entropy) is essential for the analysis of information flow between two systems or between constituent subsystems of a complex system. However, the estimation of these measures from a set of finite samples is not trivial. The current extensive literatures on entropy and mutual information estimation provides a wide variety of approaches, from approximation-statistical, studying rate of convergence or consistency of an estimator for a general distribution, over learning algorithms operating on partitioned data space to heuristical approaches. The aim of this paper is to provide a detailed overview of information theoretic approaches for measuring causal influence in multivariate time series and to focus on diverse approaches to the entropy and mutual information estimation. <s> BIB003
Many information-theoretic tools have been designed for the practical implementation of Granger causality ideas. We will not show all of the different measures proposed, because they are almost always particular cases of the measures issued from directed information theory. Furthermore, some measures might have been proposed in different fields (and/or at different periods of time) and have received different names. We will only consider the well-accepted names. This is the case, for example, of 'transfer entropy', as coined by Schreiber in 2000 BIB001 , but which appeared earlier under different names, in different fields, and might be considered under slightly different hypotheses. Prior to developing a unified view of the links between Granger causality and information theory, we will provide a survey of the literature, concentrating on studies where information theory and Granger causality are jointly presented. Furthermore, we will not review any practical aspects, nor any detailed applications. In this spirit, this report is different from BIB003 , which concentrated on the estimation of information quantities, and where the review is restricted to transfer entropy. For reviews on the analysis of dependencies between systems and for applications of Granger causality in neuroscience, we refer to  , BIB002 . We will mention however some important practical points in our conclusions, where we will also discuss some current and future directions of research in the field.