neural systems process information .this processing is of fundamental biological importance for all animals and humans alike as its main ( if not sole ) biological purpose is to ensure the survival of an individual ( in the short run ) and its species ( in the long run ) in a given environment by means of perception , cognition , action and adaption .information enters a neural system in form of sensory input representing some aspect of the outside world , perceivable by the sensory modalities present in the system . after processing this information or parts of it ,the system may then adjust its state and act according to a perceived change in the environment .this general model is applicable to very basic acts of cognition as well as to ones requiring higher degrees of cognitive processing .yet , the underlying principle is the same . thus measuring , modeling and ( in the long run )understanding information processing in neural systems is of prime importance for the goal of gaining insight to the functioning of neural systems on a theoretical level .note that this question is of theoretical and abstract nature so that we take an abstract view on information in what follows .we use shannon s theory of information as a tool that provides us with a rigid mathematical theory and quantitative measures of information .using information theory , we will have a conceptual look at information in neural systems .in this context , information theory can provide both explorative and normative views on the processing of information in a neural system as we will see in section [ sec : neuralsystems ] . in some cases , it is even possible to gain insights on the nature of the `` neural code '' , i.e. the way neurons transmit information via their spiking activity .information theory was originally used to analyze and optimize man - made communication systems , for which the functioning principles are known . none the less, it was soon realized that the theory could also be used in a broader setting , namely to gain insight into the functioning of systems for which the underlying principles are far from fully understood , such as neural systems for example .this was the beginning of the success story of information - theoretic methods in many fields of science such as economics , psychology , biology , chemistry , physics and many more .the idea of using information theory to quantitatively assess information processing in neural systems has been around since the 1950s , see the works of attneave , barlow and eckhorn and ppel . yet , as information - theoretic analyses are data - intensive , these methods were rather heavily restricted by ( a ) the limited resources of computer memory and computational power available and ( b ) the limited accuracy and amount of measured data that could be obtained from neural systems ( on the single cell as well as at the systems level ) at that time .however , given the constant rise in available computing power and the evolution and invention of data acquisition techniques that can be used to obtain data from neural systems ( such magnetoencephalography ( meg ) , functional magnetic resonance imaging ( fmri ) or calcium imaging ) , information - theoretic analyses of all kinds of biological and neural systems became more and more feasible and could be carried out with greater accuracy and for larger and larger ( sub-)systems .  over the last decades such analyses became possible using an average workstation computer , a situation that could only be dreamed of in the 1970s . additionally , the emergence of new non - invasive data - collection methods such as fmri and meg that outperform more traditional methods like electroencephalography ( eeg ) in terms of spatial resultion ( fmri , meg ) or noise - levels ( meg ) made it possible to even obtain and analyze system - scale data of the human brain in vivo .  the goal of this chapter is to give a short introduction to the fundamentals of information theory and its application to data analysis problems in the neurosciences .and although information - theoretic analyses of neural systems were not often used in order to gain insight on or characterize neural dysfunction so far , this could prove to be a helpful tool in the future .the chapter is organized as follows .we first talk a bit about the process of modeling in section [ sec : setting ] that is fundamental for all what follows as it connects reality with theory .as information theory is fundamentally based on probability theory , following this we give an introduction to the mathematical notions of probabilities , probability distributions and random variables in section [ sec : probabilities ] .if you are familiar with probability theory , you may well skim or skip this section .section [ sec : informationentropy ] deals with the main ideas of information theory .we first take a view on what we mean by information and introduce the core concept of information theory , namely _entropy_. starting from the concept of entropy , we will then continue to look at more complex notions such as _ conditional entropy _ and _ mutual information _ in section [ sec : mutualinformation ] .we will then consider a variant of _ conditional mutual information _ called _ transfer entropy _ in section [ sec : transferentropy ] .we conclude the theoretical part by discussing methods used for the estimation of information - theoretic quantities from sampled data in section [ sec : estimation ] .what follows will deal with the application of the theoretical measures to neural data .we then give a short overview of applications of the discussed theoretical methods in the neurosciences in section [ sec : neuralsystems ] , and last ( but not least ) , section [ sec : software ] constrains a list of software packages that can be used to estimate information theoretic quantities for some given data set .in order to analyze the dynamics and gain a theoretical understanding of a given complex system , one usually defines a model first , i.e. a simplified theoretical version of the system to be investigated . the rest of the analysisis then based on this model and can only capture aspects of the system that are also contained in the model .thus , care has to be taken when creating the model as the following analysis crucially depends on the quality of the model .when building a model based on measured data , there is an important thing we have to pay attention to , namely that any data obtained by measurement of physical quantities is only accurate up to a certain degree and corrupted by noise .this naturally also holds for neural data ( e.g. electrophysiological single- or multi - cell measurements , eeg , fmri or meg data ) .therefore , when observing the state of some system by measuring it , one can only deduce the true state of the system up to a certain error determined by the noise in the measurement ( which may depend both on the measurement method and the system itself ) . in order to model this uncertainty in a mathematical way, one uses probabilistic models for the states of the measured quantities of a system .this makes probability theory a key ingredient to many mathematical models in the natural sciences .the roots of the mathematical theory of probability lie in the works of cardano , fermat , pascal , bernoulli and de moivre in the 16th and 17th century , in which the authors attempted to analyze games of chance .pascal and bernoulli were the first to treat the subject as a branch of mathematics , see for a historical overview .mathematically speaking , probability theory is concerned with the analysis of random phenomena . over the last centuries , it has become a well - established mathematical subject . for a more in - depth treatment of the subjectsee .let us consider an experiment that can produce a certain fixed number of outcomes ( say a coin toss , where the possible outcomes are heads or tails or the throw of a die where the die will show one of the numbers to ) .the set of all possible outcomes is called the _ sample space _ of the experiment .one possible result of an experiment is called _ outcome _ and a set of outcomes is called an _ event _ ( for the mathematically adept : an event is a subset of the power set of all outcomes ) .take for example the throw of a regular , -sided die as an experiment .the set of results in this case would be the set of natural numbers and examples of events are or corresponding to the events `` an odd number was thrown '' and `` an even number was thrown '' , respectively .the classical definition of the probability of an event is due to laplace : `` the probability of an event to occur is the number of cases favorable for the event divided by the number of total outcomes possible '' .  we thus assign each possible outcome a _ probability _ , a real number between and that is thought of as to describe how `` likely '' it is that the given event will occur , where means `` the event does nt ever occur '' and means `` the event always occurs '' .the sum of all the assigned numbers is restricted to be as we assume that one of our considered events always occurs . for the coin toss , the possible outcomes heads and tailsthus each have probability ( considering that the number of favourable outcomes is and the number of possible outcomes is ) and for the throw of a die this number is for each digit .this assumes that we have a so - called _ fair _ coin or die , i.e. one that does not favor any particular outcomes over the others .the probability of a given event to occur is then just the sum of the probabilities of the outcomes the event is composed of , e.g. when considering the throw of a die , the probability of the event `` an odd number is thrown '' is .such types of experiments in which all possible outcomes have the same probability ( they are called equiprobable ) are called _laplacian experiments_. the simplest case of an experiment not having equiprobable outcomes is the so called _bernoulli experiment_. here , two possible outcomes `` success '' and `` failure '' , with probabilities  ] over the real numbers .in fact , when choosing the set we can construct the so called _borel -algebra _ that contains all closed intervals  ] its length .the measure is called _borel measure_. it is the standard measure of length that we know from geometry and makes a measure space .this construction can easily be extended to arbitrary dimensions ( using closed sets ) resulting in the measure space that fulfills the properties of a -dimensional geometric measure of volume .let us look at some examples of measure spaces now :  1 .let , and with .this makes a measure space for our coin toss experiment .note that in this simple case , equals the full power set of .2 .  let and let with and , where denotes an arbitrary number between and .this makes a measure space .having understood the general case of a measure space , defining a probability space and a probability distribution is easy .a _ probability space _ is a measure space for which the measure is normed , i.e.  ] the expectation value of taken with respect to the probability distribution of .let us now look an example .if we consider the throw of a fair die with for each digit and take as the random variable that just assigns each digit its integer value , we get =\frac{1}{6 } ( 1+\dots+6)=3.5 ] , the variance -x)^2]=\frac{1}{2}\cdot ( 0-\frac{1}{2})^2 + \frac{1}{2}\cdot ( 1-\frac{1}{2})^2 = \frac{1}{4} ] and -(e[x])^2=\frac{91}{6}- \frac{49}{4}=\frac{35}{12}\approx 2.92 ] .its probability mass function is given by the two values and .the _ binomial probability distribution _ is a discrete probability distribution parametrized by two parameters and \subset \mathbb{r} ] with the expectation value taken with respect to , which in term can be interpreted as `` expected distance of and '' , measured in terms of the information content .another interpretation can be given in the language of codes : is the average number of extra bits needed to code samples from using a code book based on .analogous to previous examples , the kl - divergence can also be defined for continuous random variables in a straightforward way via    where and denote the pdf of two continuous probability distributions and .expanding the logarithm in equation [ eq : kldefinition ] we can write the kullback - leibler divergence between two probability distributions and in terms of entropies as    where and denote the pdf or pmf of the distributions and and is the so - called _ cross - entropy _ of and given by    this relation lets us easily compute a closed form of the kl - divergence for many common families of probability distributions .let us for example look at the value of the kl - divergence between two normal distributions and , see figure [ fig : klnorm ] .this can be calculated as    another example : the kl - divergence between two exponential distributions and is    using the kullback - leibler divergence we can give yet another characterization of mutual information : it is a measure of how far two measured variables are from being independent , this time in terms of the kullback - leibler divergence .  thus , mutual information of two random variables can be seen as the kl - divergence of their underlying joint probability distribution from the products of their marginal probability distributions , i.e. as a measure for how far the two variables are from being independent .   in the past, mutual information was often used as a measure of information transfer between units ( modeled as random variables ) in some system .this approach faces the problem that mutual information is a symmetric measure and does not have an inherent directionality . in some applications this symmetry is not desired though , namely whenever we want to explicitly obtain information about the `` direction of flow '' of information , for example to measure causality in an information - theoretic setting , see section [ sec : causality ] .  in order to make mutual information a directed measure ,a variant called _ time - lagged mutual information _ was proposed , calculating mutual information for two variables including a previous state of the source variable and a next state of the destination variable ( where discrete time is assumed ) .  yet , as schreiber points out , while time - lagged mutual information provides a directed measure of information transfer , it does not allow for a time - dynamic aspect as it measures the statically shared information between the two elements . with a suitable conditioning on the past of the variables ,the introduction of a time - dynamic aspect is possible though . the resulting quantity is commonly referred to as _ transfer entropy _ .its common definition is the following .let and be discrete random variables given on a discrete time scale and let be two natural numbers . then the _ transfer entropy from to with memory steps in and memory steps in is defined as    where we denoted by the value of and at time and by the past values of , counted from time on : , and analogously .although this definition might look complicated at first , the idea behind it is quite simple .it is merely the kullback - leibler divergence between the two conditional probability distributions and ,    i.e. a measure of how far the two distributions are from fulfilling the generalized markov property ( see section [ sec : stochasticproc ] )    note that for small values of transfer entropy , we can say that has little influence on at time , whereas we can say that information is transferred from to at time when the value is large . yet , keep in mind that transfer entropy is just a measure of statistical correlation , see section [ sec : causality ] .another interpretation of transfer entropy is seeing it as a conditional mutual information , measuring the average information the source constrains about the next state of the destination that was not contained in the destination s past ( see ) or alternatively as the average information provided by the source about the state transition in the destination , see .as so often before , the concept can be generalized to the continuous case , although the continuous setting introduces some subtleties that have to be addressed .from source to target at time as a measure of the average information present in about the future state .the memory vectors and are shown in gray . ]  concerning the memory - parameters and of the source and the destination , although arbitrary choices are possible , the values chosen fundamentally influence the nature of the questions asked . in order to get correct measures for systems being far from markovian( i.e. systems which states are not influenced by more than a certain fixed number of preceding system states ) , high values of have to be used , and for non - markovian systems the case has to be considered .on the other hand , commonly just one previous state of the source variable is considered , setting  , this being also due to the growing data intensity in and and the usually high computational cost of the method .note that akin to the case of mutual information , there exist point - wise versions of transfer entropy ( also called _ local transfer entropy _ ) , as well as extensions to the multivariate case , see .as we have seen in the preceding sections , one needs to know the full sample spaces and probability distributions of the random variables involved in order to precisely calculate information - theoretic quantities such as the entropy , mutual information or transfer entropy . butobtaining this data is in most cases impossible in reality , as the spaces are usually high - dimensional and sparsely sampled , rendering the direct methods for the calculation of such quantities impossible to carry out . a way around this problem is to come up with estimation techniques that estimate entropies and derived quantities such as mutual information from the data .over the last decades a large body of research was published concerning the estimation of entropies and related quantities , leading to a whole zoo of estimation techniques , each class having its own advantages and drawbacks .so rather than a full overview , we will give a sketch of some central ideas here and give references to further literature .the reader is also pointed to the review articles .  before looking at estimation techniques for neural ( andother ) data let us first give a swift and painless review some important theoretical concepts regarding statistical estimation .   from a statistical point of view, the process of estimation in its most general form can be regarded in the following setting : we have some data ( say measurements or data obtained via simulations ) that is believed to be generated by some stochastic process with an underlying non - autonomous , i.e. time - dependent , or autonomous probability distribution .we then want to estimate either the value of some function defined on that probability distribution ( for example the entropy ) or the shape of this probability distribution as a whole ( from which we can then obtain an estimate of a derived quantity ) .this process is called _ estimation _ and a function mapping the data to an estimated quantities estimator . in this sectionwill will first look at estimators and their desired properties and then look at what is called maximum likelihood estimation , the most commonly used method for the estimation of parameters in the field of statistics .let be a set of realizations of the random variable that is believed to have a probability distribution that comes from a family of probability distributions parametrized by a parameter and assume that the underlying probability distribution of is .let be an estimator for the parameter with the true value .for the value of the estimated parameter we usually write .the _ bias _ of is the expected difference between and : ,\ ] ] and an estimator with vanishing bias is called _unbiased_.  we usually also want the estimator to be _ consistent _ , i.e. we want the estimated value to converge to the value of the true parameter in probability as the sample increases in size , i.e. as :   another important property of an estimator is its variance and an unbiased estimator having the minimal variance among all unbiased estimators of the same parameter is called _yet another measure often used when assessing the quality of an estimator is its mean squared error    and as we can see , any unbiased estimator with minimal variance minimizes the mean squared error .  without further going into detail here ,it is noted that there exists a theoretical lower bound to the minimal variance obtainable by an unbiased estimator , the _ cramr - rao bound_. the cramr - rao bound sets the variance of the estimator in relation with the so called _ fisher information _ ( that can be set into relation with mutual information , see ) .the interested reader is pointed to ._ maximum likelihood estimation _ is the most - widely used estimation technique in statistics and , as we will see in the next few paragraphs , a straightforward procedure that in essence tells us what the most likely parameter value in an assumed family of probability distributions is , given a set of realizations of a random variable that is believed to have a underlying probability distribution from the family considered .  in statistical applicationsone often faces the following situation : we have a finite set of realizations of a random variable .we assume to have a probability distribution in a certain parametrized class of probability distributions , where the true parameter is unknown .the goal is to get an estimate of using the realizations , i.e. to do statistical inference of the parameter .let us consider the so called _ likelihood function _  as a function of .it is a measure of how likely it is that the parameter of the probability distribution has the value , given the observed realization of . in maximum likelihood estimation , we look for the parameter that maximizes the likelihood function .this is :    choosing a value of minimizes the kl - divergence between and for all possible values of . the value , often written as is called the _ maximum likelihood estimate _( mle for short ) of .  in thissetting , one often not uses the likelihood function directly , but works with the of the likelihood function ( this is referred to as log - likelihood ) .the likelihood functions are often very complicated and situated in high dimensions , making it impossible to find a maximum of the function analytically .thus , numerical methods ( such as newton s method and variants or the simplex method ) have to be employed in order to find a solution .these numerical methods work best ( and can be shown to converge to a unique solution ) if the function they operate on is concave ( bowl - shaped , where the closed end is on the top ) . the log - function has the property to make the likelihood function concave in many cases , that being the reason why one considers the log - likelihood function , rather than the likelihood function directly , see also .having looked at some core theoretical concepts regarding the estimation of quantities depending on probability distributions let us now come back to dealing with real data .  as in real - world data , the involved probability distributions are often continuous and infinite - dimensional , the resulting estimation problem is very difficult ( if not impossible ) to solve in its original setting . as a remedy ,the problem is often _regularized _ ,i.e. mapped to a discrete , more easily solvable problem .this of course introduces errors and often makes a direct estimation of the information - theoretic quantities impossible , but even in that simplified model we can estimate lower bounds of the quantities that by using shannon s _ information processing inequality _    where and are ( discrete ) random variables and and are measurable maps .  by choosing the mappings and as our regularization mappings ( you might also regard them as parameters ) we can change the coarseness of the regularization .the regularization can be chosen arbitrarily coarse , i.e. choosing and as constant functions , but this of course comes with a price .for example in the latter case of constant and the mutual information would be equal to , clearly not a very useful estimate .this means that a trade - off between complexity reduction and the quality of the estimation has to be made . in general, there exists no all - purpose recipe for this , each problem requiring an appropriate regularization .as this discretization technique has become the standard method in many fields , we will solely consider the regularized , discrete case in the following and point the reader to the review article concerning the continuous case .  in the neurosciences ,such a regularization technique was also proposed and is known as the `` direct method '' . here , spike trains of recorded neurons are discretized into time bins of a given fixed width and the neuronal spiking activity is interpreted as a series of symbols from an alphabet defined via the observed spiking pattern in the time bins .commonly , two different classes of estimation techniques regarding the shape of probability distributions are distinguished .parametric estimation techniques assume that the probability distribution is contained in some family of probability distributions having some prescribed shape ( see section [ sec : paramprobdistr ] ) . here , one estimates the value of the parameter from the data observed , whereas non - parametric estimation techniques make no assumptions about the shape of the underlying distribution .we will solely look at non - parametric estimation techniques in the following as in many cases one tries to not assume prior information about the shape of the distribution .histogram - based estimation is the most popular and most - widely used estimation technique .as the name implies , this method uses a histogram obtained from the data to estimate the probability distribution of the underlying random generation mechanism .  forthe following , assume that we obtained a finite set of samples of some real random variable defined over some probability space .we then divide the domain of into equally sized bins and subsequently count the number of realizations in our data set contained in each each bin . here , the number of bins can be freely chosen .it controls the coarseness of our discretization , where the limit is the continuous case .this allows us to define relative frequencies of occurrences for with respect to each bin that we interpret as estimations ( note that we make the dependence on the number of bins explicit in the notation ) of the probability of taking a value in bin which we denote by .the law of large numbers then tells us that our estimated probability values converge to the real probabilities as .note that although histogram - based estimations are usually called non - parametric as they do not assume a certain shape of the underlying probability distribution , they do have parameters , namely one parameter for each bin , the estimated probability value .these estimates can also be interpreted as maximum - likelihood estimates of .the following defines an estimator of the entropy based on the histogram .it is often called `` plug in '' estimator :    the are some problems with this estimator , though .its convergence to the true value can be slow and it is negatively biased , i.e. its value is almost always below the true value , see .this shift can be quite significant even for large , see figure [ fig : hest ] and .more specifically , one can show that the expected value of the estimated entropy is always smaller than the true value of the entropy ,  \leq h(x),\ ] ]  where the expectation value is taken with respect to the true probability distribution .bias generally is a problem for history - based estimation techniques and although we can correct for the bias , this may not always be a feasible solution .none the less we will have a look at a bias - corrected version of the estimator given in equation [ eq : hxmle ] below .   of the entropy of a given distribution with true entropy bits .estimated values are shown for three different sample sizes . adapted from , figure 1 . ]  as a remedy to the bias problem , miller and madow calculated the bias of the estimator of equation [ eq : hxmle ] and came up with a bias - corrected version of the maximum likelihood estimator for the entropy , referred to as _ miller - madow _ estimator :    where is an estimate of the number of bins with non - zero probability .we will not go into the detail of the method here , the interested reader is referred to .another way of bias - correction is the so called `` jack - knifed '' version of the maximum - likelihood estimator by efron and stein :    yet another bias - corrected variant of the mle estimator based on polynomial approximation is presented in , for which also bounds on the maximal estimation error were derived .  in an effort to overcome the problems faced by histogram - based estimation, many new and more powerful estimation techniques have emerged over the last years , both for entropy and other information - theoretic quantities . as our focus hereis to give an introduction to the field , we will not review all of those methods here but rather point the interested reader to the literature , where a variety of approaches are discussed .there exist methods based on the idea of adaptive partitioning of sample space , ones using entropy production rates and allowing for confidence intervals , ones using bayesian methods and ones based on density estimation using nearest - neighbors , along with many more .see for an overview concerning several estimation techniques for entropy and mutual information .we note here that in contrast to estimations of entropy , estimators of mutual information are usually positively biased , i.e. tend to overestimate mutual information .some time after its discovery by shannon , neuroscientists started to recognize information theory as a valuable mathematical tool to assess information processing in neural systems . using information theory , several questions regarding information processing and the neural codecan be addressed in a quantitative way , among those  * how much information single cells or populations carry about a stimulus and how this information is coded , * what aspects of a stimulus are encoded in the neural system and * how `` effective connectivity '' in neural systems can be defined via causal relationships between units in the system .see figure [ fig : shannonchannel - neuron ] for an illustration of how shannon s theory can be used in a neural setting .attneave and barlow were the first to consider information processing in neural systems from an information - theoretic point of view .subsequently , eckhorn and ppel  applied information - theoretic methods to electrophysiologically recorded data of neurons in a cat . but being data - intensive in nature these methods faced some quite strong restrictions during that time , namely the limited amount of computing power ( and computer memory ) and the limited amount ( and often low quality ) of data obtainable via measurements at that time .but over the last decades , available computing became more and more available and classical measurement techniques were improved , along with new ones emerging such as fmri , meg and calcium imaging .this made information theoretic analyses of neural systems more and more feasible and through the invention of recording techniques such as meg and fmri it is nowadays even possible to perform such analyses on a system - scale for the human brain in vivo . yet , even with the newly available recording techniques today there are some conceptual difficulties with information - theoretic analyses as it is often a challenge to obtain enough data in order to get good estimates of information - theoretic quantities .special attention has to be paid to using the data efficiently and the validity of such analyses has to be assessed to their statistical significance .  in the following we will discuss some conceptual questions relevant when regarding information theoretic analyses of neural systems .more detailed reviews can be found in .marr described `` three levels at which any machine carrying out an information - processing task must be understood '' [chapter 1.2 ] .they are :  1 .  computational theory : what is the goal of the computation , why is it appropriate , and what is the logic of the strategy by which it can be carried out ? 2 .  representation and algorithm: how can this computational theory be implemented ?in particular , what is the representation for the input and output , and what is the algorithm for the transformation ?hardware implementation : how can the representation and algorithm be realized physically ?when performing an information - theoretic analysis of a system one naturally faces the fundamental problem related to the coding of the information : in order to calculate ( i.e. estimate ) information theoretic quantities , one has to define a family of probability distributions over the state space of the system , each member of that family describing one system state that is to be considered .as we know , all information - theoretic quantities such as entropy and mutual information ( between the system state and the state of some external quantity ) are determined by the probability distributions involved .the big question now is how to define the system state in the first point , a question which is especially difficult to answer in the case of neural systems on all scales .one possible way to construct such a probabilistic model for a sensory neurophysiological experiment involving just one neuron is the following . typically , the experiment consists of many trials , where per trial in some defined time window a stimulus is presented eliciting a neural response consisting of a sequence of action potentials . presenting the same stimulus many timesallows for the definition of a probability distribution of responses of the neuron to a stimulus .this is modeled as a conditional probability distribution .as noted earlier , we usually have no direct access to but rather have to find an estimate from the available data .note that in practice , usually the joint probability distribution is estimated and estimates of conditional probability distributions are subsequently obtained from the estimate of the joint distribution .let us now assume that the stimuli are drawn from the set of stimuli according to some probability distribution ( that can be freely chosen by the experimenter ) .we can then compute the mutual information between the stimulus ensemble and its elicited response     using the probability distributions and , see section [ sec : mutualinformation ] .  as usual , by mutual informationwe assess the expected shared information between the stimulus and its elicited response averaged over all stimuli and responses . in order to break this down to the level of single stimuli we can either consider the point - wise mutual information or employ one of the proposed decompositions of mutual information such as _ stimulus - specific information _ or _ stimulus - specific surprise _ , see for a review .  having sketched the general setting let us come back to the question of coding of information by the neurons involved .this is important as we have to adjust our model of the neural responses accordingly , the goal being to capture all relevant features of the neural response in the model .  regarding neural coding ,there are two main hypotheses of how single neurons might code information : neurons could use a _ rate code _ , i.e. encode the information via their mean firing rates neglecting the timing patterns of spikes orthey could employ a _ temporal code _, i.e. a code where the precise timing of single spikes plays an important role .yet another hypothesis would be that neurons code information in bursts of spikes , i.e. groups of spikes emitted in a small time window , which is a variant of the time code . for questions regarding coding in populationssee the review .note that the question of neural coding is a highly debated one in the neurosciences as of today ( see ) and we do not want to favor one view point over the other in the following . as with many things in naturethere does not seem to be a clear black an white picture regarding neuronal coding .rather it seems that a gradient of different coding schemes is employed depending on which sensory system is considered and at which stage of neuronal processing , see .let us now compute the entropy of spike trains and subsequently single spikes , assuming that the neurons we model either employ a rate or a time code .we are especially interested in the maximal entropy attainable by our model spike trains as these can give us upper bounds for the amount of information such trains and even single spikes can carry in theory .the following examples here are adapted from . concerning the topics of spike trains and their analysis , the interested reader is also pointed to .first , we define a model for the spike train emitted by a neuron measured for some fixed time interval of length .we can consider two different models for the spike train , a continuous and a discrete one . in the continuous case , we model each spike by a dirac delta function and the whole spike train as a combination of such functions .the discrete model is obtained from the continuous one by introducing small time bins of size in a way that one bin can at most contain one spike , say ms .we then assign to each bin in which no spike occurred a value of and ones in which a spike occurred a value of , see figure [ fig : spikebins ] .let us use this discrete model for the spike train of a neuron , representing a spike train as a binary string in the following .fixing the time span to be and the bin width to be , each spike train has length .we want to calculate the maximal entropy among all such spike trains , subject to the condition that the number of spikes in is a fixed number which we call the spike rate of .let us now calculate the entropy in the firing pattern of a neuron of which we assume that spike timing carries important information , i.e. a neuron employing a time code . in order to keep the model simple , let us further assume that the spiking behavior is not restricted in any way , i.e. that all possible binary strings are equiprobable .then we can calculate the entropy of this uniform probability distribution as    where denotes the binomial coefficient , the number of all distinct binary strings of length having exactly non - zero entries .the entropy in equation [ eq : entropytimecode ] can be approximated by    where denotes the natural logarithm to the base .the expression in equation [ eq : entropytimeln ] is obtained by using the approximation formula    which is valid for large and and in turn based on stirling s approximation formula for .see figure [ fig : entropyrate]a for the maximum entropy attainable by the time code as a function of bin size for different firing rates .   as a function of the size of the time bins .( b ) rate code using poisson and exponential spiking statistics .figure adapted from fig . d.4 . ]  on the other hand , modeling a neuron that reacts to different stimuli with a graded response it its firing rate is usually done using a rate code . assuming a rate code where the timing of spikes does not play any role yields different results , as we will see in the following , see figure [ fig : entropyrate]b . in the rate code only the number of spikes occurring in a given time interval of length matters , i.e. we consider probability distributions parametrized by and describing how likely the occurrence of spikes in a time window of length is .being well - backed with experimental data , a popular choice of is taking a poisson distribution with some fixed mean , where is thought of as the mean firing rate of the neuron .the probability of observing spikes in an interval of length now is given by the pmf of the poisson distribution    and the entropy of computes as    again using stirling s formula this can be written as    dividing the entropy by the number of spikes that occurred yields the entropy per spike .see figure [ fig : entropyrate]b for a plot of the entropy per spike as a function of the number of observed spikes .an interesting question is to ask for the maximal information ( i.e. entropy ) that spike trains can carry , assuming a rate code . assuming continuous time and prescribing mean and variance of the firing rate , this leaves the exponential distribution as the one with the maximal entropy .the entropy of an exponentially distributed spike train with mean rate is    see also figure [ fig : entropyrate]b .note that while it was possible to compute the exact entropies in the preceding as we assumed full knowledge of the underlying probability distributions .this is of course not the case for data obtained by recordings . herethe estimation of entropies faces the bias - related problems of sparsely sampled probability distributions as discussed earlier . concerning entropy estimation in spike trains the readeris also pointed to .the principle of efficient coding ( also called _ infomax principle _ ) was first proposed by attneave and barlow .it views the early sensory pathway as a channel in shannon s sense and postulates that early sensory systems try to maximize information transmission under the constraint of an efficient code , i.e. that neurons maximize mutual information between a stimulus and their output spike train , using as few spikes as possible .this minimization of spikes for a given stimulus results in a maximal compression of the stimulus data , minimizing redundancies between different neurons on a population level . one key prediction of this optimality principle is that neurons involved in the processing of stimulus data ( and ultimately the whole brain ) is adapted to natural stimuli , i.e. some form of natural ( and structured ) sensory input such as sounds or images rather than noise .for some sensory systems it could be shown that there is strong evidence that early stages of processing indeed perform an optimal coding , see e.g.  . while first mainly the visual system was studied and it was shown that the infomax principle holds here , other sensory modalities were also considered in the following years .but whereas the infomax principle could explain certain experimental findings in the early sensory processing stages , the picture becomes less clear the more upstream the information processing in neural networks is considered . here ,other principles were also argued for , see for example .on the system - level , friston et al . proposed an information theoretic measure of free energy in the brain , that can be understood as generalization of the concept of efficient coding . also arguing for optimal information transfer, norwich gave a theory of perception based on information - theoretic principles .he argues that the information present in some stimulus is relayed to the brain by the sensory system with negligible loss .many empirical equations of psychophysics can be derived from this model .there are many scales at which information - theoretic analyses of neural systems can be performed . from the level of a single synapse over the level of single neurons over the population level up to the system level . in the former cases the analyses are usually carried out on electrophysiologically recorded data of single cells , whereas on the system level datais usually obtained by eeg , fmri or meg measurements .notice that most of the information - theoretic analyses of neural systems were done for early stages of sensory systems , focusing on the assessment of the amount of mutual information between some stimulus and its neural response . heredifferent questions can be answered , about the nature and efficiency of the neural code and the information conveyed by neural representations of stimuli , see .this stimulus - response - based approach has already provided a lot of insight into the processing of information in early sensory systems , but things get more and more complicated the more downstream an analysis is performed .  on the systems level , the abilities of neural systems to process and store information are due to interactions of neurons , populations of neurons and sub - networks .as these interactions are highly - nonlinear and in contrast to the early sensory systems neural activity is mainly driven by the internal network dynamics ( see ) , stimulus - response - type models often are not very useful here . here , transfer entropy has proven to be a valuable tool here , making analyses of information transfer in the human brain in vivo possible .transfer entropy can also be used as a measure for causality , as we will discuss in the next section .   the idea of _ causality _ , namely the question of what are the causes resulting in the observable state and dynamics of complex systems of physical , biological or social nature is a deep , philosophical question that has been driving scientists in all fields ever since . in a sensethis question lies at the heart of science itself and as such is often notoriously difficult to answer .  in the neurosciences ,this principle is related to one of the core questions of neural coding and subsequently neural information processing : what stimuli make neurons spike ( or change their membrane potential , for non - spiking neurons ) ? for many years now , neuroscientists have investigated neurophysiological correlates of information presented to a sensory system in form of stimuli .  while considerable progress has been made regarding the answer to this question in the early stages of sensory processing ( see the preceding sections ) , where often a clear correlation between a stimulus and the resulting neuronal activity could be found , things get less and less clear the further downstream this question is addressed . in the latter case , neuronal activity is subject to higher and higher degrees of internal dynamics and a clear stimulus - response relation is often lacking .  considering early sensory systems , even though merely a correlation between a stimulus and neural activity can be measured ,it is justified to speak of causality here , as it is possible to actively influence the stimulus and observe the change in neural activity .note that the idea of intervention is crucial here , see .looking at more downstream systems or at the cognitive level , an active intervention albeit possible ( but often not as directly as for sensory systems ) may not have the same easy to detect effects on system dynamics . here , often just statistical correlations can be observed and in most cases it is very hard if not impossible to show that the principle of causality in its purest form holds .yet , one can still make some statements regarding what one might call `` statistical causality '' in this case , as we will see .  in an attempt to give a statistical characterization of the notion of causality, the mathematician wiener came up with the following probabilistic framing of this concept that came to be known as _ wiener causality _ : consider two stochastic processes and .then is said to wiener - cause if the knowledge of past values of diminishes uncertainty about the future values of .note that wiener causality is a measure of predictive information transfer and not one of causality and thus the naming is a bit unfortunate , see .the economist granger employed wiener s principle of causality and developed the notion of what is nowadays called _ wiener - granger causality _subsequently , the linear wiener - granger causality and its generalizations were often employed as measure of statistical causality in the neurosciences , see . another model for causality in the neurosciencesis _ dynamic causal modeling _ .in contrast to dynamic causal modeling , causality measures based on information - theoretic concepts are usually purely data - driven and thus inherently model - free .this fact can be of advantage in some cases but we do not want to make a judgment here , calling one method better per se , as each has its advantages and drawbacks .the directional and time - dynamic nature of transfer entropy allows using it as a measure of wiener - causality , as was proposed in the field of neurosciences recently . as such, transfer entropy can be seen as a non - linear extension of the concept of wiener - granger causality , see for an comparison of transfer entropy to other measures .note again that transfer entropy still essentially is a measure of conditional correlation rather than one of direct effect ( i.e. causality ) and that correlation is not causation .thus it is a philosophical question to which extent transfer entropy can be used to infer some form of causality , a question that we will not further pursue here , rather pointing the reader to .  in any casethe statistical significance of the inferred causality ( remember that transfer entropy just measures conditional correlation ) has to be verified . for trial based data - sets as often found in the neurosciences ,this testing is usually done against the null - hypothesis of average transfer entropy obtained by random shuffling of the data .given the fact that information - theoretic analyses can provide insights about the functioning of neural systems , the next logical step is to ask how these this might help in better understanding neural dysfunction and neural diseases , maybe even giving hints to new treatments .  the field one might call `` computational neuroscience of disease '' is an emerging field of research within the neurosciences , see the special issue of _ neural networks _ .the discipline faces some hard questions as in many cases dysfunction is observed on the cognitive ( i.e. systems- ) level but has causes on many scales of neural function ( sub - cellular , cellular , population , system ) .  over the last years, different theoretical models regarding neural dysfunction and disease were proposed , among them computational models applicable to the field of psychiatry , models for brain lesions , models of epilepsy , models for deep brain stimulation , models for aspects of parkinson s and alzheimer s disease , of abnormal auditory processing and for congenital prosopagnosia ( a deficit in face identification ) .some of these models employ information - theoretic ideas in order to assess differences between the healthy and dysfunctional states .for example , information - theoretic analyses of cognitive and systems - level processes in the prefrontal cortex were carried out recently and differences in information - processing could be assessed between the healthy and dysfunctional system by means of information - theory .  yet, computational neuroscience of disease is a very young field of research and it remains to be elucidated if and in what way analyses of neural systems employing information - theoretic principles could be of help in medicine on a broader scale .there exist several open source software packages that can be used to estimate information - theoretic quantities of neural data .the list below is by no means complete , but should give a good overview of things , see also .  * ` entropy : entropy and mutual information estimation ` + url : http://cran.r-project.org/web/packages/entropy + authors : jean hausser and korbinian strimmer .+ type : r package + from the website : this package implements various estimators of entropy , such as the shrinkage estimator by hausser and strimmer , the maximum likelihood and the millow - madow estimator , various bayesian estimators , and the chao - shen estimator .it also offers an r interface to the nsb estimator .furthermore , it provides functions for estimating mutual information . * ` information - dynamics - toolkit ` + url : http://code.google.com/p/information-dynamics-toolkit + author : joseph lizier + type : standalone java software + from the website : provides a java implementation of information - theoretic measures of distributed computation in complex systems : i.e. information storage , transfer and modification .includes implementations for both discrete and continuous - valued variables for entropy , entropy rate , mutual information , conditional mutual information , transfer entropy , conditional / complete transfer entropy , active information storage , excess entropy / predictive information , separable information . * ` ite ( information theoretical estimators ) ` + url : https://bitbucket.org/szzoli/ite/ + author : zoltn szab + type : matlab / octave plugin + from the website : ite is capable of estimating many different variants of entropy , mutual information and divergence measures . thanks to its highly modular design, ite supports additionally the combinations of the estimation techniques , the easy construction and embedding of novel information theoretical estimators , and their immediate application in information theoretical optimization problems .ite can estimate shannon- , rnyi entropy ; generalized variance , kernel canonical correlation analysis , kernel generalized variance , hilbert - schmidt independence criterion , shannon- , l2- , rnyi- , tsallis mutual information , copula - based kernel dependency , multivariate version of hoeffding s phi ; complex variants of entropy and mutual information ; l2- , rnyi- , tsallis divergence , maximum mean discrepancy , and j - distance .ite offers solution methods for independent subspace analysis ( isa ) and its extensions to different linear- , controlled- , post nonlinear- , complex valued- , partially observed systems , as well as to systems with nonparametric source dynamics . * ` pyentropy ` + url : http://code.google.com/p/pyentropy + authors : robin ince , rasmus petersen , daniel swan , stefano panzeri + type : python module + from the website : pyentropy is a python module for estimating entropy and information theoretic quantities using a range of bias correction methods . * ` spike train analysis toolkit ` + url : http://neuroanalysis.org/toolkit + authors : michael repucci , david goldberg , jonathan victor , daniel gardner + type : matlab / octave plugin + from the website : information theoretic methods are now widely used for the analysis of spike train data .however , developing robust implementations of these methods can be tedious and time - consuming . in order to facilitate further adoption of these methods ,we have developed the spike train analysis toolkit , a software package which implements several information - theoretic spike train analysis techniques .* ` trentool ` + url : http://trentool.de + authors : michael lindner , raul vicente , michael wibral , nicu pampu and patricia wollstadt + type : matlab plugin + from the website : trentool uses the data format of the open source matlab toolbox fieldtrip , that is popular for electrophysiology data ( eeg / meg / lfp ) .parameters for delay embedding are automatically obtained from the data . te valuesare estimated by the kraskov - stgbauer - grassberger estimator and subjected to a statistical test against suitable surrogate data .experimental effects can then be tested on a second level .results can be plotted using fieldtrip layout formats .the author would like to thank _ nihat ay _ , _ yuri campbell _ , _ jrg lehnert _ , _ timm lochmann _ , _ wiktor mynarski _ and _ carolin stier _ for their useful comments on the manuscript .

Abstract: given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems , information - theoretic analyses became increasingly necessary during the past few decades in the neurosciences . such analyses can provide deep insights into the functionality of such systems , as well as a rigid mathematical theory and quantitative measures of information processing in both healthy and diseased states of neural systems . this chapter will present a short introduction to the fundamentals of information theory , especially suited for people having a less firm background in mathematics and probability theory . to begin , the fundamentals of probability theory such as the notion of probability , probability distributions , and random variables will be reviewed . then , the concepts of information and entropy ( in the sense of shannon ) , mutual information , and transfer entropy ( sometimes also referred to as conditional mutual information ) will be outlined . as these quantities can not be computed exactly from measured data in practice , estimation techniques for information - theoretic quantities will be presented . the chapter will conclude with the applications of information theory in the field of neuroscience , including questions of possible medical applications and a short review of software packages that can be used for information - theoretic analyses of neural data .


the close formal similarity between coulomb s law of electricity and newton s law of gravitation has led to a description of newtonian gravitation in terms of a gravitoelectric field .the classical tests of general relativity can all be described via post - newtonian gravitoelectric corrections brought about by relativity theory . moreover ,any theory that combines newtonian gravitation and lorentz invariance in a consistent framework must involve a gravitomagnetic field in close analogy with electrodynamics .the gravitomagnetic field is generated by the motion of matter .for instance , the mass current in the rotating earth generates a dipolar gravitomagnetic field that has not yet been directly observed ; in fact , the main objective of the gp - b is to measure this field in a polar earth orbit via the gravitomagnetic precession of superconducting gyroscopes on board a drag - free satellite .gravitomagnetism had its beginning in the second half of the last century .developments in electrodynamics led holzmller and tisserand to postulate the existence of a solar gravitomagnetic field .in fact , attempts were made to account for the excess perihelion precession of mercury since the planetary orbits would be affected by the gravitomagnetic field of the sun . however , the excess perihelion precession of mercury was successfully explained by einstein s general relativity theory in terms of a small relativistic correction to the newtonian gravitoelectric potential of the sun .it was later shown by thirring and lense that general relativity also predicts a certain gravitomagnetic field for a rotating mass , but the magnitude of this field in the solar system is generally small and would lead to a retrograde precession of the planetary orbits .this lense - thirring precession of planetary orbits is too small to be detectable at present .  for the purposes of confronting the theory with observation ,gravitomagnetic phenomena are usually described in the framework of the post - newtonian approximation ; however , it is possible to provide a fully covariant treatment of certain aspects of gravitoelectromagnetism .in fact , extensions of the jacobi equation ( i.e. the relativistic tidal equation ) may be employed to identify the gravitoelectric and gravitomagnetic components of the curvature tensor in close analogy with the lorentz force law .this analogy is incomplete , however , since the purely spatial components of the curvature tensor do not in general have an analog in the electromagnetic case ; in fact , this is expected since linear gravity is a spin-2 field in contrast to the spin-1 character of the electromagnetic field .some of the main theoretical aspects of gravitomagnetism are discussed in section 2 .we then turn our attention to how gravitomagnetism affects the spacetime structure in general relativity .of primary importance in this connection is the gravitomagnetic clock effect , which in its simplest form may be formulated in terms of the difference in the proper periods of two clocks moving on the same circular orbit but in opposite directions about a rotating mass .let be the period for prograde ( retrograde ) motion , then for .to lowest order , this remarkable result is independent of newton s constant of gravitation and the radius of the orbit .the effect and its consequences are discussed in section 3 for circular equatorial orbits in the kerr geometry and the intimate connection between the clock effect and the gravitomagnetic gyroscope precession is demonstrated .the ppn approximation for this effect is developed in section 4 and a brief discussion of its observability is given in section 5 .the sign of the clock effect is quite intriguing , as it implies that prograde equatorial clocks are slower than retrograde equatorial clocks .this is completely opposite to what would be expected on the basis of `` inertial dragging '' .in fact , gravitomagnetism is historically connected with the question of the origin of inertia as this was thirring s motivation in his original paper on gravitomagnetism .the present status of the problem of inertia is the subject of section 6 .finally , section 7 contains a brief discussion .  unless specified otherwise , we use units such that for the sake of convenience .this section is devoted to a brief discussion of certain essential theoretical aspects of gravitoelectromagnetism .the larmor theorem has played an important role in the field of magnetism ; therefore , we begin by an account of the gravitational analog of larmor s theorem .  _ gravitational larmor theorem _  a century ago , larmor established a theorem regarding the local equivalence of magnetism and rotation .that is , the basic electromagnetic force on a slowly moving particle of charge and mass can be locally replaced in the linear approximation by the inertial forces that arise if the motion is referred instead to an accelerated system in the absence of the electromagnetic field .the translational acceleration of the system is related to the electric field , , and the rotational ( larmor ) frequency is related to the magnetic field via .the charge - to - mass ratio is not the same for all particles ; otherwise , a geometric theory of electrodynamics could be developed along the same lines as general relativity .it turns out that in general relativity one can provide an interpretation of einstein s heuristic principle of equivalence via the gravitational larmor theorem .this is due to the experimentally well - established circumstance that the gravitational charge - to - mass ratio is the same for all particles .einstein s heuristic principle of equivalence is usually stated in terms of the gravitoelectric field , i.e. the translational acceleration of the `` einstein elevator '' in minkowski spacetime .the gravitational larmor theorem would also involve the gravitomagnetic field , i.e. a rotation of the elevator as well .it follows from the theoretical study of the motion of test particles as well as ideal test gyroscopes in a gravitational field that in general relativity the gravitoelectric charge is , while the gravitomagnetic charge is ; in fact , since general relativity involves the tensor potential , i.e. ( linear ) gravitation is a spin-2 field .thus and in this case .indeed is the gravitomagnetic precession frequency of an ideal test gyroscope at rest in a gravitomagnetic field , i.e. far from a rotating source , where  ,\ ] ]  and is the total angular momentum of the source .let us note that a gyro spin is in effect a gravitomagnetic dipole moment that precesses in a gravitomagnetic field .locally , the same rotation would be observed in the absence of the gravitomagnetic field but in a frame rotating with frequency in agreement with the gravitational larmor theorem .it is the goal of the gp - b to measure the gravitomagnetic gyroscope precession in a polar orbit about the earth and thereby provide _ direct _ observational proof of the existence of the gravitomagnetic field ._ gravitoelectromagnetic field _let us consider the gravitational field of a `` nonrelativistic '' rotating astronomical source in the linear approximation of general relativity .the spacetime metric may be expressed as , where is the minkowski metric .we define , where ; then , the gravitational field equations are given by    where the lorentz gauge condition has been imposed .we focus attention on the particular retarded solution of the field equations given by    where the nature and distribution of the `` nonrelativistic '' source must be taken into account .we are interested in sources such that , and , where is the gravitoelectric potential , is the gravitomagnetic vector potential and we neglect all terms of order and lower including the tensor potential .it follows that is the effective gravitational charge density and is the corresponding current .thus , far from the source    where and are the total mass and angular momentum of the source , respectively .it follows from the lorentz gauge condition that    since the other three equations all involve terms that are of and therefore neglected .the spacetime metric involving the gravitoelectromagnetic ( `` gem '' ) potentials is then given by    the gem fields are defined by    in close analogy with electrodynamics .it follows from the field equations ( [ feq ] ) and the gauge condition ( [ gauge ] ) that          which are the maxwell equations for the gem field .using classical electrodynamics as a guide , one can investigate the various implications of these equations .a thorough approach to the determination of the gravitomagnetic field of a rotating mass ( such as the earth ) is contained in the papers of teyssandier .the fact that the magnetic parts of equations ( [ e1 ] ) - ( [ e4 ] ) always appear with a factor of as compared to standard electrodynamics is due to the circumstance that the effective gravitomagnetic charge is twice the gravitoelectric charge .that is , and are the effective gravitoelectric and gravitomagnetic charges of the source .the linear approximation of general relativity involves a spin-2 field .this field , once its spatial components are neglected , can be interpreted in terms of a gravitoelectromagnetic vector potential . to sustain the electromagnetic analogy , however , we need to require that the gravitomagnetic charge be twice the gravitoelectric charge .this factor of 2 is a remnant of the spin-2 character of the original field , while for a pure spin-1 field ( i.e. the electromagnetic field ) the ratio of the magnetic charge to the electric charge is unity .the equation of motion of a test particle of mass in this linear gravitational field can be obtained from the variational principle , where is given by  ^{1/2 } , \label{e12}\ ] ]  using equation ( [ e6 ] ) . to linear order in and , one can write ( [ e12 ] ) as    let us note that the deviation of equation ( [ e13 ] ) from a free - particle lagrangian is given to lowest order in by .this deviation would be of the form in electrodynamics ; therefore , the slow motion of the test particle is very similar to that of a charged particle in electrodynamics except that here and as expected .it thus follows from the geodesic motion of a test particle of mass far from the source in this gravitational background that the canonical momentum of the particle is given approximately by , where is the kinetic momentum . in this electrodynamic analogy , the attractive nature of gravityis reflected in our convention of positive gravitational charges for the source and negative gravitational charges for the test particle .the gravitomagnetic charge is always twice the gravitoelectric charge as a consequence of the tensorial character of the gravitational potentials in general relativity .the gauge transformations    leave the gem fields ( [ gem ] ) and hence the gem equations ( [ e1])-([e4 ] ) invariant .the lorentz gauge condition ( [ gauge ] ) is also satisfied provided .however , the quantity in the lagrangian is not invariant under the gauge transformation ( [ e14 ] ) .the gauge invariance of this lagrangian is restored , however , if the gauge function is independent of time , . in this case, we can start from a coordinate transformation in the metric ( [ e6 ] ) resulting in the gauge transformations ( [ e14 ] ) with left invariant .the gravitational field corresponding to the metric ( [ e6 ] ) is given by the riemann curvature tensor    where and are gravitoelectric and of , while is gravitomagnetic and of .the components of the curvature tensor as measured by the _ standard _geodesic observers are given by , where is the tetrad frame of the test observer .in the linear approximation under consideration here , is in effect equal to in the calculation of the measured curvature .the components of this tensor may be expressed in the form of a symmetric matrix , where and range over ( 01 , 02 , 03 , 23 , 31 , 12 ) ; hence ,    where and are symmetric matrices and is traceless .we find that the electric and magnetic components of the curvature are given by      and the spatial components are given by    that is traceless is consistent with equation ( [ e2 ] ) and the fact that and are symmetric is consistent with equation ( [ e3 ] ) at .it is therefore clear that gravitoelectromagnetism permeates every aspect of general relativity : the gravitational potentials ( gem potentials ) , the connection ( gem field ) and the curvature . in the exterior of the rotating source ,the spacetime is ricci - flat and hence , is traceless and is symmetric. these restrictions on the curvature are consistent with the gem field equations ( [ e1])-([e4 ] ) in the source - free region .the general treatment of gravitoelectromagnetism presented here has been based on a certain approximate form of the linear gravitation theory and can be used in the theoretical description of many interesting gravitational phenomena . in particular , we use this formalism below to investigate the microphysical implications of the gravitomagnetic precession of spin ._ free fall is not universal _the assumption that all free test particles fall in the same way in a gravitational field is reflected in general relativity via the geodesic hypothesis .that is , the worldline of a free test particle is an intrinsic property of the spacetime manifold and is independent of the intrinsic aspects of the particle . in this way ,general relativity is a geometric theory of gravitation .this circumstance originates from the well - tested equality of inertial and gravitational masses .an important consequence of einstein s geometric theory of gravitation is the fact that an ideal test gyroscope would precess in the gravitomagnetic field of a rotating source .here we pose the question of whether all spins should `` precess '' like a gyroscope ; evidently , the treatment of intrinsic spin would go beyond classical general relativity .it follows from the consideration of spin - rotation - gravity coupling that the intrinsic spin of a particle ( e.g. a nucleus ) would couple to the gravitomagnetic field of a rotating source ( such as the earth ) via the interaction hamiltonian    such that the heisenberg equations of motion for the spin would be formally the same as that of an ideal test gyro . intuitively , this interaction is due to the coupling of the gravitomagnetic dipole moment of the particle with the gravitomagnetic field just as would be expected from the electromagnetic analogy .it follows from equation ( [ e20 ] ) that the particle is subject to a gravitational stern - gerlach force given by    that is purely dependent upon its spin and not its mass and therefore violates the universality of free fall .the point is that a particle is in general endowed with mass and spin in addition to other intrinsic properties ; indeed , the irreducible unitary representations of the inhomogeneous lorentz group are characterized by mass and spin . in its interaction with a gravitational field , the mass interacts primarily with the gravitoelectric field while the spin interacts primarily with the gravitomagnetic field .whereas the former dominant interaction is consistent with the universality of free fall , the latter is not .for instance , the bending of light by the gravitational field of a rotating source depends on the state of polarization of the radiation .the differential deflection of polarized radiation by the sun is too small to be measurable at present .the predicted violation is also extremely small for a nucleus in a laboratory on the earth : the weight of the particle is , depending on whether the spin is polarized vertically up or down and .thus the predicted violation of the universality of free fall is extremely small.  it may still be possible to measure this relativistic quantum gravitational effect by detecting the change in the energy of a particle in the laboratory when its spin is flipped .this would require , for instance , significant refinements in modern variations of nmr and optical pumping techniques , since    is a factor of below the sensitivity of recent experiments .here is the planck length , , is the angular momentum of the earth and is its average radius .the smallest detected energy shift is about corresponding to a frequency shift of 2 nhz .however , it appears that significantly lower energy shifts may soon be detectable .  to clarify the nature of the force ( [ e21 ] ) ,let us consider the motion of a classical spinning test body in a _stationary _ gravitational field .such a system is necessarily extended and thus couples to spacetime curvature resulting in a mathisson - papapetrou force    where is the spin tensor of the system , is the velocity vector such that and the spin vector is given by    for the calculation of , it suffices to set , in the linear approximation , , and .then , and    in agreement with equation ( [ e21 ] ) .thus the existence of the force ( [ e21 ] ) may be ascribed to the intrinsic nonlocality of a particle in the quantum theory and hence the coupling of spin to the magnetic part of the spacetime curvature in a stationary field .it is important to remark here that our considerations are distinct from proposals to measure the classical spin - spin force as discussed in .our results ultimately follow from detailed considerations of dirac - type wave equations in the gravitational field of a rotating mass ( see the references cited in ) ; however , one can arrive at equations ( [ e20])-([e21 ] ) on the basis of certain general arguments such as the local isotropy of space , the extended hypothesis of locality and the gravitational larmor theorem .  assuming the approximate validity of equations ( [ e20])-([e21 ] ), it would be difficult to imagine a basic gravitational theory founded purely on the universality of free fall and riemannian geometry .however , such a theoretical structure would clearly be an excellent effective theory in the macrophysical domain ._ gem stress - energy tensor _let us imagine a congruence of geodesic test particles in a gravitational field .taking one of the test particles as the reference observer , how does the motion of the other neighboring test particles appear to the fiducial observer ?the result is best expressed in a fermi coordinate system that is set up along the reference worldline .let be the fermi coordinates of the test particles , while the reference observer is at the origin of spatial fermi coordinates .then the equation of motion of the test particles is given by the generalized jacobi equation    which is valid to first order in the relative separation and to all orders in the relative velocity .here are components of the curvature tensor as measured by the fiducial observer , i.e. they are the projections of the riemann tensor onto the nonrotating tetrad frame of the reference observer . neglecting the second and third order terms in the relative rate of separation , equation ( [ e26 ] ) can be written as the gem analog of the lorentz force law    where , and    it is important to notice that the spacetime interval in the neighborhood of the reference worldline can be expressed in fermi coordinates as , where        letting and , we find that    so that the corresponding gem fields using equation ( [ gem ] ) agree with the results in equation ( [ e28 ] ) to linear order in the separation .one can verify directly that and , so that the source - free pair of maxwell s equations are satisfied _ along the reference worldline_. moreover , it is possible to combine the gem fields together to form a gem faraday tensor ,    such that and as in standard electrodynamics .then the other pair of maxwell s equations is given by , where is easily obtained to linear order in using equation ( [ e33 ] ) . is a conserved current such that along the fiducial trajectory .this treatment should be compared and contrasted with the linear approximation developed above , in particular , the gem current is different here .it is now possible to develop the classical field theory of the gem field in the fermi frame ; in particular , one can define the maxwell stress - energy tensor and the corresponding angular momentum for the gem field .thus    is the maxwell stress - energy tensor for the gem field that is quadratic in the spatial separation and vanishes at the location of the fiducial observer .physical measurements do not occur at a point , as already emphasized by bohr and rosenfeld ; moreover , the fiducial observer is arbitrary here .therefore , a physically more meaningful quantity is obtained by averaging equation ( [ e34 ] ) over a sphere of radius in the fermi system . here is a constant invariant lengthscale associated with the gravitational field .we find that    where is a constant numerical factor and    for a ricci - flat spacetime , reduces to the bel - robinson tensor ; in this case , reduces to the weyl tensor and in equation ( [ e36 ] ) with .the magnitude of depends on whether we average over the surface or the volume of the sphere ; in any case , one can always absorb into the definition of .thus the pseudo - local gem stress - energy tensor may be defined for any observer with the tetrad frame as    in this way , the pseudo - local gem energy density , poynting flux and stresses are defined up to a common multiplicative factor .it is important to note that the spatial components of the curvature have been ignored in our construction of the gem tensor . for a ricci - flat spacetime ,however , the spatial components of the curvature are simply related to its electric components ; therefore , the pseudo - local tensor defined via equation ( [ e37 ] ) using the bel - robinson tensor contains the full ( weyl ) curvature tensor and is thus the _ gravitational stress - energy tensor_.  it follows from a simple application of these results to the field of a rotating mass that there exists a steady poynting flux of gravitational energy in the exterior field of a rotating mass ._ oscillations of a charged rotating black hole _imagine a black hole of mass , charge and angular momentum that is perturbed by external radiation .the black hole is stationary and axisymmetric ; therefore , the perturbation is expressible in terms of eigenmodes + , where depends upon the frequency of the radiation , the total angular momentum parameters of the eigenmode and the spin of the external field .it turns out that for a fourier sum of such eigenmodes , the response of the black hole far away and at late times is dominated by a superposition of certain damped oscillations of the form , where with . for these quasinormal modes ,the amplitude depends , among other things , on the strength of the perturbation while depends only on the black hole parameters .moreover , these black hole oscillations are in general denumerably infinite and are numbered as , such that is least damped and increases with .the intrinsic ringing of a black hole is due to the fact that once perturbed , the black hole undergoes characteristic damped oscillations in order to return to a stationary state .the fundamental modes of oscillations of black holes were originally found by numerical experiments and initial attempts to explain the numerical results via the properties of black hole effective potentials were unsuccessful .the solution of the problem was first given around 1980 .this work provided the stimulus for many subsequent investigations by a number of authors .a detailed discussion of black hole oscillations is contained in .  for the modes of oscillation of a charged rotating black hole , the only reliable results are for .expressions for have been obtained in the case of for a general kerr - newman black hole ; however , the results have been generalized to the case of only for a slowly rotating charged black hole . to express in terms of in the latter case ,let be the `` keplerian '' frequency for the motion of a neutral particle in a circular geodesic orbit of radius about a reissner - nordstrm black hole of mass and charge .here we use boyer - lindquist type of coordinates for the kerr - newman geometry .timelike circular geodesic orbits exist down to a null orbit of radius such that .let , then it can be shown that for a slowly rotating black hole    where    is an effective black hole rotation frequency .the -fold degeneracy in the spectrum of oscillations of the spherical black hole is removed by its rotation .we note that is proportional to the gravitomagnetic precession frequency at . moreover ,    where is the mode number and  ] .for instance , in the nonrelativistic motion of a free particle in the heisenberg picture and =i\hbar m^{-1}\delta_{jk} ] . in the limit of an infinitely massive particle, the connection disappears and the position and velocity commute ; that is , one recovers classical mechanics when the system is so massive that the perturbation due to an act of observation on the system is negligible and the system therefore behaves classically .mach s argument involves classical quantities ( c - numbers ) , whereas the quantum condition involves operators ( q - numbers ) ; nevertheless , the quantum condition implies that the intrinsic and extrinsic aspects of the particle are directly related through planck s constant . for instance , in the schrdinger picture the extrinsic state of the particle is given by the wave function and the schrdinger equation for involves , which characterizes the intrinsic state of the particle in mach s analysis .the relationship under discussion here is not merely formal but can be verified observationally .in fact , this kinematic connection is particularly well illustrated by the example of a free particle passing through a slit .the resulting diffraction angle is inversely proportional to the mass of the particle , so that the diffraction is absent in the limit of large mass and the particle behaves classically . to the extent that classical mechanics can be thought of as a limiting form of quantum mechanics , the epistemological problem of newtonian mechanics so clearly brought out by mach disappears .that is , the problem of the origin of inertia is resolved through the wave nature of matter .thus far the inertial mass of the particle has provided the quantum connection to the inertial reference frames of newtonian mechanics .the invariance group of minkowski spacetime is the poincar group whose irreducible unitary representations can be described in terms of mass and spin .thus in the relativistic theory the inertial properties of a particle are characterized by mass and spin .the inertial properties of intrinsic spin have been discussed in .inertia has its origin in the fact that matter is intrinsically extended in space and time and through this nonlocality inertial reference frames can be `` recognized '' ; then , a physical system tends to preserve its state with respect to such frames .this is beautifully illustrated by experiments involving macroscopic quantum systems that have phase coherence , such as the recent demonstration of earth s absolute rotation via superfluid he .the quantum aspects of the origin of inertia are further developed in .in this paper we have examined some of the main theoretical aspects of gravitomagnetism in general relativity .the influence of the proper rotation of a source on the spacetime structure can be studied in various ways .attention has been focused here on certain features of the gravitomagnetic clock effect and its relation with the gravitomagnetic gyro precession .however , other approaches exist and should be mentioned .the detection of the gravitomagnetic field of the earth via the lense - thirring precession of satellite orbits has been investigated by ciufolini _moreover , gravitomagnetic effects in the spacetime curvature can be measured in principle using gravity gradiometry . in this connection , it is interesting to note that gravity gradiometers of high sensitivity that are based on atomic interferometry are being developed for space applications .newton s introduction of the concepts of absolute space and time was truly revolutionary in his day and allowed him to formulate the basic classical laws of particle motion .leibniz and berkeley criticized the notions of absolute space and absolute time and emphasized instead the idea of relativity of all motion . later , however , maxwell relied on absolute space and time for his fundamental extension of the newtonian ideas of motion to electromagnetic field propagation . on the other hand, mach revived the principle of relativity of all motion on the basis of a profound analysis of the foundations of classical mechanics .mach s work played a significant role in einstein s development of the theory of relativity .mach s deep physical treatment of the relativity of classical particle motion was motivated by his epistemological stance on the relativity of all measurement . according to mach ,the result of a measurement is the establishment of a _ relation _ and not of `` absolute '' notions , since in mach s view the latter refer to processes or objects that are not empirically verifiable in principle .mach s analysis of the relativity of particle motion in his great work on classical mechanics was not extended to electromagnetic wave motion in his later work on physical optics . in this book ,mach discussed the wave theory of light as well as the speed of light ; however , he apparently made no attempt to put these in the context of his epistemological stance on the relativity of all motion .there is no evidence that mach ever wavered in his opposition to absolute motion .however , a number of mach s contemporaries pointed out the absolute character of the constancy of the speed of light and were troubled by the fact that this aspect of the relativity theory was in conflict with the relativity of all motion . among the physicists and philosophers who raised such doubts about the epistemological stance of _ the theory of relativity _ one can mention friedrich adler , hugo dingler , philipp frank , anton lampa and joseph petzoldt . although it is claimed in the book of blackmore that mach rejected the principle of the constancy of the speed of light because it was in contradiction to his phenomenalistic epistemology due to its constant validity independent of all sensations and conscious data, there is actually no evidence that mach ever directly or indirectly commented on the constancy of the speed of light .a historical analysis of the situation and the influence of these criticisms on mach can be found in the monographs of blackmore and wolters .an exposition of the reasons for the supposed opposition to the theory of relativity based on epistemological considerations and experimental facts was promised to appear in a sequel to mach s book on optics in collaboration with his son ludwig , but this was never published .although the preface to the `` optics '' is generally regarded as the most obvious evidence of mach s reluctance to accept the relativity theory , there is every reason to believe that it was written by ludwig only after the death of his father and expresses ludwig s opinion on the theory of relativity , despite the fact that ernst mach is stated to be the author of this preface .more on this conjecture can be found in .o. heaviside , _ electromagnetic theory _( the electrician printing and publishing co. , london , 1894 ) ; j.d .north , _ the measure of the universe _ ( clarendon press , oxford , 1965 ) , ch . 3 ; r. anderson , h.r .bilger and g.e .stedman , _ am . j. phys . _* 62 * ( 1994 ) 975 .a. matte , _canadian j. math ._ * 5 * ( 1953 ) 1 ; l. bel , _ c. r. acad ., paris _ * 247 * ( 1958 ) 1094 ; r. debever , _ bull . soc ._ * 10 * ( 1958 ) 12 ; r.t .jantzen , p. carini and d. bini , _ ann .( ny ) _ * 215 * ( 1992 ) 1 ; w.b .bonnor , _ class .quantum grav . _* 12 * ( 1995 ) 1483 ; r. maartens and b.a .bassett , _ class .quantum grav . _ * 15 * ( 1998 ) 705 .b. mashhoon , _ phys .a _ * 173 * ( 1993 ) 347 ._ , in : _ near zero : festschrift for william m. fairbank _ , ed .everitt ( freeman , san francisco , 1986 ) . v. b. braginsky , c.m .caves and k.s .thorne , _ phys .d _ * 15 * ( 1977 ) 2047 .p. teyssandier , _ phys .* 16 * ( 1977 ) 946 ; _ ibid_.* 18 * ( 1978 ) 1037. b. mashhoon , _ gen ._ * 31 * ( 1999 ) 681 .jacobs , w.m .klipstein , s.k .lamoreaux , b.r .heckel and e.n .fortson , _ phys .* 52 * ( 1995 ) 3521 .mallet , `` optically induced gravitational frame - dragging , '' preprint ( 1999 ) .n. bohr and l. rosenfeld , _ det .selakab . _* 12 * , no . 8s. chandrasekhar and s. detweiler , _ proc .london a _ * 344 * ( 1975 ) 441 .b. mashhoon , in : _ proc .third marcel grossmann meeting on general relativity _( shanghai , 1982 ) , ed .hu ning ( science press and north - holland , amsterdam , 1983 ) , pp .599 - 608 .hongya liu and b. mashhoon , _ class .quantum grav ._ * 13 * ( 1996 ) 233 .frolov and i.d .novikov , _ black hole physics _ ( kluwer academic publishers , dordrecht , 1998 ) .b. mashhoon , _ phys .* 31 * ( 1985 ) 290 .a. einstein , _ the meaning of relativity _( princeton university press , princeton , 1950 ) , pp .100 - 103 ; w. davidson , _ mon . not .* 117 * ( 1957 ) 212 ; j.d .nightingale , _ am .j. phys . _* 45 * ( 1977 ) 376 .

Abstract: the main theoretical aspects of gravitomagnetism are reviewed . it is shown that the gravitomagnetic precession of a gyroscope is intimately connected with the special temporal structure around a rotating mass that is revealed by the gravitomagnetic clock effect . this remarkable effect , which involves the difference in the proper periods of a standard clock in prograde and retrograde circular geodesic orbits around a rotating mass , is discussed in detail . the implications of this effect for the notion of `` inertial dragging '' in the general theory of relativity are presented . the theory of the clock effect is developed within the ppn framework and the possibility of measuring it via spaceborne clocks is examined .