quantum key distribution ( qkd ) is a method by which two parties , alice and bob , may generate a shared secret key over an insecure quantum channel monitored by an eavesdropper , eve .any qkd protocol relies on several assumptions , namely , any eavesdropper must obey the laws of quantum mechanics ; alice and bob have the freedom to choose at least one of two measurement settings ; and there is no classical information leaking from alice or bob s laboratories .most conventional qkd protocols further assume that alice and bob have near perfect control of their measurement devices as well as their state preparation .device - independant qkd is a protocol that , remarkably , is free from making these additional assumptions ; alice and bob need no knowledge of the inner workings of their devices nor even the dimension of the space their quantum states reside in .  in this paper, we use the novel approach of combining the encoding scheme of with the results of to create a device - independent quantum cryptography protocol for continuous variables ( cvs ) . cv quantum information offers higher efficiency detectors , cheap off - the - shelf components and the experimentally accessible gaussian resources .furthermore , by encoding the cv space of a harmonic oscillator into a finite - dimensional code space we are able to take advantage of results which have previously only been applied to discrete - variable ( dv ) qkd .  the first proposals for continuous - variable qkd relied on ` non - classical ' states of light such as squeezed states .in fact , one of these protocols was proven unconditionally secure .as the field matured it was recognized that such non - classical states were not required and that the more experimentally available class of coherent states were sufficient .device - independent qkd provides a way by which two parties may share a private key despite having no knowledge of the inner workings of their respective devices .conversely , in conventional qkd protocols it is regularly assumed that both parties have a high degree of control over both state preparation as well as measurement .although , recently relaxing the condition of trusting the measurement device was achieved .the security in this device - independent approach comes instead from the fact that the two parties are able to violate a bell inequality , which can remarkably be used to put a bound on the amount of information that a potential eavesdropper could , in principle , obtain .  herewe introduce a cv version of device - independent qkd .our protocol goes as follows .alice first generates a bell state which has been created using an encoding based on the gottesman - kitaev - preskill scheme where a qubit is encoded into an infinite dimensional space of a harmonic oscillator .after this the protocol continues in a similar fashion where she keeps one encoded qubit and sends the other qubit to bob over an insecure quantum channel .hence , the results of dv device - independent qkd can then be applied to the system yielding the first implementation of device - independent qkd for cvs .it is known in the field of cv quantum information that all gaussian resources are insufficient for violating a bell inequality .this means that one should already expect non - gaussian states or measurements as being a requirement , despite the fact that they are typically more difficult to produce in a lab .this highlights the challenges faced when attempting to create a cv version of device - independent qkd because most current cv - qkd protocols use gaussian states .fortunately , if we use for example , a single mode of the electromagnetic field as our harmonic oscillator , we are able to use cv resources , including high efficiency detectors and off - the - shelf components .the major drawback of dv device - independent qkd is that in order to close the detector loophole one needs high efficiency detectors .the detector loophole issue is often overcome by cv quantum information where we can take advantage of such high detection efficiencies  .this paper is structured as follows . inwe discuss separately the necessary encoding scheme as well as the results of dv device - independent qkd . inwe relate these concepts to cv quantum information and discuss formally the kind of measurements that are necessary . following this we investigate the resources required in order to implement the protocol in .since we are only capable of making approximations of the desired encoding in the real world , we consider the effects of such approximations on the encoding and resulting key rate in . finally , presents some discussions and concluding comments as well as some interesting open questions .the premise of this paper is to propose a method of implementing device - independent qkd with cv states .this is accomplished by embedding a two - level hilbert space into the full infinite - dimensional space and then using results from dv - qkd . herewe discuss the encoding scheme proposed by gottesman , kitaev , and preskill ( gkp ) in as well as the dv version of device - independent qkd .the gkp encoding provides a method to encode a qubit in the infinite - dimensional space of an oscillator in such a way that one can protect against arbitrary , but small , shifts in the canonical variables and as well as carrying out fault - tolerant universal quantum computation on the encoded space .the stabilizer generators of a two - dimensional hilbert space in an infinite - dimensional hilbert space with canonical variables are given by   the stabilizers are simply shift operators for , and if the eigenvalues are then the allowed values of and are integer multiples of .since the codewords are invariant under shifts by integer multiples of we can define a basis for the encoded qubit as for , and where the subscript indicates the ( ` position')-basis .these states can be approximated optically using schrdinger cat states , or by a variety of other methods .encoded pauli gates are defined as and ; since these operators commute with the stabilizers they also preserve the code subspace .the set of clifford operations on the encoded subspace correspond to symplectic ( or gaussian ) transformations on the cv space of the oscillator ; these operations can be implemented in a fault tolerant way . to achieve universal quantum computation we must be able to implement a non - clifford gate on the encoded subspace , for example , the addition of a -gate ( t - gate ) to the clifford group will make for a universal set of gates .the t - gate can be implemented with a non - symplectic transformation on the oscillator ; this is more experimentally difficult than symplectic transformations and requires a non - gaussian resource such as photon counting .the physical resources required to implement these gates are discussed in more detail in .the dv - qkd protocol begins with alice and bob sharing a quantum channel that emits pairs of entangled particles . to consider the worse case scenario , we allow eve full control over the source which, if she is honest , emits the state .but in general she is free to create any arbitrary state which may be entangled between not only alice and bob but herself as well . to generate a secret key, alice chooses a basis to measure in from while bob chooses a basis from and they get outcomes of , respectively  .after all measurements are performed , if alice had chosen measurement and bob chosen measurement they extract a single bit of raw key corresponding to their measurement outcome . instead , if they had measurement settings corresponding to , their outcomes are completely uncorrelated and so this case is discarded . for all other measurement settings alice andbob use their results to violate the chsh inequality  the chsh inequality puts a bound on the values of consistent with local hidden - variable theories in accordance with bell s theorem .violation of this inequality by quantum mechanics arises due to the fact that entanglement can provide nonlocal correlations that can not be produced by shared randomness .if alice and bob share a nonlocal correlation then , regardless of how this correlation came to exist , eve can not have full knowledge of the correlation or else she would be in possession of a local variable capable of reproducing the correlations .a set of measurements which give the desired behaviour in the above protocol and which maximize the violation of the chsh inequality are given by  for the moment , and in the above expression ( [ eq : bases ] ) have no relation to the encoded pauli gates and ; although we will make this connection in . the main result shown by acn et al . is that the holevo quantity between eve and bob , after alice and bob have symmetrized their marginals , is bounded as where is the holevo quantity and is the binary entropy .this provides a method which alice and bob can use to keep eve honest and bound her knowledge using only their violation of the chsh inequality .the cv version of device - independent qkd begins with alice creating an encoded bell state .this encoding is based on the gkp encoding as given in .once this bell state is created she keeps one qubit for herself and sends the other entangled qubit to bob over an insecure and lossy quantum channel .apart form this initial encoding , the protocol follows the same steps as in typical dv - qkd protocols .a set of measurements which maximize the violation of the chsh inequality , for the encoded state , consist of measurements , as defined in , which act on the encoded subspace .we can destructively measure the observables and by performing a suitable homodyne measurement of the or quadrature , respectively . by measuring the quadraturewe expect that the only outcomes possible will be integer multiples of ; even multiples corresponding to a state and odd multiples corresponding to .imperfections in the measurement and the encoded state will result in other measured values , but we can apply classical error correction and adjust the value to the nearest for an integer . the outcome of the measurement is then given by .we can measure the other three observables by first applying a change of basis gate which takes us to the basis , and in this way we need only consider homodyne measurements of the quadrature .we assume in this section that we are able to implement clifford gates as well as -gates on our encoded space and also that we can carry out homodyne measurements on the cv space ; the resources required to do this are discussed in . from hereonwards we drop the over - bar notation to denote encoded operations ; all gates are to be understood as acting on the encoded space while symplectic transformations are understood to be in relation to the oscillator .it is readily seen that we can measure in the basis by using the change of basis gate , and one can easily verify that . since the hadamard gate is in the clifford group we can implement an encoded by carrying out symplectic transformations on the full cv space .unfortunately , it is not possible to change from the or basis to the basis by using only clifford gates , which means that we will need to go beyond symplectic transformations in the cv space .this can be readily seen by recognizing that , suppose there existed a clifford gate such that .this would imply that and so is not a clifford gate by definition .  the required change of basis gates can be calculated as : , and , where , and is an encoded -gate .it is important to note that while the latter two gates are not clifford gates , they can be decomposed exactly as a composition of clifford gates with only one non - clifford -gate .furthermore , it is possible to shift the problem of implementing a -gate to a state preparation problem , and since preparation can be done ` offline ' we require only gaussian operations and one auxiliary state to carry out our cv device - independent qkd .in order for alice and bob to implement the necessary measurements they must be able to perform gates on the encoded states as well as homodyne detection on one quadrature .the necessary set of gates include ( no need for , since ) .the first two gates correspond to clifford operations while the last one is a non - clifford gate .the set of clifford gates on the encoded states correspond to symplectic transformations on the cv space , given as : , , and .the encoded gate is used not by alice or bob but in the preparation of the encoded bell state by eve .  in order to implement an encoded -gatewe need a non - symplectic transformation which requires a non - gaussian resource .the addition of photon counting to gaussian resources is sufficient to carry out non - symplectic transformations . in particular , one is able to create either a state or a cubic phase state which can then be used to implement a -gate on the encoded space . fortunately , one can generate these states offline and use them as required throughout the protocol , effectively shifting the issues of non - gaussian operations to state preparation . in this way, one needs only to have a supply of non - gaussian states and be capable of performing symplectic transformations ( including homodyne detection ) in order to implement the qkd protocol . in the case of an optical mode ,the set of symplectic transformations can be achieved with linear optics ( phase shifters and beam splitters ) and squeezing operations ( non - linear crystals ) .fortunately , alice and bob do not need to choose a measurement basis , which is used to check for a chsh violation , very often ; the probability to choose between the possible options need not be uniform , although this would work as well . if we suppose that alice and bob share quantum states , it is enough to use pairs to check for a chsh violation , so long as the measurements are causally independent . this condition would be satisfied for memoryless devices , or devices which could have internal memory reliably cleared after every run .one protocol , also provides security against coherent attacks , which is the most general form attack .since we have chosen the measurement basis corresponding to generating a key as , this means that in the limit of large almost all of the time we need only perform gaussian operations .hence , we need only perform non - clifford operations a small fraction of the time , in order to estimate the chsh violation and thus keep the eavesdropper honest .many other such dv device - independent qkd protocols exist and offer different key rates with different underlying assumptions , but typically one still requires the ability to make measurements in a set of four bases which violates the chsh inequality .in practice , the encoded gkp states will not consist of delta peaks at intervals , but instead the peaks will have some finite width and they will be modulated by a larger envelope to ensure the state is of finite energy .one way to produce an ideal gkp state is to prepare a momentum eigenstate , and then measure the value of . since the position is completely undetermined for a momentum eigenstate all values of are equally likely , and this measurement will project out a state that differs from a eigenstate by a shift of which can then be corrected .if instead of an unphysical momentum eigenstate , which corresponds to infinite squeezing , we can consider a finitely - squeezed state given by  , where for squeezing parameter .in the position representation this state is given by .an ideal homodyne measurement of is a projection - valued measure ( pvm ) with projectors corresponding to position eigenstates , or infinitely squeezed states in position .if we allow the homodyne measurement to have a gaussian acceptance of width , we replace the pvm with a positive - operator valued measure ( povm ) which consists of an ideal homodyne measurement convolved with a gaussian window .this leads to povm elements given by an ideal measurement of is described by the pvm with elements for , and if we let we obtain the result for a homodyne detector with a gaussian acceptance . without loss of generalitysuppose we obtain a result corresponding to , then the state will be transformed to , where is a squeezed vacuum state in position with width .if we obtain a result other than we can simply apply a shift to correct the state .this is of the same type of approximate codeword proposed in the gkp paper .notice that our initial squeezing determines the size of the overall envelope , width , while the precision of our homodyne measurement determines the width of the individual peaks .if we further approximate by replacing in the summation above , which corresponds to scaling each peak by a constant factor , we find , and thus bound the error by adding up the contribution from all of the tails further than from their respective peak . assuming that the probability of error is bounded as .  the errors from incorrectly identifying an encoded state will determine the amount by which one is able to violate the chsh inequality .consider one term in the chsh quantity .the correlator is defined as for outcomes and measurement choices .if we assume that our gates are perfect then all errors will come from incorrectly identifying an encoded state .we can calculate the value of after error correction by computing the expectation values of the various measurements .this value is plotted in , and it can be seen that we start to violate the chsh inequality for parameters corresponding to squeezing greater than 5 db .this shows that the value of the chsh quantity is scaled according to the error rate , assuming perfect gates .the quantum bit error rate ( qber ) is defined as since we are only extracting a key for the cases where alice does measurement and bob does measurement .this corresponds to either alice or bob incorrectly identifying the state while the other party does not make an error .the secret - key rate , under collective attacks , with one - way classical post - processing from bob to alice , is lower bounded by the devetak - winter rate  where is the mutual information between alice and bob ( being the binary entropy ) , and is the holevo quantity .  , where is the width of the individual peaks and is the width of the gaussian envelope in the gkp encoding .note that currently the maximal amount of single - mode squeezing achieved is db .the shaded region indicates a violation of the chsh inequality . ]  in , we plot both the qber and the key rate . notice that the extractable key rate remains zero even for values of slightly larger than two .the key rate grows rapidly for squeezing beyond 6 db , for example , a squeezing of 10 db yields a key rate of .note that the well known critical qber of for bb84 as well as for dv device - independent qkd are higher than the critical qber for this proposal .this is due to the fact that one requires a suitable enough approximation to a gkp encoded state in order to have a high enough violation of the chsh inequality , and by doing so one immediately achieves a corresponding low probability of error .typically one desires a high critical qber as it generally tolerates more imperfections in the protocol .however , in this case the difficulty arises from the need to violate the chsh inequality and if one is able to do so then one already obtains a small qber .intuitively , as the width of the individual peaks in the encoded state become larger , and equivalently the qber , the overall state resembles a gaussian state and thus can not violate the chsh inequality .by harnessing the results of discrete - variable qkd , with a qubit encoding in a harmonic oscillator , we provided the first device - independent qkd protocol for continuous variables .this protocol derived its security from the ability to violate a bell inequality and , remarkably , does not require alice or bob to know the inner workings of their devices .we showed how both the chsh violation and the resulting extractable key - rate depended on the quality of the approximate codewords .we also showed that , in terms non - gaussian resources , we required a modest one -gate for each of of total bell pairs .thankfully , from an experimental point of view , what this means is that only gaussian operations ( e.g. , homodyne detection ) are needed most of the time .it should be noted that our encoding scheme is experimentally challenging .however , it is still practical , with many proposals already existing .it is hoped that our paper will further motivate experimental advances using such encodings .given the technological challenges , distances in our scheme will be limited ( although not fundamentally ) .however , it should be noted that such limitations are also faced by the discrete - variable version of device - independent qkd , which is currently limited to a few kilometers .this is because it requires a detection efficiency of approximately to achieve a key rate on the order of per pulse .interestingly , one can also consider the distances over which our continuous - variable protocol will perform well .we do this by calculating the wigner function of an approximate encoded state and then send it through an amplitude damping channel .we numerically find that , for example , at 2.3 km , with 0.2db / km loss , we get a key rate of 0.35 bits / state .this is comparable to discrete - variable device - independent qkd where such schemes are limited to only a few kilometers .furthermore , we note that the distance of our protocol can also be improved by using distillers as was shown for discrete - variable states or by the application of heralded amplifiers  .it is an interesting open question whether one can devise a device - independent continuous - variable qkd protocol with more readily accessible states .one possible avenue to explore is lifting the requirement of a chsh inequality violation by considering a protocol where only one party trusts their device .this one - sided device - independent qkd requires one to violate only an epr - steering inequality , which amounts to alice and bob checking that they have entanglement and ruling out local hidden state models .here we justify the use of several results in the paper . inwe identified the approximate encoded state that would result when a finitely squeezed state was measured with a povm eq .( [ eq : povm ] ) subject to mod . supposing that we find an outcome corresponding to we have thus we recover the fact that this measurement projects the state onto a superposition of squeezed states in with a spacing of and width of weighted by an overall gaussian envelope of width .in we stated that the value of , with error correction , can be calculated by finding the appropriate expectation values .consider the box function to calculate the value of tr , with , for an element of the density matrix , we simply perform the integral where - 1/2\right\rbrace12 & 12#1212_12%12[1][0] link:\doibase  10.1103/revmodphys.74.145 [ * * ,  ( ) ] link:\doibase 10.1103/revmodphys.81.1301 [ * * ,  ( ) ] * * ,  ( ) link:\doibase  10.1103/physrevlett.98.230501 [ * * ,  ( ) ] * * ,  ( ) link:\doibase 10.1103/physreva.64.012310 [ * * ,  ( ) ] link:\doibase 10.1103/revmodphys.84.621 [ * * ,  ( ) ] link:\doibase 10.1103/physreva.61.022309 [ * * ,  ( ) ] link:\doibase 10.1103/physreva.63.022309 [ * * ,  ( ) ] link:\doibase 10.1103/physrevlett.88.057902 [ ** ,  ( ) ] link:\doibase  10.1103/physrevlett.108.130503 [ * * ,  ( ) ] link:\doibase 10.1103/physrevlett.108.130502 [ * * ,  ( ) ]  ( ) ,  _ _ ( , ) link:\doibase 10.1103/physreva.79.012101 [ * * ,  ( ) ] link:\doibase  10.1103/physreva.67.012105 [ * * ,  ( ) ] link:\doibase  10.1103/physreva.71.022105 [ * * ,  ( ) ] link:\doibase 10.1103/physrevlett.99.210405 [ * * , ( ) ] link:\doibase 10.1103/physreva.85.042116 [ * * ,  ( ) ] http://stacks.iop.org/1367-2630/16/i=1/a=013033 [ * * ,  ( ) ] * * , ( ) ( ) ,  link:\doibase 10.1103/physrevlett.112.120504 [ * * ,  ( ) ] link:\doibase 10.1364/ol.35.003261 [ * * ,  ( ) ] link:\doibase 10.1103/physreva.66.052322 [ * * ,  ( ) ] http://stacks.iop.org/0295-5075/68/i=3/a=323 [ * * ,  ( ) ] link:\doibase  10.1140/epjd/ e2005 - 00306 - 3 [ * * ,  ( ) ] * * ,  ( ) link:\doibase 10.1103/physrevlett.82.1784 [ * * ,  ( ) ] link:\doibase 10.1126/science.283.5410.2050 [ * * ,  ( ) ] http://stacks.iop.org/1367-2630/11/i=4/a=045021 [ * * ,  ( ) ] * * ,  ( ) * * ,  ( ) link:\doibase  10.1103/physreva.79.062318 [ * * ,  ( ) ] * * ,  ( ) link:\doibase 10.1103/physrevlett.95.010503 [ * * ,  ( ) ] link:\doibase 10.1103/physrevx.3.031006 [ * * ,  ( ) ] ( ) ,  link:\doibase 10.1103/revmodphys.77.513 [ * * , ( ) ] link:\doibase 10.1098/rspa.2004.1372 [ * * , ( ) ] link:\doibase 10.1103/physrevlett.104.251102 [ * * ,  ( ) ] link:\doibase  10.1364/oe.19.025763 [ * * ,  ( ) ] link:\doibase 10.1103/physrevlett.85.441 [ * * ,  ( ) ] link:\doibase 10.1103/physreva.84.010304 [ * * ,  ( ) ]link:\doibase 10.1103/physrevlett.105.070501 [ * * ,  ( ) ] link:\doibase  10.1103/physreva.84.022325 [ * * ,  ( ) ] link:\doibase 10.1103/physreva.85.010301 [ * * ,  ( ) ]

Abstract: we present the first device - independent quantum cryptography protocol for continuous variables . our scheme is based on the gottesman - kitaev - preskill encoding scheme whereby a qubit is embedded in the infinite - dimensional space of a quantum harmonic oscillator . the novel application of discrete - variable device - independent quantum key distribution to this encoding enables a continuous - variable analogue . since the security of this protocol is based on discrete - variables we inherit by default security against collective attacks and , under certain memoryless assumptions , coherent attacks . we find that our protocol is valid over the same distances as its discrete - variable counterpart , except that we are able to take advantage of high efficiency commercially available detectors where , for the most part , only homodyne detection is required . this offers the potential of removing the difficulty in closing the loopholes associated with bell inequalities .


distributional semantic models  , also called _ count models _ have been popular in the computational linguistics literature for several decades . in the last few years , however , _predict models _ such as the skip - gram model and continuous bag - of - words model have become the de facto standard for word modeling  .these methods have origins in neural language modeling  , where the goal was to improve classic language models .a recent evaluation study  suggested that the predict models are superior to the count models on a range of word similarity tasks .newer work , however , attributes a large amount of the differences in performance between count and predict models to a lack of proper hyperparameter optimization  . this has prompted the need for understanding the differences between the two types of models .  to this end , levy and goldberg  made an interesting connection between two key models : a traditional count model based on pointwise mutual information ( pmi ) and a predict model , namely the skip - gram model with negative sampling ( sgns ) .the key result is that the sgns model is equivalent to a shifted version of the pmi method , where all values get shifted by a factor of .however , the proof assumes that the input and output dimensions are the same ; hence the word vectors will still be of the size of the vocabulary .as we show in our analysis , this assumption has important implications for subsequent steps , such as dimensionality reduction methods like singular value decomposition ( svd ) .the aim of this paper is to analyze connections between count and predict models in a more detailed fashion . in particular , we investigate the differences between sgns and pmi methods more deeply .we make several observations that help this understanding .the first useful result is that the shifted pmi model comes out as an explicit version of sgns in which the context word vectors are fixed at their one - hot representations .we point out two differences between the shifted pmi model and the sgns model from an optimization perspective .first , sgns usually works in lower dimensions , making it possible for all word vectors two interact .second , as count models have usually more parameters , this might cause them to overfit on the data .we use the insight from our analysis to propose several interesting extensions to classic word embedding models .for example , our analysis allows regularization to be added into pmi methods in a systematic way .we also propose a new convex word vector model based on cbow which offers interpretability and a well - defined training objective . in short ,we draw connections between existing count and prediction models and augment them in several ways .however , in order to fully understand the factors that differentiate various methods , a comprehensive set of experiments is needed .we plan to carry out these experiments as part of future work .consider a corpus that is a sequence of words over a vocabulary .we will use when referring to the index of a target word and when referring to the index of a context word .we define the exact notion of context in section [ sec : context ] .note that and are integers taking values from to .each word and context word has a vectorial representation .we will use and to denote the word vector and context vector corresponding to word and context word . for vector , will denote the -th component of .let and denote the set of all word and context word vectors .one can also think of and as matrices and whose -th and -th rows are given by and respectively .we will use and to synonymously refer to and .let us use # to denote a count and weighting function . in the simplest case , for example ,# could denote the number of occurrences of the ( word , context word ) pair in corpus .let us define and , the total counts for word and context vector .an important design choice in count models is the definition of _ context_. the definition of context determines which co - occurrences get considered when creating a model , along with the importance of single co - occurrences . a common way of defining context is based on choosing a window around each occurrence of a word in .there are many options to customize this window ; for example , it can either be symmetric or asymmetric around . there could also exist weights ( e.g. , dependent on the relative positions of and ) associated with each occurrence in .one may also include down - weighting of common words , possibly for both for the current word as well as for the context words .all these choices determine the value of # .note that since the weightings can be real , # , and can take non - negative real values in general .if the window is symmetric around and as well as the weighting functions are symmetric , then  1 . is symmetric , i.e. , and 2 . if and denote the same word  we will refer to this as the _ symmetry condition_. in section [ subsec : symm ] we will revisit this condition and see that it helps our understanding of non - uniqueness and interchangeability of word vectors and context word vectors .  * example .* consider an asymmetric window that includes words to the left and words to the right of a word .in other words , take one ( word , context word ) pair , where .let be the probability with which word is chosen for inclusion into the training set .for example , mikolov et al  suggest the following down - sampling probability : where is the frequency of word in the corpus , and is a threshold . let be the weight associated with as context word . hereagain we may want to down - sample frequent words similar to ( [ downsamp ] ) .let be the weight associated with the position of the context word in the context window . using all these individual weights we can define the overall weight for the pair : accumulating all this info over the entire corpus gives us the total value for ( word , context word ) pairs : now # and # can be formed using # as described in the beginning of this section .remember that the goal of this paper is to investigate the differences between count and prediction models . to do this , we start by summarizing two special instances of count and prediction models .we first introduce pmi models as representatives of count models in section [ sec : pmi ] , and then discuss the skip - gram model with negative sampling ( sgns ) in section [ sec : sgns ] .pmi models have been used extensively in distributional semantic models  to compute similarities between words .as the name implies , the key quantity in these models is _ pointwise mutual information ( pmi)_. its definition and mle estimate from data are given by where is the simple count function . for a given word , pmi represents by forming a vector whose components are for all .  note that pmi is not defined when . to circumvent this problem , positive pmi ( ppmi ) replaces all negative values , i.e. ,   more recently , levy and goldberg  have defined shifted variants of pmi and the ppmi metrics . shifted pmi ( spmi )is just defined as , and shifted positive pmi ( sppmi ) is defined as where is some positive integer .pmi and ppmi are , respectively , special cases of shifted pmi and shifted ppmi , with set to .we will return to shifted pmi and shifted ppmi in the next section , where we talk about the equivalence of certain methods .  as all vectors produced by pmi have the vocabulary size as the dimensionality, dimensionality reduction techniques are often applied to the original matrix to decrease the memory and computational requirements .sgns  is a popular predict model that aims to predict which word occurs with a context word .one can derive the sgns  model from a binary classification setting where the target variable specifies whether a word occurs with context word .sgns tries to solve this task with the logistic loss applied to the input score .remember that our corpus is a sequence of words , . to make notation easier , here we assume that the context is defined by all the words within a window around each word .let the corresponding sequence of sets of context words be . to construct the training set for the sgns model , one forms one training example for each and each context word . for each , a set of negative context words , is chosen at random and , the logistic loss function is applied .this loss is the aggregation of losses for one positive example and negative examples .more formally , the loss of one training example is : where is the logistic loss corresponding to the word - context pair and target .the loss over the entire corpus is then the sum of all individual losses :   let us refer to this way of writing the training objective ( sum over occurrences in the corpus ) as the _ corpus format_.  levy and goldberg  showed that , by accumulating data over co - occurrences of words and context words , the objective function in ( [ corpusform ] ) can be rewritten as here and the one in  are negatives of each other . ] : let us refer to this ( equivalent ) way of writing the training objective as the _ co - occurrence format_. note that in the co - occurrence format , we do not compute losses for each token individually , but instead compute a loss for each unique pair .  inwe will from now on also consider other loss functions apart from the logistic loss like in the original formulation .we consider general loss functions , ( here is the target variable ) with the logistic loss being a special case .this generalization allows us to define custom loss functions that can incorporate domain knowledge or other side information .table [ tab : losslist ] lists a set of popular loss functions .[ sec : sgns ]  .a list of popular loss functions , . [ cols="^,^",options="header " , ]   [ obs : xalosses ] for building semantic representations that are used for computing similarity , it is often not desirable to have negative components in the word vectors .poscondition in the last column of table [ tab : losses ] checks when this case occurs .this condition turns out to be the same for all losses , which is interesting .there is no difference between squared loss , squared hinge loss and huber loss .this is because , in the interval  ] .these three losses have an expression for that is quite different from that for the logistic loss .hinge loss prefers to set at the extremes : or .one of the key results of levy and goldberg  is that the vectors created by shifted pmi are a solution to the sgns objective .we use the quadratic analysis of section [ quadratic ] to say this more cleanly .let denote the unit vector in whose -th component is and all other components are zero .suppose we use a one - hot representation for context vectors , i.e. , we fix for all .thus , we are fixing .[ ob : minpmi ] suppose we fix each context vector to the one - hot representation given above .then , only remains as the set of variables , and the following hold .  1 .the minimizer of ( [ ellwc ] ) is given by  , where is a vector with as components . is the shifted pmi representation as defined in section [ sec : pmi ] ., together form an optimal solution of sgns .  in other words, we have a closed - form solution for sgns ( assuming ) . though levy and goldberg  not mention the above construction , this is a simple observation that easily follows from their analysis .* proof of observation [ ob : minpmi ] .* let s take one .by the way we defined , we have . in by ( [ ellall ] ) , the variable occurs only in the term .therefore , , which proves part ( a ) of observation [ ob : minpmi ] .part ( c ) follows from the fact that , the pair , is such that is minimized for every pair and it is not possible to do better than that .empirically , there seems to be evidence that predict and count models perform differently ( e.g. , ) .this is interesting since they all consider the same input data namely the co - occurrences of words in text .what are the reasons for this ? although the previous section pointed out a strong connection between sgns and pmi methods ( with observation [ ob : minpmi ] even indicating a near equivalence ) , we believe there are two main differences between the two methods pertaining ( a ) the dimension of the embeddings , and ( b ) being fixed as the one - hot representation .let us now expand on the two factors .[ obs : dim ] * dimension of embeddings . * as already mentioned in observation [ obs : blowup ] , the small embedding dimension used by sgns for words can be crucial for learning good word vectors .for example , co - occurring words can influence each other .the full dimension used by pmi methods does not allow this to happen ; the full dimension also has the disadvantage of suffering from overfitting due to an excessive number of variables .[ obs : onehot ] * one - hot representation for . *similar to the previous observation , there is a difference in which variables are optimized .sgns operates with both , and as variables .as shown in subsection [ subsec : connectsgns ] , pmi methods , on the other hand , correspond to using a fixed one - hot representation for .an important consequence of such a representation is that it does not allow close context words to influence each other .future work should empirically investigate the role of this factor .recall that the solution for sgns given by shifted pmi is unusable in practice , since we have entries that are .also , the shifted pmi solution has a high number of dimensions , and this might not be useful in practice .levy and goldberg present two heuristics remedy these problems .first , they propose omitting negative terms in the objective function to make a solution feasible .second , they suggest a subsequent svd step to reduce the dimensionality of the shifted ppmi matrix . we belowdiscuss each of these heuristics and their consequences in more detail .   since in practice, we can not work with vectors that have entries , goldberg and levy propose shifted ppmi instead of shifted pmi to remedy this problem .shifted ppmi corresponds to leaving out all terms from ( [ ellall ] ) that have negative shifted pmi values .this is equivalent to a modified sgns method in which during negative sampling examples not satisfying the poscondition , i.e. , those with are left out .there is two issues with the above approach .first , we no longer can guarantee optimality for this solution .second , this also seems inconsistent with the main idea behind negative sampling , which is to sample from unobserved pairs . levy and goldberg make the following statement in the second paragraph of section 5.1 of their paper : we observe that shifted ppmi is indeed a near - perfect approximation of the optimal solution , even though it discards a lot of information when considering only positive cells ." however , this does not explain the role of the discarded terms .in particular , when training sgns with a low number of embedding dimensions , discarding those terms could be of real importance .   levy and goldberg also propose to apply svd to the sppmi matrix in order to obtain low - dimensional embeddings . if we follow this step , we loose again some of the optimality we had with the shifted pmi solution .more formally , consider a sppmi matrix whose term is . to form word vectors of dimension lower than ,goldberg and levy suggest to apply svd to the matrix .let denote the svd .if one is interested in an embedding of dimension , the best rank approximation of , is used . to form word vectors, one can use either or the symmetric version " , .[ obs : svd ] levy and goldberg  propose that svd is done on or one of its variants .however , if remaining faithful to sgns is the aim , ( [ tsg ] ) indicates that the weighting term also be included and that we solve it is non - trivial to solve this problem ; an svd based approach will not work .let us now look at the limiting full case , i.e. , , to point out some relations and differences within the methods in the pmi class .[ obs : fullsvd ] we can use the full svd of and define word vectors and . note that these word vectors are not the same as shifted ppmi which uses itself as word vectors .however , because , the dot products between any two word vectors is identical for ( i.e. , ) and . on the other hand , in general , .what this means is that svd is consistent with shifted ppmi , but symmetric svd is not consistent with shifted ppmi .levy and goldberg  recommend shifted ppmi and refer to the spectral word vectors for it as _ svd _ and _ symmetric svd_. their experiments showed the symmetric version to yield better results than sgns .the above subsections went into various ways of connecting sgns and pmi methods and also brought out various differences .figure [ fig : diffs ] gives a rough and quick view of the various morphings of sgns into different pmi methods .the use of a small embedding dimension by sgns as opposed to the full dimensional one hot representation used by pmi methods is probably the most important difference .the discarding of certain negative examples , the approximation of the original non - linear objective by a quadratic , and the leaving out of the factors from the quadratic objective , are additional differences .carefully designed experiments are needed to understand the individual effects of each difference on the quality of the resulting word vectors .the various differences also imply that , even though the methods at the two left ends of the figure involve low dimensional embeddings , they could be vastly different due to the various reasons given in this section .in the following two subsections , we propose two extensions to the basic pmi model .first , we show how to add regularization to pmi models and give explicit solutions for and regularization .we hope that regularization will help improve issues with data sparsity . after that, we propose a new convex model for word embeddings that is not only easy to learn , but also yields intuitive word vectors since each dimension corresponds exactly to one context word .let us continue with the formulation of section [ quadratic ] and add regularization . if overfitting is one of the causes of the inferior performance of pmi methods compared to sgns , then regularization should help improve performance .consider the decoupled determination of .our modified objective is now where is the regularization term . as we argued in section [ quadratic] , each is decoupled from other variables .hence , we can solve each one - dimensional optimization problem in isolation even with regularization added .we now derive closed - form solutions for and regularization .regularized solution .note that , even though is a nonlinear function , the straight line from a to b approximates that part of the curve well . ]  we can down - weigh frequently occurring words and context words and add a standard regularizer as follows : we will discuss regularizer later .let us also focus only on the logistic loss .dividing by we get setting the derivative of the objective to zero and simplifying , we get to be the solution of the line from to in figure [ fig : l2reg ] approximates nicely in the region where the optimal lies . the equation of this line is given by solve for to get the optimal as    similarly , it is easy to work out closed form expressions for for regularization . in this case, many of the optimal values go to zero : the larger the value of , more is the number of zeros .there are three cases to consider . for regularization , the value , plays a key role.  * case 1 . * . for this caseit is easy to check that , at , the left hand side derivative of the objective in ( [ wcstar ] ) is non - positive , and the right hand side derivative is non - negative .hence is the optimal solution .* case 2 . * . for this case ,the right hand side derivative at is negative and so the optimum value is positive .the optimum is found by solving , which yields   * case 3 . *for this case , the left hand side derivative at is positive and so the optimum value is negative .the optimum is found by solving , which yields    adding regularization to count models has two benefits .first , while the non - regularized solution ( see table [ tab : losses ] ; same as spmi ) is unusable because goes to extreme values when , regularized solutions are always well - defined .second , we expect regularization to help if overfitting is degrading performance with pmi models .future work is needed to empirically study and validate the usefulness of regularization in count models .   motivated by the analysis in section [ quadratic ], we develop a new and simple convex formulation for determining word vectors .similar to the skip - gram model , we phrase our model as the task of predicting a word conditioned on context .however , instead of learning representations for both words and their context words , we fix the context vectors to a one - hot representation .this makes our objective function _ convex_.another advantage is that this results in _ intelligible _ models each vector entry refers to a weight given to a context word . also , to obtain compact representations , we instead use regularization on the word vectors instead of learning embeddings in lower - dimensional spaces like traditional predict models .let us describe the formulation in some more detail .let denote the dimension of the vector representation of context .depending on the context modeling employed , can be one to several times of .let denote the context representation . is a function of the context , , written as .the weight vectors are also points in .the score of a positive example is .remember that the input corpus is a sequence of words , .let the corresponding sequence of contexts be .we solve the following minimization problem . in the above , is a regularization constant dependent on , for example , make dependent on the number of context words to which word is attached . ] which can be tuned to balance sparsity of and performance ; includes the effect of positive as well as negative examples .other forms of regularization are also possible , for example a combination of and regularization ( generalized elastic net ) .  ** we can define context in several ways . hereare a few possibilities :  1 .a simple way is to have a context , with each dimension corresponding to one word in the vocabulary .similar to the sgns model , we only assume _ individual _ interactions between words and context pairs , i.e. , each will exactly contain one non - zero entry .this way , we can rewrite the objective in the co - occurrence form and obtain all vectors in closed form by accumulating the statistics for each ( word , context word ) pair .we can do better than option 1 above and pair a word with a window of context words _ simultaneously _, i.e. , can now have multiple non - zero entries .a bag - of - words representation can be used to represent the context input using the context words .for example , we can set if occurs in the context window and zero otherwise , where is some weighting that is a function of how far is from .3 .  if we want to make use of word order , we can define context inputs where is the length of the context window and one block of inputs is used for each context word , and blocks concatenated in the proper order of occurrence in the window .4 .  depending on the purpose for which the word vectors are developed we can also use dependency - tree based context features  .again , we can use one - hot representations of these features .  ** except option 1 , aggregating at the individual level ( to convert the objective into the co - occurrence form ( [ ellall ] ) ) as well as forming a closed form solution , is in general infeasible .this means we have to train our model using the objective in _ corpus format_. to do the training , we can choose from various optimization strategies .we can employ : ( a ) proper softmax over all words ; ( b ) hierarchical softmax  ; ( c ) noise - contrastive estimation ( nce )  ; or , ( d ) negative sampling like in sgns  .we now present examples for two of the four optimization strategies mentioned above . with softmax ,our objective is let us now consider optimization via negative sampling .let denote a randomly sampled set of negative examples .then where .[ obs : interp ] one of the advantages of the convex formulation described above is interpretability . if the component of some is large and positive , it can be directly translated to the role played by the context term of .this direct interpretability also means that domain knowledge may be easy to infuse into the formulation by specification of constraints and extra regularization on the weights .[ highorder ] the model developed in this section is in between conventional low - dimensional word embeddings ( no interpretability , allows for higher - order effects ) and distributional word representations ( full interpretability , no higher - order effects ) . here, the term higher - order effects " refers to the effects ( influences ) that occur during the joint optimization of context and word representations , e.g , two words can be made similar because they occur in contexts that are also similar .in this report , we showed how to explicitly solve the sgns objective for a broad range of loss functions .this step allowed us to connect shifted pmi models to sgns models under general loss functions .furthermore , we pointed out two important differences between the shifted pmi model and sgns model .first , in the sgns model , far fewer embeddings are used in practice , making the sgns model less prone to overfitting .second , in the pmi model , the context vectors are fixed ; hence , there is also no interaction between context vectors and word vectors .  finally , we proposed two extensions to existing models .first , we showed how we can incorporate regularization into pmi models to alleviate overfitting .second , we presented a new embedding model that not only has a convex objective , but also results in intelligible embeddings .future work is needed to empirically validate our proposed methods and extensions .

Abstract: in a recent paper , levy and goldberg  pointed out an interesting connection between prediction - based word embedding models and count models based on pointwise mutual information . under certain conditions , they showed that both models end up optimizing equivalent objective functions . this paper explores this connection in more detail and lays out the factors leading to differences between these models . we find that the most relevant differences from an optimization perspective are ( i ) predict models work in a low dimensional space where embedding vectors can interact heavily ; ( ii ) since predict models have fewer parameters , they are less prone to overfitting .  motivated by the insight of our analysis , we show how count models can be regularized in a principled manner and provide closed - form solutions for l1 and l2 regularization . finally , we propose a new embedding model with a convex objective and the additional benefit of being intelligible .


the growing interest in biological noise has led to many efforts to measure gene expression at the single cell level , revealing a very distinct dynamics when compared to population cell experiments . in two well studied examples ,the p53mdm2 regulatory network and the nf signaling pathway , sustained oscillations are observed in single cells following activation signals , while cell population experiments only show damped oscillations . in both casesthe core circuit consists of a negative feedback loop , one of the most common network designs , where the active transcription factor promotes the transcription of its own repressor .stable oscillations are not trivially generated in a single negative feedback loop .a loop composed of only two agents does not oscillate for plausible macroscopic equations .sustained oscillations require at least three agents , where the third one introduces a time delay that repeatedly causes the system to overshoot or undershoot above and below the steady state .some models achieve sustained oscillations by introducing _ ad hoc _ time delays to reproduce those that a system incurs when manufacturing the various molecular components .the dynamics can also be enriched by considering combinations of negative and positive feedback loops , bistable switches , or by inheriting oscillatory signals from upstream regulators .  in this paperwe show that the stochastic fluctuations in gene expression in a negative feedback loop can produce sustained pulses of protein expression .it has been suggested that protein fluctuations are driven by underlying messenger rna ( mrna ) fluctuations .we show that the mrna stochastic fluctuations can be amplified during translation and induce a sustained excitatory behavior characterized by a series of sustained anti correlated pulses in the expression of the positive and negative regulator of the loop .noise induced oscillations have already been found in other systems .oscillations in a circadian clock consisting of a combination of a positive and a negative feedback loop are enhanced by the intrinsic biochemical noise .resonant amplification of the stochastic fluctuations can lead to cycling behavior in the volterra system and in self regulatory genes .here we show that a simple negative feedback loop consisting of an activator protein and its repressor is capable of producing protein pulses when the stochastic fluctuations of the mrna are taken into account .this result does not rely on having a large number of molecules or on the particular statistical properties of the noise , neither depends on upstream pulsating signals or couplings to additional loops .recently several experimental studies have shown that gene expression occurs in bursts of transcriptional activity .these bursts are usually ascribed to random upstream events , such as chromatin remodeling or random promoter transitions . herewe demonstrate that sustained pulses of protein expression can be produced merely by the stochastic nature of mrna kinetics .  in view of the crucial significance of mrna fluctuations, we asked ourselves how gene expression can be accurately regulated in the noisy cellular environment .regulation of gene expression is a complex , multi layered process that involves many different players . since their discovery more than a decade ago, regulatory rnas ( termed `` regrnas '' in this paper ) have emerged as key regulators in virtually all the cellular processes studied to date .regrnas are non coding rnas that regulate gene expression by base pairing to a partially or fully complementary mrna target. micrornas ( mirna ) and antisense rna are two examples of regulatory rnas .in the mammalian genomes regulatory rnas often share the same transcriptional regulation as their targets , giving rise to a diversity of feed forward loops .such pairs of target and regulatory rna transcripts are found to be co regulated , co expressed or inversely expressed more frequently than expected by chance , presumably due to sharing of common transcription factors . in particular , some transcription factors have been found to bind to overlapping transcript pairs , thus potentially coupling the regulation of a coding gene and its regulatory rna .it was suggested that one potential purpose of such design is to filter transcription noise .  in this workwe consider the case in which the positive regulator in the loop transcribes both mrna and regrna .the latter could be either mirna or an antisense .additionally , we assume that the regulatory transcript binds to the mrna and prevents its translation , but without promoting its degradation .we show that the presence of regrna reduces the excitability of the system by increasing its capacity to buffer the noise .we consider three alternative designs of a negative feedback loop . the main loop is composed of a transcription factor ( the positive regulator , ) and its repressor ( the negative regulator , ) .we assume that in response to some external cellular signal , has been activated and is promoting the transcription of .two of the designs also model the transcript dynamics . in the three models , is degraded at the post translational level via protein protein interaction with .schematic representations of the three models , as well as the individual chemical reactions , are shown in fig .[ fig : models ] .( color online ) schematic representation of the reduced , base , and extended models together with the chemical reactions .the reduced model , making up the simplest feedback loop , considers only the production and degradation of the proteins ( ) and ( ) , and assumes that mrna is in quasi equilibrium .the base model includes mrna dynamics by expanding the chemical reaction of the production of ( red box ) .the additional chemical reactions describe the transcription of sense mrna ( ) , the translation of , and the degradation of .finally , the extended model puts into play regulatory rnas , ( dashed blue box ) , by adding their transcription , degradation , and the formation and degradation of a sense regrna complex . , width=302 ]  the description of the biochemical parameters and their range of variation are summarized in table [ tab : parameters ] ..[tab : parameters ] symbols used as parameters of the models , their values or range of variation , and their description .[ cols="^ , > , < " , ]   the values adopted correspond to typical mammalian cells .the base model consists of the two proteins regulators and , and an mrna transcript .( the latter will be interchangeably refered to as `` transcript '' , `` sense transcript '' , or `` sense mrna '' . ) promotes the transcription of , which in turn encodes for the negative regulator .the deterministic equations are :  [ eq_net_i ]   we assume that the cooperative binding of molecules of at the promoter site of are required for efficient transcription , and we model the transcription rate with a hill s function .the translation rate of is proportional to the transcript level , . finally , the rate of the degradation of is proportional to the concentration of both proteins .this model introduces a regulatory rna ( regrna ) , denoted , that targets . promotes the transcription of both and transcripts . regulates by base pairing to it , thus creating a hybrid - complex , .we assume that the complex molecule can unbind but not degrade , _i.e. _ , it returns to the system both complementary rna transcripts .the new set of equations describing the model are given by :  [ eq_net_iii ]   the extended model contains parameters described in table [ tab : parameters ] . in this workwe focus on analysing how the temporal behavior of the protein levels is influenced by the mrna and regrna transcription rates , the translation rate , and the complex formation and destruction .therefore , we systematically explored how the variation in , , , , and , affect the emergent system bahvior .we maintained the remaining parameters constant throughout the simulations ; random exploration showed that changes in their value did not affect the behavior of the system appreciably .finally , to isolate the effect of rna fluctuations we consider a reduced model where the rna transcripts are assumed to be in quasi equilibrium . from a molecular perspective, this model corresponds to the limit where the time scales associated with rna transcription and degradation are much shorter than those associated with protein production and degradation .thus , the transcript levels adjust rapidly to changes in and , and in eq .( [ eq_net_i:2 ] ) .the equations become :  [ eq_net_ii ] modeled the stochastic behavior at the single cell level using the gillespie algorithm . to quantify the importance of the stochasticity we solved the deterministic equations and compared their trajectories with the stochastic dynamics for each set of chemical parameters in all three models .the deterministic equations always reached a steady state , which is the same in all three models for a given set of parameters .we chose this value as initial condition of the stochastic simulations in order to minimize the initial transient period .each single stochastic simulation represents a possible cell realization , the equivalent of a single cell experiment in this description . to simulate the behavior of a random distribution of cells, we computed realizations with different random seeds for each set of chemical parameters .the typical length of each simulation was min .  for each molecular species , where stands for , we computed the average molecular count ( averaged across all simulations and over time ) , and the coefficient of variation , , defined as the ratio between the standard deviation and the average level ^ 2\rangle_t^{1/2}}{\bar{x}}.  \label{eq : cov}\ ] ] significantly increases with the amplitude of pulses of , therefore it provides a general signature of the stochasticity and strength of the pulsations for the three models .  to identify the pulsating dynamics in more detail , we computed the normalized autocorrelation function , given by \left[x(t+\tau)-\bar{x}\right ] \rangle_t } {   \langle\left[x(t)-\bar{x}\right]^2\rangle_t}.\ ] ] the autocorrelation function for a each model and parameters was averaged over realizations of the initial conditions .  additionally , we compared the results of the autocorrelation with the power spectrum , defined as = , where is the fourier transform of and denotes average over realizations .[ fig : examples ] provides a snapshot of the stochastic behavior of the three models .    for each model , an example of a typical simulationis shown on the left panels , while the corresponding correlation functions and power spectra ( averaged over runs ) are depicted on the right panels .overall , the base model is characterized by a clear excitatory behavior in the positive regulator .this is confirmed by the peaked correlation function , which provides a characteristic period of the pulses . at the other extreme, the reduced model completely lacks pulses , and the correlation function rapidly decays to zero .finally , the extended model shows a more complex behavior : the time evolution is characterized by a combination of short pulses and long term fluctuations .the correlation function has a smooth , oscillating shape , that corresponds to the long term fluctuations , and a characteristic period of the pulses can not be identified .the detailed description of the results for each model , as well as an analysis of the conditions for the presence or lack of excitations , are provided next .we first considered the base network , eqs .( [ eq_net_i ] ) , which contains the positive regulator , the mrna , and the negative regulator .[ fig : examples](a ) shows the outcome of a typical stochastic simulation . for comparison ,the deterministic value is also shown .the stochastic and deterministic simulations exhibited remarkable differences .while the deterministic concentration was locked in a steady state , the stochastic integration showed a sustained excitatory behavior , where and were expressed in a series of anti correlated pulses .the presence of regular pulses of protein expression in the base model is revealed by the autocorrelation function of , as shown in the right panel of fig .[ fig : examples](a ) .the autocorrelation presents a pulsating behavior that provides quantitative information about the dynamics of the system .the negative anticorrelation peak indicates the characteristic width of the pulses , around min .the first positive peak provides the characteristic interval between consecutive pulses , around min , and that is in agreement with the maximum in the power spectrum .minimal transcription and translation rates of the negative regulator were required to identify an excitatory behavior with an observable pattern , as illustrated in fig .[ fig : timelag ] .( color online ) color map of the characteristic inter pulse interval for the base model as a function of the transcription and translation coefficients ( and respectively ) .a sustained excitatory behavior with a characteristic period appears once minimum translation and transcription rates are attained .the four surrounding plots , whose corresponding ( , ) values are indicated with dots on the color map , illustrate the behavior of the autocorrelation function , averaged over runs , for extreme combinations of transcription and translation rates . except for very low transcription and translation rates, the autocorrelation shows peaks of correlation that identify the characteristic inter pulse interval.,width=302 ]  below a minimal values of and the amount of was insufficient to implement an effective negative feedback : the pulses of protein expression became irregular , while the average interval between consecutive pulses increased .at extremely low transcription and translation rates the pulses disappeared altogether . above these extreme conditions we observed that in general a characteristic inter pulse period emerges in the base model for all tested sets of parameters ( fig .[ fig : timelag ] ) .the autocorrelation shows clear peaks that provide a characteristic period .the average period depends on the transcription and translation rates , and ranges between and min for realistic rates .the origin of the pulses can be understood by observing the mrna time trace . as shown in fig .[ fig : examples](a ) , the mrna is transcribed in a series of micro pulses that are subsequently amplified during translation and inherited by .the strong correlation between the pulses of mrna and is evident in the figure .we also observed a strong correlation between the mrna inter pulse time lags and the typical width of the pulses of . as seen in fig .[ fig : examples](a ) , only reached significant levels after a substantial decay in prior to a new pulse of .therefore the typical length of the pulses depends on the decay time of , which has been taken to be around min .the fast degradation also explains the small width of pulses compared to the typical time gap between consecutive pulses .finally , we note that the excitatory behavior of the network crucially depends on having a low number of mrnas . in the base model ,the average number of mrna copies was below for most of the explored parameter space ( figs .[ fig : ratio ] and [ fig : cov ] ) .( color online ) ratio between the stochastic level averaged over time and over 300 simulations and the deterministic steady state in the base model , as a function of the transcription and translation rates .contour lines show equal stochastic levels of ( black ) , ( dashed white ) and mrna ( dotted red ) .the black cross indicates the location of the example shown in fig .[ fig : examples](a).,width=287 ]   ( color online ) color map of the coefficient of variation of in the base model , as a function of the transcription and translation coefficients ( and respectively ) .contour lines show equal average stochastic levels of ( black ) , ( dashed white ) and mrna ( dotted red ) . exhibits maximum coefficient of variation at low and high .the black cross indicates the position of the example shown in fig .[ fig : examples](a ) .the four plots illustrate the behavior for four different values of the coefficient of variation , showing that the pulsating behavior of is maximum at high translation rates .,width=302 ]  this low average value is in agreement with measurements of transcripts copy number in mammalian cells , where it was found that many transcripts are present in less than one copy per cell on average .   to further test the impact of the mrna stochastic dynamics , we considered an alternative circuit design , the reduced model eqs .( [ eq_net_ii ] ) , where the mrna was assumed to be in quasi equilibrium and therefore was not explicitly included in the circuit .[ fig : examples](b ) shows an example of stochastic simulation run with the same biochemical parameters as the base model in fig .[ fig : examples](a ) .the protein dynamics was considerably different in both models .while the pulses were clearly present in the base model , no trace of pulses could be found in the reduced model .similar conclusions are reached by comparing the auto correlation and power spectrum in both models .opposed to the base model , intrinsic mrna fluctuations were not allowed in the reduced model .hence , the protein levels showed very uniform patterns with only small deviations around the deterministic steady state .the protein levels were also characterized by a very small coefficient of variation .the latter is in agreement with the experimental evidence that when the contribution of external factor affecting cell to cell variability is substracted , noise in protein expression is dominated by the stochastic production and destruction of mrnas .we stress that both networks were simulated with the same algorithm , kinetic parameters , and initial conditions .  an additional difference between both modelswas given by the average of the stochastic simulations .the average , which provides an insight into cell population dynamics , converged towards the deterministic trajectory in the reduced model , but not in the base model .the deviation in the base model became especially important at low transcription rates .[ fig : ratio ] shows the ratio of the stochastic copy number ( averaged over time and over simulations ) , and the deterministic steady state . for transcription rates below transcripts per minute ,the deterministic description clearly underestimated the real circuit behavior .the fact that the divergence appeared at low transcription rates , and that it was absent in the reduced model , hints at the mrna stochastic dynamics as the source of this divergence .this is another interesting example of a biological network presenting a deviation between the stochastic average and the deterministic equations .such deviations are commonly expected in non linear systems and/or systems containing a small number of molecules , although more generic systems can also present large deviations .   to achieve a given protein concentration an organism can adopt diverse strategies characterized by different transcription and translation rates .one strategy consists of producing a few mrna transcripts and translating them efficiently .alternatively , the organism can transcribe a larger number of transcripts and translate each one of them inefficiently .the former strategy is energetically favorable since a lower number of transcripts has to be produced ; however it was suggested that it may lead to a noisier pattern of protein expression .  in order to determine the influence of these two strategies on noise , we ran simulations for different transcription and translation rates in the base model .for each set of parameters we computed the coefficient of variation , eq .( [ eq : cov ] ) , as shown in fig .[ fig : cov ] . as a visual help we also plotted curves corresponding to equal average levels of , and of stochastic simulations at some extreme values are shown in the figure , as well .two axes of variation can be roughly identified . along the main diagonal ( from the bottom left to the upper right corner )the average level of grows as the transcription and translation rates increase .the opposite behavior is seen for , reflecting the negative feedback loop between both proteins . at the same time , the coefficient of variation of remains practically constant along this axis . along the other diagonalthe coefficient of variation of changes from a minimum at high transcription and low translation rates and reaches a maximum at low transcription and high translation rates .this behavior is seen for the coefficient of variation of as well ( data not shown ) .we also observed that higher levels of noise are obtained when the average number of mrna transcripts is low , and vice versa ( see the red equi lines in fig . [fig : cov ] ) , demonstrating the importance of the mrna fluctuations to the excitatory behavior .experimental measurements of protein expression reveal that in many cases the variance in protein levels is roughly proportional to the mean .this trend is usually explained in terms of an increase in the transcription noise with the expression level .yet , some genes are observed to deviate from this scaling law .our results show a general agreement with the previous observation ; at a fixed transcription rate , the noise increases with the expression level . however, our results also show that noise fundamentally depends on the interplay between the transcription and translation rates .for instance , along the lines of constant protein levels the coefficient of variation reaches a maximum at low transcription and high translation rates .this suggests that the variability in protein expression depends not only on the average expression level , but also on the ratio of transcription versus translation efficiency .additional factors such as non linear regulatory loops ( as in our model system ) may also lead to deviations from the scaling rule .we want to address here the question of how the presence of a regulatory rna transcript , which targets the mrna transcript , modifies the excitatory behavior inherited from the mrna stochastic fluctuations .[ fig : examples](c ) shows a typical stochastic simulation of the extended model , that includes a regulatory rna , run with the same biochemical parameters as the base model , fig .[ fig : examples](a ) . while pulses of protein expression were present in both networks , the extended model , eqs .( [ eq_net_iii ] ) , showed broader peaks with significantly lower amplitude relative to the average expression .a comparison of the auto correlation in both models showed a weaker spiky behavior in the extended model .similarly , while the power spectrum shared a similar trend with the base model , it lacked the clear maximum present in the latter one .the deterministic levels of ( red line ) is also shown for both models in fig .[ fig : examples ] .while none of the systems was correctly described by the deterministic solution , the departure from the deterministic dynamics was smaller in the model with regrna .this suggests that , in the presence of a regulatory rna molecule , the individual cell s dynamics is more robust to transcript fluctuations , hence leading to a reduced cell to cell variability .  to verify that the reduced excitability in the extended model is due to the action of the regrna , we tested the dynamics of the extended model at different regrna transcription rates and compared it with the base model dynamics . if our hypothesis is correct , the two systems should show differences in their excitatory dynamics as the amount of regrna increases .simulations started at a minimum sense transcription rate of transcripts / min in order to have a sufficient amount of for effective repression of . at low regrna copy numbers ,the coefficient of variation was maximized at low sense transcription rate , in agreement with the qualitative behavior of the base model .however , at high regrna copy number ( above transcripts ) , the coefficient of variation became independent of the sense transcription rate .the origin of this difference becomes clear when one observes the time traces of the different rna transcripts in both models .[ fig : buffering](b ) shows the mrna time trace in the base model , where no regrna is present in the system .clear mrna micro pulses with a typical length of the order of the mrna half life are observed .these micro pulses are amplified during translation , producing the observed pulses of protein expression .an equivalent plot for the extended model , fig .[ fig : buffering](c ) , reveals a completely different dynamics .the regrna serves as a capacitor : it sequesters sense mrna when there are copies available , and releases them back when there are none . since the typical binding and unbinding rates are much faster than the rna half life , this sequester and release process is repeated many times during a typical mrna micro pulse , effectively erasing memory from any previous mrna stochastic state . for completeness ,[ fig : buffering](c ) also shows the concentrations of sense , regrna and sense regrna complex .notice also that the average mrna concentration is significantly closer to the deterministic trajectory in the extended model , which evidences the reduction of the stochastic fluctuations in the network .  based on these observationswe conclude that the regrna is able to filter out some of the stochastic fluctuations and induce a smoother protein expression pattern .such noise dampening capacity was predicted before in a circuit in which a shared transcription factor regulates a pair of sense and a highly expressed antisense transcript .the filtering capability of the regrna depends crucially on having fast coupling rates between both rna strands .[ fig : complex](a ) shows the coefficient of variation of in the extended model as a function of the sense regrna binding and unbinding rates , and for two different values of the regrna transcription rate .the lowest coefficient of variation is reached in the panel with higher regrna transcription rate .the figure also shows curves of equal amount of sense , regrna and complex . in both panels the minimum coefficient of variationwas reached as a trade off between having fast binding and unbinding dynamics , and having the highest number of regrna molecules ( magenta line ) .we conclude that the noise buffering capability needs both a sufficient number of regrna transcripts and a fast kinetics to be efficient .this is clearly illustrated in fig .[ fig : complex](b ) , which shows examples of stochastic simulations for different combinations of binding and unbinding rates , and for two extreme values of the regrna transcription rate .indeed , there is practically no buffering for low regrna numbers ( left panels ) , and the time evolution of both and shows a pulsating behavior even at large binding dynamics . on the contrary , for high regrna numbers ( right panels ) , such a pulsating behavior is observed only at very low binding dynamics , to rapidly disappear as soon as the binding unbinding kinetics , and therefore the buffering capacity , increases .we have analyzed the response of three different network architectures of a feedback loop and we have observed remarkable differences among them that emerge from the rna dynamical description .the fluctuations inherited from the transcription process can be amplified during translation and produce pulses of protein expression .the deterministic approach fails to describe such dynamics both at the single cell level , where no trace of excitatory behavior is observed , and at the cell distribution level , where the deterministic steady state underestimates the protein levels .deviations between the stochastic average and the deterministic equations may have different origins . in our casethe difference emerges as a result of the non linear terms , which induce correlations that are not present in the deterministic framework .these correlations become particularly important at low mrna levels , where the largest deviation is observed .indeed , several works in the literature point at the possibility of having very few transcripts per cell , in some cases , less than 1 on average . at such low copy numbers ,our findings stress the importance of using stochastic models to accurately describe the network dynamics .the excitatory behavior was observed only after simulating the circuit dynamics with the gillespie algorithm .there are alternative methods to model stochastic fluctuations .the langevin approach , for instance , adds a small stochastic term to the continuous deterministic equations to account for noise .this approach is not suitable in our system because of the strong rna fluctuations , which prevents the characterization of a smooth continuous background .an alternative approach is based on the linear noise approximation of the master equation .it assumes that the system contains a large number of particles and models noise as a continuous linear gaussian perturbation .this linearized description has been applied to processes involving two molecular species .opposed to the linear approaches , the gillespie algorithm provides a simple yet powerful method to obtain stochastic and dynamical solutions compatible with the full master equation .it does not require external assumptions on the noise , the number of molecules and species , or the dynamical regime of the system .the protein pulses originate in the rnas stochastic fluctuations .this result is supported by the observed correlation between the typical protein pulse length or frequency and the mrna fluctuations , as shown in fig .[ fig : examples ] .additional evidence is provided by the fact that when the rna intrinsic randomness is neglected , the stochastic behavior converges towards the deterministic concentration .these results are in agreement with experimental evidences that mrna fluctuations are a fundamental source of noise in protein expression .  to produce a given average protein concentration the cellular machinery may choose among different transcription and translation rates .minimized variability among cell population is obtained with a combination of high transcription and low translation rates , as verified for an auto regulated gene in steady state using a linearized stochastic model .our results generalize this observation .we have analyzed a dynamical , stochastic negative feedback loop and found , similarly , that noise is minimized at high transcription and low translation rates , as shown in fig .[ fig : cov ] .  the presence of a non coding regulatory rna may help buffer the mrna fluctuations while allowing the cell to maintain low rates of transcription . in this workwe have considered that a regulatory rna ( regrna ) binds to the sense transcript and sequesters it from the cellular environment , thus preventing its translation .a regrna molecule sequesters a mrna when there are some coding transcripts available in the medium , and releases them back when there are none .if this process is repeated sufficient times during a typical mrna half life , then the memory of previous states inherited from the transcription process can be erased and , therefore , noise can be partially buffered . in this way , regrna contributes to reduce the temporal variability at the single cell level and , consequently , also the cell to cell variability . a requisite for this mechanism to work efficiently is fast binding and unbinding ( compared to the typical rna life time ) between the regulatory rna and its target mrna , as shown in fig .[ fig : buffering ] . as a result of this buffering ,the extended model is less excitable .the circuit still shows pulses , yet compared to the base model where there is no regrna , the pulses are broader and with smaller amplitude , and they appear at a lower frequency .a prime example of an oscillating negative feedback loop is provided by the p53mdm2 regulatory network .this system has been shown to oscillate at the single and cell population levels ( sustained oscillations versus damped oscillations respectively ) .the maintenance and shape of the oscillations have been linked to two upstream signaling kinases as well as to an additional negative feedback loop .interestingly however , a single nucleotide polymorphism ( snp309 ) found in the mdm2 promoter that results in higher levels of mdm2 mrna and protein has been shown to disrupt the oscillations of p53 and mdm2 protein .this finding supports the idea that rnas could play an important role in the oscillatory dynamics in this network , as a high number of mdm2 mrnas would minimize the importance of the intrinsic stochastic fluctuations , and thus attenuate the pulses of protein expression .  while this work has considered a negative feedback loop , the mechanism that we have described is more general and apply to a wide variety of regulatory networks .we have shown that the rna dynamics is a fundamental source of intrinsic noise , suggesting that a realistic description of genetic networks requires the stochastic modeling of the transcription stages of protein expression .we thank arnold levine , yael aylon , yolanda espinosa parrilla , michal lapidot and reut shalgi for interesting discussions .we especially thank gil hornung and orna dahan for a critical reading of the manuscript and for raising many interesting observations .we acknowledge the support of the clore center for biological physics , the center for complexity science , the minerva foundation , the european research council ( erc ) ideas grant and the ec fp7 funding ( oncomirs , grant agreement number 201102 ) .

Abstract: single cell experiments of simple regulatory networks can markedly differ from cell population experiments . such differences arise from stochastic events in individual cells that are averaged out in cell populations . for instance , while individual cells may show sustained oscillations in the concentrations of some proteins , such oscillations may appear damped in the population average . in this paper we investigate the role of rna stochastic fluctuations as a leading force to produce a sustained excitatory behavior at the single cell level . opposed to some previous models , we build a fully stochastic model of a negative feedback loop that explicitly takes into account the rna stochastic dynamics . we find that messenger rna random fluctuations can be amplified during translation and produce sustained pulses of protein expression . motivated by the recent appreciation of the importance of non coding regulatory rnas in post transcription regulation , we also consider the possibility that a regulatory rna transcript could bind to the messenger rna and repress translation . our findings show that the regulatory transcript helps reduce gene expression variability both at the single cell level and at the cell population level .