realistic networks show heterogeneous interconnection delays . the cortex is a paradigmatic example of such networks . in the brain ,neurons form large - scale assemblies ( termed neural populations ) subject to an intense noise .they communicate through the emission of spikes ( action potentials ) that are transmitted through synapses that can be chemical or , less often , electrical .the transport of spikes through the axons takes a specific time , function of the length of the axon and of the time taken for the signal to be transmitted through synapses , and therefore are highly heterogeneous .moreover , chemical synapses modify the membrane potential of the post - synaptic neuron through a complex mechanism of release and binding of neurotransmitters .this mechanism takes a specific time , and results in random variations of the current transmitted through the different synapses and also from spike to spike .models describing the emergent behavior arising from the interaction of neurons in large - scale networks largely overlooked both phenomena of heterogeneous delays and stochastic synapses .most models have relied on continuum limits ever since the seminal work of wilson and cowan and amari .such models tend to represent the activity of the network through a macroscopic variable , the population - averaged firing rate , that is generally assumed to be deterministic .many analytical and numerical results and properties have been derived from these equations and related to cortical phenomena , for instance for the problem of spatio - temporal pattern formation in spatially extended models ( see e.g.  ) .this approach tends to implicitly make the assumption that the effect of noise and heterogeneities vanishes in large populations .a different approach has been to study regimes where the activity is uncorrelated .a number of computational studies on the integrate - and - fire neuron showed that under certain conditions neurons in large assemblies end up firing asynchronously , producing null correlations . in these regimes , the correlations in the firing activity decrease towards zero in the limit where the number of neurons tends to infinity .the emergent global activity of the population in this limit is deterministic , and evolves according to a mean - field firing rate equation .however , these states arise in sparsely connected networks which is not the case of cortical columns . in order to go beyond asynchronous states and take into account the stochastic nature of the firing and how this activity scales as the network size increases, different approaches have been developed , such as the population density method and related approaches .most of these approaches involve expansions in terms of the moments of the resulting random variable , and the moment hierarchy needs to be truncated which is a quite hard task that can raise a number of technical issues ( see e.g. ) .homogeneous spatially extended systems of neurons were recently analyzed using the theory of probability  .these were the first studies considering individual delays between neurons .in these studies , the propagation delays between neurons were considered constant , and in these cases , no specific delay - induced transition was identified .heterogeneous networks have been considerably less analyzed .one notable exception is the work of sompolinsky and collaborators  recently analyzed from a mathematical viewpoint using large deviations techniques  , where the authors considered random heterogeneous connections between neurons .the main question we shall address in this article is the effect of heterogeneous delays on the macroscopic dynamics of such neuronal areas in the presence of stochastic coupling .although it now firmly established that delays are fundamental for shaping the activity of large - scale neuronal networks  , very little is understood about the type of delays involved in such networks .the question of how these individual delays translate in macroscopic limits , and how the heterogeneity in the distribution of delays affect macroscopic behaviors , hence has deep implications , and is still largely open . at the microscopic scale, it is relatively clear that transmission delays are well approximated by constant delay depending on the distance between neurons and on the typical synaptic time scale .these constant delays hence vary depending on the pair of neurons considered and are related to the topology of the network in the physical space .  considering random heterogeneous delays between neurons , we establish the limit , as the number of neurons tend to infinity , of the network equations .we will show that the propagation of chaos property occurs in these systems , and that the asymptotic macroscopic behavior in the limit where the number of neurons tends to infinity is described by a mckean - vlasov equation with distributed delays , the delay measure being exactly the statistical distribution of individual delays .the dynamics of the obtained mckean - vlasov equations with distributed delays is very hard to analyze for general neuron models . in order to uncover the macroscopic dynamics of the neurons and their dependence on the distribution of delays , we eventually instantiate a particularly suitable model , the classical firing - rate model . in that case , that solutions are gaussian , and the mean and standard deviation satisfy a closed system of delay differential equations with distributed delays .this reduction allows using classical tools of the domain of infinite - dimensional dynamical systems to show generic qualitative transitions between different qualitative states , as a function of the delays and of the variance of the synaptic weights .in this section , we describe the mathematical formalism used throughout the manuscript .we shall analyze the dynamics of a relatively general neuron model , valid for most usual models used in computational neuroscience .the state of each neuron in the network is classically described by a -dimensional variable , typically corresponding to the membrane potential of the neuron and possibly additional variables such as those related to ionic concentrations and gated channels ( see e.g.  ) .we consider networks composed of a finite number of populations .the populations differ through the intrinsic properties of each neurons and the input they receive .we assume that the number of neuron in each population , denoted increases as the network size increases , and tends to infinity in the limit considered , which we will be denoting .we define the population function that associates to a neuron the population it belongs to : .let us consider a single neuron belonging to population .the state of the neuron , , has an intrinsic dynamics governed by a nonlinear drift function ( including the intrinsic dynamics and the deterministic inputs ) and a diffusion matrix .each individual process takes values in , and when there is no interaction , is governed by the equation : where are independent brownian motions .this equation corresponds to the intrinsic dynamics of each individual diffusion , i.e. the behavior of a single neuron when disconnected of the network .when included in the network , the neurons interact with all other neurons and this interaction is modeled through a continuous function , multiplied by a coefficient , called the synaptic weight .these synaptic weights are themselves stochastic , due to the nature of the synaptic transmission .incoming interactions are known to be bounded , the total incoming connectivity from a given population ( say ) to a single neuron from population has a fixed averaged and standard deviation .we consider that the synaptic weights are stochastic , and are assumed to be equal , for a neuron in population and a neuron in population , to where are independent -dimensional `` white noise '' processes related to the independent brownian motions introduced to neuron when the sign of changes .other more biologically relevant problems involve enforcing the synaptic weight not to change sign by modeling it as the solution of particular stochastic differential equation that does not change sign , such as the cox - ingersoll - ross process , and by fit our framework by formally adding the synaptic weights of neuron in the state variable of neuron . ] .the delay occurring in the interaction between neuron and , and related to the information transmission through the axons and the synapses , depend on the precise distance between neurons and .we assume that for any fixed neurons , the delays are pairwise independent ( and independent of the brownian motions involved in the network dynamics ) and identically distributed in each population , with a law denoted .this is a reasonable assumption from the modeling viewpoint . indeed ,neural populations can be considered to form certain clusters in the space in which the specific locations of neurons can be considered independent , and delays hence depend on the typical distance between the populations , their dispersion , and on the characteristic time constants of the synapse . from the referential of one given neuron , the resulting delays are hence independent . from the global network viewpoint , these delays are of course correlated ( distance between two points are symmetrical , topological relationship between neuronal locations induces relationship between distances , e.g. triangular inequality ) .let us denote by the largest possible delay in the network .all delay distributions hence charge only the interval  ] ( i.e. on the variable ) ] with values in .let us denote by ,e^p) ] an initial condition of the network system. under the assumptions of the section , there exists a unique strong solution to the network equations .the proof of this proposition is a direct application of mao s result  , and essentially uses the same arguments as those of the proof theorem [ thm : existenceuniqueness ] .the interested reader is invited to follow the steps of this demonstration to prove proposition [ pro : existenceuniquenessnetwork ] .we address the problem of the asymptotic behavior , as the number of neurons goes to infinity , of these network equations , in particular the question of the propagation of chaos property in this randomly delayed , randomly interacting system .assuming translation invariance in the neural field , the delays distribution is only function of the neural population belongs to . in that case , for any realization of the synaptic delays , the propagation of chaos property states that , provided that the initial condition on the network is chaotic ( i.e. for all in population , the initial condition } ] where is the density of , if it exists .the same result holds when the neural field is not translation invariant ( i.e. varies depending on the choice of neuron in population ) , but in that case the result holds for the law of the limit distribution averaged on the law of the delays .  in that view, the equation is an implicit equation on the law of ( denoting when no confusion is possible the vector ) . in order to prove the propagation of chaos property, we will use the coupling method ( see e.g.  ) .the proof is in two steps : ( i ) we prove that the equation has a unique solution , and ( ii ) that the law of converges towards the law of and more precisely that for any finite set of neurons the law of ) ] where the processes are independent and have the law of given by .in this section , we show existence and uniqueness of solutions for the delayed mckean - vlasov equation  that constitute the putative mean - field limit .let us denote by the set of probability distributions on the set continuous functions \mapsto e^{p} ] a square - integrable process . under the assumptions ( h1)-(h4 ) , there exists a constant depending on the parameters of the system and on the horizon , such that for any solution of the mean - field equation with initial condition : }{\mathbbm{e } } [ \vert x_t \vert^2]\leq c(t).\ ] ]  the proof is based on the application of it s formula for the squared modulus of , standard inequalities and gronwall s lemma .let be a solution of the mean - field equations , and the first time the process exceeds the quantity .due to the non - standard nature of the equation , let us underline the fact that it s formula is valid , i.e. that is a semimartingale . by definition ,it is clear that both and are measurable for all  ] by . we hence have for all : & \leq { \mathbbm{e}}[\vert \zeta_0^{\alpha}(0 ) \vert^2 ] + 2\int_0^{t\wedge \tau_n } \big\ { k(1+{\mathbbm{e}}[\vert x_s^{\alpha}\vert^2 ] )  + p \sqrt{\tilde{k } } ( 1+{\mathbbm{e}}[\vert x_s^{\alpha}\vert^2 ] ) \\   &\qquad + \frac { \tilde{k } } 2  \sum_{\gamma=1}^p \int_{-\tau}^0  ( 1+{\mathbbm{e}}[\vert x_s^{\alpha}\vert^2 ] )  d\eta_{\alpha\gamma}(u )  \big\}\,ds \\    & \leq { \mathbbm{e}}[\vert \zeta_0^{\alpha}(0 ) \vert^2 ] + 2\int_0^{t\wedge \tau_n } ( k+  \sqrt{\tilde{k } } p + \frac { \tilde{k } } 2 p ) ( 1+{\mathbbm{e}}[\vert x_s^{\alpha}\vert^2])\,ds   \end{aligned}\ ] ] summing over and applying gronwall s lemma directly yields : } { \mathbbm{e}}[\vert x_t\vert^2 ] \leq { \mathbbm{e}}[\vert \zeta_0(0)\vert^2 ] + { \mathbbm{e}}[1+\vert \zeta_0(0)\vert^2 ] ( e^{k ' t}-1)\ ] ] with , and hence , letting , we conclude that : } { \mathbbm{e}}[\vert x_t\vert^2 ] \leq \max(\sup_{[-\tau,0 ] } { \mathbbm{e}}[\vert\zeta_0(s)\vert^2 ] \;\;,\;\ ; { \mathbbm{e}}[\vert \zeta_0(0)\vert^2 ] + { \mathbbm{e}}[1+\vert \zeta_0(0)\vert^2 ] ( e^{k ' t}-1))\ ] ]  we now prove the existence and uniqueness theorem , using this boundedness property of the possible solutions .[ thm : existenceuniqueness ] for any ,e^{p}) ] for any .we start by showing the existence of solutions , and then prove the uniqueness property .we recall that by application of lemma [ lem : solul2 ] , any possible solution has a bounded second moment . the proof is based on a classical contraction argument on iterates of the map .there are two main differences with usual proofs .first is the fact that our functions are not globally lipschitz - continuous , issue which is solved through a truncation argument and using the result of the _ a priori _ estimate of lemma [ lem : solul2 ] , see  .second is the fact that we are considering delayed equations , which leads us to deal with uniform in  ] a given stochastic process .we consider that the drift and diffusion functions are -lipschitz continuous ( up to truncation ) .we introduce the sequence of probability distributions on defined by induction as .we denote by a sequence of processes independent of the collection of processes and having the same law .we start by controlling one of the components of the sequence of processes , and for compactness of notations denote the component of .we decompose into six elementary terms the difference : \eta_{\alpha\gamma}(u ) \ , ds \\      &\quad\quad + \int_0^t \sum_{\gamma=1}^p  \int_{-\tau}^0 { \mathbbm{e}}_z\big [ b_{\alpha\gamma}({}^{\alpha}x^{k-1}_s , { } ^{\gamma}z^{k}_{s+u } ) - b_{\alpha\gamma}({}^{\alpha}x^{k-1}_s , { } ^{\gamma}z^{k-1}_{s+u } ) \big ] d\eta_{\alpha\gamma}(u ) \ , ds \\      &\quad\quad + \int_0^t \big ( g_{\alpha}(s,{}^{\alpha}x^{k}_s)-g_{\alpha}(s,{}^{\alpha}x^{k-1}_s ) \big ) \ , dw_s^{\alpha}\\     & \quad\quad + \sum_{\gamma=1}^p \int_0^t \int_{-\tau}^0 { \mathbbm{e}}_z \big [ \beta_{\alpha\gamma}({}^{\alpha}x^{k}_s ,  { } ^{\gamma}z^{k}_{s+u } )  - \beta_{\alpha\gamma}({}^{\alpha}x^{k-1}_s , { } ^{\gamma}z^{k}_{s+u } ) \big]d\eta_{\alpha\gamma}(u ) \cdot db_s^{\alpha\gamma } \\      &\quad\quad + \sum_{\gamma=1}^p\int_0^t \int_{-\tau}^0 { \mathbbm{e}}_z  \big [ \beta_{\alpha\gamma}({}^{\alpha}x^{k-1}_s , { } ^{\gamma}z^{k}_{s+u } ) - \beta_{\alpha\gamma}({}^{\alpha}x^{k-1}_s , { } ^{\gamma}z^{k-1}_{s+u } ) \big]d\eta_{\alpha\gamma}(u ) \cdot db_s^{\alpha\gamma } \\     & \stackrel{\text{def}}{= } a_t^{\alpha } + b_t^{\alpha } + c_t^{\alpha } + d_t^{\alpha } + e_t^{\alpha } + f_t^{\alpha }   \end{aligned}\ ] ] where we simply identify each of the six terms , , , , and with the corresponding expression in the previous formulation . by a simple convexity inequality ( hlder )we have : and treat each term separately .the term is easily controlled using cauchy - schwarz inequality , fubini identity and standard inequalities and we obtain : } } \vert a_s \vert^2\big ]                      & \leq k^2 \,t \ , \int_0^t { \mathbbm{e}}\big [ \sup_{-\tau\leq u\leq s } \vert x^{k}_u - x^{k-1}_u \vert^2\big ] \ , ds   \end{aligned}\ ] ] similarly , the martingale term is bounded using the burkholder - davis - gundy theorem to the -dimensional martingale and we obtain :              & \leq 4 k^2 \, \int_0^t { \mathbbm{e}}\big [ \sup_{-\tau \leq u\leq s } \vert x^{k}_u - x^{k-1}_u \vert^2 \big ]  \ , ds    \end{aligned}\ ] ]  let us now deal with the deterministic interaction terms and .we have : )\,d\eta_{\alpha\gamma}(u)\,ds \right\vert^2\\     & \leq \sum_{\alpha=1}^p t\,p\ ,  \int_0^t \sum_{\gamma=1}^p \int_{-\tau}^0 { \mathbbm{e}}_z\left [ \vert b_{\alpha\gamma}({}^{\alpha}x^{k}_s,{}^{\gamma}z^{k}_{s+u } ) - b_{\alpha\gamma}({}^{\alpha}x^{k-1}_s,{}^{\gamma}z^{k}_{s+u})\vert^2\right ] )     \,d\eta_{\alpha\gamma}(u)\,ds\\     & \leq t p^2  l^2 \int_0^t \vert x^k_s - x^{k-1}_s\vert^2\,ds \leq t p^2  l^2 \int_0^t \sup_{-\tau \leq u\leq s}\vert x^k_u - x^{k-1}_u\vert^2\,ds   \end{aligned}\ ] ] hence easily conclude that }\vert b_s \vert^2 ] \leq t p^2  l^2 \int_0^t { \mathbbm{e}}[\sup_{-\tau \leq u\leq s}\vert x^k_u - x^{k-1}_u\vert^2]\,ds\ ] ]  and similarly , since and have the same law as and , }\vert c_s \vert^2 ] \leq t p^2  l^2 \int_0^t { \mathbbm{e}}[\sup_{-\tau \leq u\leq s}\vert x^k_u - x^{k-1}_u\vert^2]\,ds\ ] ]  eventually , the terms and use burkholder - david - gundy ( bdg ) inequality in place of cauchy - schwarz together with similar arguments as used for and . using the independence of the brownian motions , bdg inequality yields , for the term ( and similarly for the term ) : }\vert \theta_s \vert^2 ]\leq 4 p^2  l^2 \int_0^t { \mathbbm{e}}[\sup_{-\tau \leq u\leq s}\vert x^k_u - x^{k-1}_u\vert^2]\,ds\ ] ] for equal to or .putting all these evaluations together , we get : } \vert x^{k+1}_s - x^k_s \vert^2 \big ] \leq 6(t+4)(k^2 + 2p^2\,l^2\ , ) \int_0^t { \mathbbm{e } } [ \sup_{-\tau \leq u\leq s } \vert x^{k}_u - x^{k-1}_u \vert^2 ] ds\ ] ] moreover , since for  ] and hence the a.s .uniform convergence of the partial sums : on  ] whence on  ] .we decompose this expression into the sum of the eight terms involved in equation using hlder s inequality and upperbound each term separately .the terms and are treated exactly as in the proof of theorem [ thm : existenceuniqueness ] , and the control all the terms except and essentially uses on the same ingredients as in the proof of theorem [ thm : existenceuniqueness ] .we denote a common lipschitz constant for and and deduce , similarly to the analysis performed in the proof of theorem [ thm : existenceuniqueness ] , the following inequalities : & \leq k^2\ , t\ , \int_0^{t\wedge \tau_u } { \mathbbm{e}}[\sup_{-\tau\leq u\leq s } \vert x_u^{i , n}-\bar{x}_u^i\vert^2 ] \ , du\\       { \mathbbm{e}}[\sup_{0\leq s\leq t\wedge \tau_u } \vert b_s(n ) \vert^2 ] & \leq 4\ , k^2\ , \int_0^{t\wedge \tau_u } { \mathbbm{e}}[\sup_{-\tau\leq u\leq s } \vert x_u^{i , n}-\bar{x}_u^i\vert^2 ] \ , du\\       { \mathbbm{e}}[\sup_{0\leq s\leq t\wedge \tau_u } \vert c_s(n ) \vert ^2 ] & \leq t l^2 p^2  \int_0^{t\wedge \tau_u }  { \mathbbm{e}}[\sup_{-\tau\leq u\leq s}\vert x^{i , n}_u-\bar{x}^i_u \vert^2 ] \ , ds\\       { \mathbbm{e}}[\sup_{0\leq s\leq t\wedge \tau_u } \vert d_s(n ) \vert ^2 ] & \leq t\,l^2 p^2  \int_0^{t\wedge \tau_u } \max_{j=1\cdots n } { \mathbbm{e}}[\sup_{-\tau \leq u \leq s}\vert x^{j , n}_u-\bar{x}^j_u \vert^2 ] \ ,ds\\       { \mathbbm{e}}[\sup_{0\leq s\leq t\wedge \tau_u}\vert f_s(n ) \vert ^2 ] & \leq 4 l^2 p^2  \int_0^t  { \mathbbm{e}}[\sup_{-\tau\leq u\leq s}\vert x^{i , n}_u-\bar{x}^i_u \vert^2 ] \ , ds\\       { \mathbbm{e}}[\sup_{0\leq s\leq t\wedge \tau_u}\vert g_s(n ) \vert ^2 ] & \leq 4 l^2 p^2  \int_0^t \max_{j=1\cdots n } { \mathbbm{e}}[\sup_{-\tau\leq  u \leq s}\vert x^{j , n}_u-\bar{x}^j_u \vert^2 ] \ , ds     \end{cases}\ ] ] let us for instance treat the case of for the sake of clarity , all other terms are treated along the same lines .we have : & \leq p \sum_{\gamma=1}^p { \mathbbm{e}}\bigg[\sup_{0 \leq s\leq t } \big\vert \int_0^s \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } \beta_{\alpha\gamma}(x^{i , n}_u , x^{j , n}_{u-\tau_{ij}})\\   & \quad -\beta_{\alpha\gamma}(\bar{x}^{i}_u , x^{j , n}_{u-\tau_{ij } } ) db^{i\gamma}_u\big\vert^2\bigg]\\   \text{(bdg)}&\leq4p \sum_{\gamma=1}^p { \mathbbm{e}}\left[\int_0^t \left\vert\frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } \beta_{\alpha\gamma}(x^{i , n}_s , x^{j , n}_{s-\tau_{ij}})-\beta_{\alpha\gamma}(\bar{x}^{i}_s , x^{j , n}_{s-\tau_{ij } } ) \right\vert^2d_s\right]\\   \text{(cauchy - schwartz)}&\leq 4p \sum_{\gamma=1}^p \int_0^t \frac 1 { n_{\gamma } }  \sum_{p(j)=\gamma } { \mathbbm{e}}\left[\left\vert \beta_{\alpha\gamma}(x^{i , n}_s , x^{j , n}_{s-\tau_{ij}})-\beta_{\alpha\gamma}(\bar{x}^{i}_s , x^{j , n}_{s-\tau_{ij}})\,\right\vert^2\right]\,ds\\   \text{(assumption~\ref{assump : loclipschb } ) } & \leq 4p^2 l^2 \int_0^t { \mathbbm{e}}\left[\left\vert x^{i , n}_s-\bar{x}^i_s\right\vert^2\right]\,ds\\   & \leq 4 p^2 l^2  \int_0^t { \mathbbm{e}}\left[\sup_{-\tau \leq u\leq s}\vert x^{i , n}_u-\bar{x}^i_u \vert^2\right ] \ , ds .   \end{aligned}\ ] ]  in the decomposition , the two terms and are of a new form and remain to be evaluated .both term involves the difference between an empirical mean of a function of processes and an expectation term , and all have bounded second moment thanks to proposition [ lem : solul2 ] and assumption [ assump : bbound ] .we have : \ } & = { \mathcal{e}}\big\{{\mathbbm{e}}[\sup_{0\leq s\leq t } \vert \int_0^s \sum_{\gamma=1}^p  \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } ( b_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})-\int_{-\tau}^0{\mathbbm{e}}_z[b_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma } ) ] ) \,d\eta_{\alpha\gamma}(u)\ , ds\vert^2]\big\}\\     \label{eq : ethtcontrol } & \leq t p  \sum_{\gamma=1}^p \int_0^t { \mathcal{e}}\big\ { { \mathbbm{e}}[\bigg \vert \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } b_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})- \int_{-\tau}^0{\mathbbm{e}}_z[b_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma})\ , d\eta_{\alpha\gamma}(u ) ] \bigg\vert^2 ] \, ds\big\ }   \end{aligned}\ ] ] and using burkholder - davis - gundy \big\ } \leq 4 p  \sum_{\gamma=1}^p \int_0^t  { \mathcal{e}}\big\{{\mathbbm{e}}[\bigg \vert \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } \beta_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})- \int_{-\tau}^0{\mathbbm{e}}_z[\beta_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma } ) \ ,d\eta_{\alpha\gamma}(u ) ] \bigg\vert^2]\big\}\ , ds\ ] ] each of these expressions involve the processes which , under the tensor product law of the brownian motion and the delays , are independent and identically distributed population - wise .moreover , we have : \eta_{\alpha\gamma}(u)]\vert^2]\big\ } \\= \frac 1 { n_{\gamma}^2 } \sum_{k , l = 1}^{n_{\gamma } } { \mathcal{e}}\big\{{\mathbbm{e}}\big[(\theta(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})-{\mathbbm{e}}_{z,\tilde{\tau}_{\alpha\gamma}}[\theta(\bar{x}^i_s , z_{s-\tilde{\tau}_{\alpha\gamma}}^{\gamma})])^t \cdot ( \theta(\bar{x}^i_s,\bar{x}^k_{s-\tau_{ij}})-{\mathbbm{e}}_{z,\tilde{\tau}}[\theta(\bar{x}^i_s , z_{s-\tilde{\tau}_{\alpha\gamma}}^{\gamma})])\big]\big\ }   \end{gathered}\ ] ] where , noting that the expectation \} ] . in the above expression, denotes a random variable with law independent of the sequence of delays and of the brownian motions .it is now easy to show that all the terms of the sum corresponding to indexes and such that the three conditions , and are satisfied are null .the simpler way to see this property is to write the expectations as integrals with respect to the measure and observe that all terms annihilate , using the mutual independence of the processes , of the random variables , and the independence between the processes and the delays .therefore , there are no more than non - null terms in the sum ( in the case there are just non - null terms ) , and moreover , all of these terms are uniformly bounded .the terms related to indexes satisfy the inequality : \right \vert^2\right]\right\ } & \leq 2\,{\mathcal{e}}\left\{{\mathbbm{e}}\left[\left \vert\theta(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})\right \vert^2 + \left \vert{\mathbbm{e}}_{z,\tilde{\tau}_{\alpha\gamma}}[\theta(\bar{x}^i_s , z_{s-\tilde{\tau}_{\alpha\gamma}}^{\gamma})]\right \vert^2\right]\right\}\\         & \leq 4 \tilde{k } ( 1+c_2(s ) )   \end{aligned}\ ] ] where is the upperbound of the norm of given by lemma [ lem : solul2 ] .the terms related to the cases ( or symmetrically ) are bounded by the same constant , since we have for all such that , by cauchy - schwartz inequality : \right ) ^t \cdot \left ( \theta(\bar{x}^i_s,\bar{x}^k_{s-\tau_{ij}})-{\mathbbm{e}}_{z,\tilde{\tau}_{\alpha\gamma}}[\theta(\bar{x}^i_s , z_{s-\tilde{\tau}_{\alpha\gamma}}^{\alpha})]\right ) ^t \right ] \right\}\\     & \qquad \leq { \mathcal{e}}\left\{{\mathbbm{e}}\left[\left \vert\theta(\bar{x}^i_s,\bar{x}^i_{s-\tau_{ij}})-{\mathbbm{e}}_{z,\tau}[\theta(\bar{x}^i_s , z_{s-\tau}^{\alpha})]\right \vert^2\right]\right\}^{1/2 } { \mathcal{e}}\left\{{\mathbbm{e}}\left[\left \vert\theta(\bar{x}^i_s,\bar{x}^k_{s-\tau_{ij}})-{\mathbbm{e}}_{z,\tau}[\theta(\bar{x}^i_s , z_{s-\tau}^{\alpha})]\right \vert^2\right]\right\}^{1/2}\\         & \qquad \leq 4 \tilde{k }( 1+c_2(s ) )   \end{aligned}\ ] ]  we note .we have : \right\ } \leq t^2p c \sum_{\gamma=1}^p\frac{3n_{\gamma}-2}{n_{\gamma}^2 } \leq 3\,t^2\,p^2\ , c \frac{1}{\min_{\gamma}(n_{\gamma})}\ ] ] and \right\ } \leq 12\,t\,p^2\ , c \frac{1}{\min_{\gamma}(n_{\gamma})}\ ] ]  assembling all the estimates , we obtain \ } & \leq 8\ , \bigg\{(k^2 + 2 l^2 p^2 )\,(t+4 ) \int_0^t\max_{j=1\cdots n}{\mathbbm{e}}\big[\sup_{-\tau \lequ\leq s}\vert x^j_u - \bar{x}^j_u\vert^2\big]\ , du \\     & \qquad + 3(t+4)t\frac{c\,p^2}{\min_{\gamma}(n_{\gamma})}\big\ }               \end{aligned}\ ] ] let us denote and .since and are equal on  ] towards solution of the mean - field equations . moreover , if and are globally lipschitz - continuous , we have for any and any : - \bar{x}^i_s\vert^2 \big]\right\ } < \frac{c(t)}{\min_{\gamma}(n_{\gamma})}\ ] ] where is a constant only depending on the parameters of the system and the time horizon .the proof proceeds as the one of theorem [ thm : propagationchaos ] by thoroughly controlling the distance between  ] .this distance is decomposed into the sum of eight terms analogous to the quenched translation invariant case , that are controlled exactly in the same manner as done in the proof of theorem [ thm : propagationchaos ] , except two terms and analogous to and . regarding these terms , we have : \ } & = { \mathcal{e}}\big\{{\mathbbm{e}}[\sup_{0\leq s\leq t } \vert \int_0^s \sum_{\gamma=1}^p  \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } { \mathcal{e}}_i\big\{(b_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})\big\}-\int_{-\tau}^0{\mathbbm{e}}_z[b_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma } ) ] ) \,d\eta_{\alpha\gamma}(u)\ , ds\vert^2]\big\}\\   \label{eq : ethtprimecontrol } & \leq t p  \sum_{\gamma=1}^p \int_0^t { \mathcal{e}}\big\ { { \mathbbm{e}}[\bigg \vert \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } { \mathcal{e}}_i\big\{b_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})\big\}- \int_{-\tau}^0{\mathbbm{e}}_z[b_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma})\ , d\eta_{\alpha\gamma}(u ) ] \bigg\vert^2 ] \ , ds\big\}\end{aligned}\ ] ] and using burkholder - davis - gundy \big\ } \leq 4 p  \sum_{\gamma=1}^p \int_0^t  { \mathcal{e}}\big\{{\mathbbm{e}}[\bigg \vert \frac 1 { n_{\gamma } } \sum_{p(j)=\gamma } { \mathcal{e}}_i\big\{\beta_{\alpha\gamma}(\bar{x}^i_s,\bar{x}^j_{s-\tau_{ij}})\big\}- \int_{-\tau}^0{\mathbbm{e}}_z[\beta_{\alpha\gamma}(\bar{x}^i_s , z_{s+u}^{\gamma } ) \ ,d\eta_{\alpha\gamma}(u ) ] \bigg\vert^2]\big\}\ , ds.\ ] ] in that case again , the expectation of the process under and the delays is  ] for .  by the variation of constant formula we have : \eta_{\alpha\gamma}(u ) \big)\,ds \\+ \int_0^t e^{-(t - s)/\theta_{\alpha } } \lambda_{\alpha}\,dw_s^{\alpha }     +  \sum_{\gamma=1}^p\sigma_{\alpha\gamma}\int_0^t e^{-(t - s)/\theta_{\alpha } } \int_{-\tau}^0 { \mathbbm{e}}_{\bar{x}}[s(\bar{x}^{\gamma}_{s+u})]d\eta_{\alpha\gamma}(u ) \,db^{\alpha\gamma}_s .\end{gathered}\ ] ] the initial condition term vanishes exponentially fast . the other terms include deterministic terms and stochastic integrals with respect to brownian motions of deterministic functions , hence are gaussian .all but the initial condition term are hence gaussian and the solutions are exponentially fast attracted towards gaussian solutions . taking the mean and covariance function of this process readily yields equations .  in the firing - rate case, we hence have an important reduction of complexity .we now exploit this simpler form of the mean - field equations to analyze the dynamics of the network using the theory of delayed differential equations ( see e.g.  ) and compare it to numerical simulations .we particularly focus on the importance of the noise in the stochastic weights and on the distribution of delays .   in order to analyze the impact of the distribution of delays on the dynamics of the network , we simplify the model considering a one - population network ( , we hence drop the population indexes since there is no ambiguity here ) .we further assume that , or more precisely consider centered sigmoid functions ( ) and no input ( ) . in order to further simplify the system , we consider where . in that case , integration by parts yields : in that simplified case ,it is easy to see that a stationary solution is given by . due to the quadratic form of the variance equation , the linearized equation onthe variance is trivial .therefore , the level of noise in the connections do not come into play in that particular case , and the stability of the fixed point only depends on the characteristic equation related to the main , which is given by the dispersion relationship : the solutions of this equations are the characteristic exponents of the system , and relate to the stability of the fixed point considered : if all characteristic exponents have negative real part , the equilibrium is asymptotically exponentially stable , and there exists a characteristic exponent with strictly positive real part , the equilibrium is unstable .changes in the sign of the real part of the characteristic exponent constitute bifurcations of the system .let us start by analyzing the presence of _ pitchfork _ bifurcations that correspond to real characteristic exponents crossing the imaginary axis .such bifurcations hence occur when the following relationship is satisfied : and hence this bifurcation occurs independently of the choice of the delay .turing - hopf bifurcations occur when the system has a pair of complex conjugate characteristic exponents with non - zero imaginary part crossing the imaginary axis .such bifurcations hence occur when there exists such that , i.e. : the existence of such bifurcations exist depending on the type of delay distribution .it is important to note at this point that there is no such bifurcation when delays are equal to zero .   assuming that the neuronal populations are highly concentrated so that delays can be considered constant ( i.e. ) .the dispersion relationship can be easily solve and one finds that hopf bifurcations exist only for i.e. for strong enough connectivity and small enough noise . under that assumptionis then easy to show that turing - hopf bifurcations arise when the parameters satisfy the relationship , when : with .these result into oscillations of the solutions at a pulsation equal by .details of the calculations are left to the reader and follow the lines of the proof used for localized delays of the next section .note that the value of the delay corresponding to the hopf bifurcation is a function of .in particular , no hopf bifurcation is to be found for .the curve of hopf bifurcations obtained in the plane and simulations the moment equations are displayed in figure fig .[ fig : deltadelay ] .these simulations illustrate the fact that , depending on the noise and the delay chosen , the law of the mean - field equations has a mean that is either periodic or constant , yielding periodic or stationary laws of the mean - field equations . since all neurons have the same probability distribution , in the periodic case all neurons will oscillate in phase .these results are valid for numerical simulations of the stochastic network as displayed in figure fig .[ fig : networkdeltadelay ] .we now consider instead of dirac delta distributions of the delays that the neurons in each population have a dispersion of order around a typical value .one of the simplest distribution modeling this kind of dispersion is the uniform distribution } dt/\delta ] and that ( without loss of generality ) , it is easy to show that the distribution of the distance between neuron at location and other neurons has the density : }+ \mathbbm{1}_{[0,r_i]}\right)\frac{dr}{a}\ ] ] and the averaged law the density :   in the case where we consider a periodic neural area ( i.e. identifying and ) , the distribution of distances is clearly the uniform distribution on  ] , the mean distance between neurons is equal to and the standard deviation to .increasing hence has the effect of linearly increasing the averaged effective delay , and quadratically the dispersion .for small values of , the effective delay created by the dispersion of the neuron increases faster than the dispersion , and therefore a smaller constant delay is necessary to trigger oscillations .as is further increased , the dispersion grows faster than the mean effective delay , and in that case the dispersion implies that larger delays are necessary to trigger oscillations , as seen in the uniformly distributed delays case .+  fixing all parameters and changing the size of the interval in which the neurons lie can hence induces transition from stationary to periodic and back to stationary .neurons are hence perfectly synchronized in a specific region of the neural field spatial extension , and otherwise asynchronous .in this article , we analyzed the limits and dynamics of large - scale neuronal network models in the presence of noise .we particularly focused on the presence of heterogeneous delays in the interactions between neurons , an ubiquitous phenomenon in neuronal network activity that is understood to have important effects on the brain function .we considered two different kinds of systems : ( i ) translation invariant systems where the delays have the same distributions whatever the neurons considered , and ( ii ) non translation invariant case where delays distribution depend on the particular neuron considered . in both cases , we derived the mean - field equations associated with such randomly delayed networks for general models , and showed that the propagation of chaos occurred , in a quenched sense for translation invariant systems ( i.e. for almost all realization of the delays ) , or after averaging across possible delay distributions in the non translation invariant case .  in both cases ,the limit equations are identical and given by a system of delayed mckean - vlasov equations with distributed delays .the equations obtained are not classical stochastic equations : they involve a term depending on the law of the solution at previous times ( due to the presence of distributed delays ) .we showed that these equations were well - posed , and admit a unique solution .nonetheless , this solution remains relatively hard to characterize at that level of generality . in order to characterize the dynamics of the network and in particular the importance of heterogeneity in the interconnection delays , we instantiated a popular model in neuroscience , the firing - rate model . in the mean - field limit ,the solutions converge towards gaussian processes , whose mean and variance satisfy a closed set of differential equations with distributed delays .this property allowed , using the theory of bifurcations of delayed differential equations ( see e.g.  ) , to analyze in detail the role of delays in the macroscopic dynamics of neuronal networks .we showed that depending on the averaged delays , the network can either present a stationary or a synchronized periodic behavior .such transitions as a function of the amplitude of delays were already observed in heuristic models including constant delays .but beyond this fact , we showed that averaged delay is not enough to predict the behavior of the system , and a knowledge of the full delay distribution is necessary .we illustrated this fact exhibiting the bifurcation diagram of the hopf bifurcations as a function of the variance of uniformly distributed delays , and probably more interestingly from the biological viewpoint the dependence of the network behavior as a function of the delays arising from the distance between the cells .we showed that the macroscopic behavior arising from such networks depend on the size of the neural area considered , that parametrizes the delay distributions in the network .this was done analytically in a simple one dimensional case , and observed that the size of the cortical column considered has non - trivial effects on the macroscopic dynamics of the neural area .this observation is very interesting from the neuroscience viewpoint .indeed , it has been observed that across different species , the cortex either organizes into columns or shows a dispersed distribution .this is for instance the case of the primary visual areas : mice and rats for instance present no specific organization of the neurons responsible for orientation selectivity ( a so called salt - and - pepper structure ) , whereas the cat and primate present structured orientation selectivity columns ( see e.g.  ) .the reason of that difference is still poorly understood , and in that article , the authors suggest that the lack of orientation maps in rodent is related to the small size of v1 in these species .they argue that making a synaptic connection is easier locally .this possibility of creating synaptic connections is not a definitive answer to that question since the authors report a few counter - examples such as the ferret or the tree shrew . including the delay distribution in this picture and how the topology and size of the cortex influence the synchronization property at the level of cortical columns could allow going deeper into the analysis of the reasons of the functional reliability of orientation selectivity across different species with sensitively different cortex size .this work brings another refinement into the theoretical implication of delays in the dynamics of the network .the equations derived here provide a model of large - scale neuronal network with heterogeneous delay , and a preliminary study of the dynamics of such networks was initiated .enriching this model by considering several populations is a straightforward extension and the present manuscript , and analyzing such models would allow going deeper into the analysis of neuronal networks and on the effect of heterogeneous delays in relationship with specific cortical functions . eventually , the cortex is far from being the only system presenting heterogeneous distributions of interconnection delays .most networks , including communication networks , internet , social networks and physical networks such as spin glass present delays in the communications , that depend on the specific distance between nodes .this study can hence be applied to these other cases , and applications of that framework to the tcp protocol is currently under investigation .

Abstract: realistic networks display heterogeneous transmission delays . we analyze here the limits of large stochastic multi - populations networks with stochastic coupling and random interconnection delays . we show that depending on the nature of the delays distributions , a quenched or averaged propagation of chaos takes place in these networks , and that the network equations converge towards a delayed mckean - vlasov equation with distributed delays . our approach is mostly fitted to neuroscience applications . we instantiate in particular a classical neuronal model , the wilson and cowan system , and show that the obtained limit equations have gaussian solutions whose mean and standard deviation satisfy a closed set of coupled delay differential equations in which the distribution of delays and the noise levels appear as parameters . this allows to uncover precisely the effects of noise , delays and coupling on the dynamics of such heterogeneous networks , in particular their role in the emergence of synchronized oscillations . we show in several examples that not only the averaged delay , but also the dispersion , govern the dynamics of such networks .


the era of ` big data ' has transformed the data analysis landscape over the last few years .transformative technologies , such as on demand cloud computing infrastructure , and programming paradigms such as hadoop and hive facilitate applications to perform large - scale data analysis .however , we argue that there are three fundamental challenges , which need to be addressed in order to allow domain experts without a computer science background to benefit from such technologies .while big data technologies such as hadoop provide the tools to facilitate the analysis of large - scale data , there is significant research that still needs to be performed in order to make these tools accessible to the wider community . in the current state - of - the - art, there is a gap between big data technologies and the end user ; an in - depth knowledge of the technologies for setting up and maintaining hardware along with programming skills for accessing application programming interfaces ( api ) are required for performing any meaningful analysis .for example , the use of hadoop or hive still requires detailed knowledge of parallel programming using the mapreduce paradigm .the research reported in this paper aims toward bridging this gap by addressing how big data technologies can be made accessible to expert practitioners who are non - computer scientists , such as social scientists , for performing big data analytics .from discussions with social scientists , we understand that a typical working pattern with a data set or variety of data sets involves the exploration of data by performing a quick analysis first in order to infer whether the data is relevant or of interest before performing more complex analysis .for example , given a large dataset related to newspaper articles from 18851975 , how can a social scientist quickly explore the dataset to find which newspapers published articles related to an event in 1925 ?such exploration needs to be performed with minimal effort without having to undertake large programming tasks . in this paper , we present how data exploration can be performed with minimal effort .research efforts have been made towards the development of tools that can facilitate big data analytics .for example , the ibm infosphere and tableau software .these tools provide fully fledged functionalities and are suitable for those who receive training for using them proficiently .such tools are not suited and required for exploring data and performing simple analytical operations on data . in this research, we aim to develop a lightweight tool that provides essential functionalities to explore data and perform basic analytical operations .  in this paper ,we address the above three challenges related to the accessibility of big data technologies for non - computer scientists , the exploration of large data sets with minimal efforts and the availability of lightweight frameworks for ad hoc analytics by developing a web - based framework , we refer to as bigexcel , for exploring big data .the three tier framework is situated between a user and the underlying infrastructure to facilitate the management of user interactions with large data sets , the construction of queries to explore the data set and the management of the infrastructure .the feasibility of bigexcel is validated on two case studies using yahoo sandbox datasets .the contributions of the research presented in this paper are ( i ) the development of a lightweight web - based framework for exploring and performing analytics in an ad hoc manner on big data , ( ii ) facilitating simplified access to big data technologies for social scientists by developing an interactive layer between the user and the underlying hardware infrastructure , ( iii ) the abstraction of technicalities related to the setup , maintenance and execution of big data and the queries associated with large datasets on big data technologies such as hive and hadoop distributed file system ( hdfs ) , and ( iv ) the demonstration of the framework on two datasets provided by yahoo sandbox .  the remainder of this paper is organised as follows .section [ framework ] presents the web - based framework developed in this research to facilitate the exploration and visualisation of big data .section [ casestudies ] presents two case studies which are used to validate the feasibility of the framework and demonstrate how typical datasets can be explored .the first case study is based on quantitative prediction of trending technologies and the second case study is based on a qualitative analysis of n - grams from news oriented websites correspond with real events .section [ conclusions ] concludes this paper by presenting a short discussion on future work .bigexcel : a web - based framework for exploring big data in real - time is illustrated in figure [ figure0 ] .bigexcel consists of three layers ; the first layer is responsible for handling user interactions , the second layer manages queries that express how data needs to be handled , and the third layer is related to the management of underlying infrastructure .  [ !this layer handles all user interactions in the bigexcel framework , such as requesting a specific analytical operation on data and the display of the operation in a meaningful way .this layer consists of one module , referred to as the ` data browser ` which is built using richfaces , a framework for developing uis and supported by jquery for data validation and navigation and javaserver faces ( jsf ) that facilitates data communication with the next layer .the browser enables a user to submit a request for managing data ( for example , loading or viewing data ) and for performing analytics on this data . using restful ( representational state transfer ) web services the application programming interface ( api ) provided by the next layer is used by the browser .this layer receives data from the user interaction layer through jsf and is responsible for constructing queries , attaching relevant modules to the queries for analytics , and handling the responses from the next layer .five modules are used in this layer , and they are as follows :   user requests obtained from the browser are converted to queries that can executed on hive .two types of queries are constructed ; the first are data management queries which are handled by the ` data manager ` and the second is analytics queries handled by the ` analytics processor ` .the data management queries are handled by this module .they are related to transferring data to the hadoop distributed file system ( hdfs ) and loading data from hdfs to hive .three types of analytics queries are handled by this module : ( a ) the set of queries which make use of built - in modules , such as sum , average and standard deviation , that apply to rows or columns of structured big data and execute on hive , ( b ) the set of queries which can incorporate user defined business logic in the form of modules that can be execute on hive , and ( c ) the set of queries which incorporates user defined logic in the form of mapreduce modules that can be executed on hive or hadoop.   it may not be always possible to derive the business logic for an analytical operation using the combination of built - in modules .user defined modules are therefore required to incorporate business logic into queries . the module repository , which is a database of user defined modules that are incorporated into queries by the ` analytics processor ` facilitates this functionality .this module prunes data for presenting to the user in the data browser . in the case of big data ,the dynamic pagination ( or lazy loading ) technique is used for facilitating real - time viewing of data on the browser . in this technique, chunks of data are displayed on - demand .the ` response processor ` also employs chart apis ( for example , google charts ) for presenting data on the browser .this layer handles the communication between the query management layer and the underlying infrastructure .the key module that facilitates this is the ` connection manager ` which follows a multi - threaded model for handling concurrent users . the amazon web services ( aws ) sdk used to connect to the cloud .the bigexcel framework presented in the previous section was tested using two case studies employing data sets from yahoo sandbox .the first case study uses marketing data and the second case study uses language data .the case studies are used to demonstrate the feasibility of the framework .the source code of the framework along with a video illustrating the functionality of the framework is available at http://bigdata.cs.st - andrews.ac.uk / projects / bigexcel - exploring - big - data - for - social - sciences/.   in this case study , the data set employed is the ` yahoo ! buzz game transactions with buzz scores ' , version 1.0 , which is nearly 90 mb ( uncompressed data ) .the data contains information on various trending technologies and the volume of searches associated with the technology , which is indicated as a buzz score , for a period from april 1 , 2005 to july 31 , 2005 .a higher buzz score for a technology indicates a larger volume of search for that technology . in the dataset, technology is grouped into items , such as browsers , tvs , e - books , video games , social networks , photo organisers , and gps / map related .each technology consists of products , for example , browsers contain sub - items such as internet explorer , safari , mozilla , and firefox .the data is relevant to a market in which stocks in various technologies are bought and sold based on current buzz scores .hence , it is important to understand the fluctuations of the buzz score for making purchasing and selling decisions .we use this data in the bigexcel framework for ( i ) predicting hourly , daily and weekly patterns of buzz scores and ( ii ) for comparing the prediction against the actual buzz scores to evaluate whether prediction is within a reasonable order of magnitude accuracy .the dataset is obtained in text ( .txt ) format .the framework converts the .txt format to comma separated file ( .csv ) format and the .csv file is then loaded into the hive warehouse .a user can then explore the data in bigexcel by plugging modules which are stored in the ` module repository ` .an hourly analysis of the buzz scores of video games as shown in figure [ figure1 ] for two working weeks , the first from 23/05/2005 to 27/05/2008 and the second from 06/06/2005 to 10/06/2005 was performed .such an analysis can capture the hourly trends which may be useful to make observations of obvious patterns that might emerge .the hourly buzz scores are calculated by aggregating all the buzz scores related to video games in that hour . in figure[ figure1 ] , for example , the buzz score shown for 09:00:00 for a given day is the average of all the buzz scores from 09:00:00 to 09:59:59 for that day .  to perform the above, the user provides as input the table name ( ` yahoo_buzz_scores ` ) , column names ( ` date ` , ` time ` and ` buzz_score ` ) , product type ( ` e - books ` and ` buzz_score ` ) and the name of the module required for analysis ( ` hourly_analysis ` ) through the data browser .the ` query constructor ` then assembles the following query :  .... select transform(date , time , buzz_score )  using ' hourly_analysis '  from yahoo_buzz_scores  where product='ebooks '  and date > = 2005 - 05 - 23  and date < = 2005 - 05 - 27 ; ....  the ` analytics processor ` embeds the ` hourly_analysis ` module from the ` module repository ` into the query and the query is executed on hive .the ` hourly_analysis ` module includes the logic for averaging the buzz scores in a one hour period to generate an aggregate buzz score for the hour .the output is parsed by the ` response processor ` to display the information in a meaningful way , for example , to generate data shown in figure [ figure1 ] .immediate observations from figure [ figure1 ] include the cluster of buzz scores for the week in both may and june at 1 am , 10 am , 11 am , 3 pm , 6 pm and 11 pm . a consistent occurrence of clusters across different weeks can be used as a pointer to predict with some certainty what the buzz scores may be for those time periods ; this information will be useful while trading stocks based on buzz scores .another observation includes the low buzz scores for both weeks on wednesdays at 12 noon and 1 pm .figure [ figure2 ] shows the actual buzz scores of e - books , online music , social network , photo organisers and video games for each day in the period from april 1 , 2005 to july 31 , 2005 .a user defined module that is stored in the ` module repository ` of the bigexcel framework is created to facilitate the computation of the buzz scores for each day .the buzz scores of one hour intervals of a day are averaged to aggregate the buzz score for a day .  [ !ht ]  to perform the above , the user provides as input the table name ( ` yahoo_buzz_scores ` ) , column names ( ` date ` , ` buzz_score ` , products and the name of the module that for analysis ( ` hourly_analysis ` ) through the data browser .the ` query constructor ` then assembles the following query :  .... select transform(date , buzz_score ) using ' daily_analysis '  from yahoo_buzz_scores where product in ( ' onlnmusic','ebooks ' ,    ' vgame ' , ' socnets ' , ' photo ' ) and date>='2005 - 04 - 01 '  and date < = ' 2005 - 07 - 26 ' ....the ` daily_analysis ` module is embedded from the ` module repository ` into the query by the ` analytics processor ` .the module includes the logic for averaging the hourly buzz scores in a given day to generate an aggregate buzz score for the day .the output is parsed by the ` response processor ` to display the information in a meaningful way , for example , to generate data that represents figure [ figure2 ] .we incorporate two techniques for predicting the buzz score of a given day stored in the ` daily_prediction ` module .in the first technique , the buzz scores of the preceding days or of the same week day in a sample of weeks are aggregated by averaging to predict the buzz score of a given day .for example , if the preceding days are taken into account , then to predict the buzz score of video games on july 23 , 2005 , the buzz scores from july 8 to july 22 are employed if 14 preceding days are considered , or the buzz scores from june 24 to july 22 are employed if 28 preceding days are used .if a sample of weeks are used , then to predict the buzz score of e - books on saturday , july 23 , 2005 , the buzz scores on saturdays , june 18 , june 25 , july 9 and july 16 are used if a four week sample is taken into account , or the buzz scores on saturdays , may 14 , june 4 , june 18 , june 25 , july 2 , july 9 and july 16 are used if an eight week sample is taken into account .we refer to this technique as ` extrapolation - based prediction ( ep ) ' with preceding days or a sample of weeks .  in the second technique, the buzz score of the preceding days or of the same week day in a sample of weeks are aggregated by linear regression to predict the buzz score of a given day .this technique is different from the first and second techniques in that linear regression is applied on the buzz scores for predicting rather than by averaging .the data points used for the regression remains the same as in the former techniques .we refer to this technique as ` regression - based prediction ( rp ) ' with preceding days or a sample of weeks .the user provides as input the product ( e - books ) , the date for which he needs to predict the buzz score ( saturday , july 23 , 2005 ) and the value ( if 14 days are selected , then july 8 to july 22 are used , or if a 4 week sample is selected , then four preceding saturdays are used ) . for example , when the user selects 14 days , the ` query constructor ` takes this input and assembles the query as follows :  .... select transform(date , buzz_score ) using ' daily_prediction '  from yahoo_buzz_scores where product in ( ' ebooks ' ) and date>='2005 - 07 - 08 '  and date < = ' 2005 - 07 - 22 ' ....  figure [ figure3 ] shows the graphs plotted when the ep and rp techniques are used for predicting the buzz score of e - books , online music , social networks , photo organisers and video games on july 23 , 2005 .observations made from the graph are ( a ) consider video game products , for which the actual buzz score is 9.74308523 .the most accurate prediction using ep techniques is when an eight week sample is taken into account with an error percentage of less than 0.2% .the most accurate prediction using rp techniques is when a four week sample is considered with less than 0.3% error .0.49   0.49   table [ table1 ] summarises the error percentage of the predicting techniques .given the sensitivity of the buzz scores upto a 2% error is acceptable .  [ cols= " <, < , < , < , < , < , < , < , < , < , < " , ]   the ` yahoo ! n - grams , ' version 2.0 , is used in this case study. this dataset is based on data gathered from over 12,000 news related sites , 14.6 million documents , 126 million unique sentences , 3.4 billion running words between february 2006 and december 2006 .the data set contains -grams ( continuous words , where to ) and their frequency .the uncompressed data is 40 gb and the data related to 5-grams is 12 gb .we use this data in the bigexcel framework for searching the frequencies of certain key words related to important events and qualitatively interpreting the results .we use the 5-gram dataset for this case study and three events that occurred in 2006 .the first event is the pandemic threat of h5n1 virus , otherwise referred to as bird flu .this event evolved throughout the year with the first sighting in early january in turkey resulting in a number of deaths , followed by similar outbreaks in a numerous african , asian and european countries leading to large number of deaths throughout the year .given the widespread nature of the event and its severity we expect that news websites may have given significant importance to this event .the second event considered is the 2006 world cup football tournament which happened in june and july .italy won the tournament by beating france .over the course of the tournament nearly 26.29 billion non - unique users watched matches and obtained the largest number of television views .we expect that this event would have received significant coverage on news websites .the third event is the purchase of youtube by google on october 9 , 2006 .google beat a number of competitors including microsoft and yahoo in its $ 1.65 billion purchase .we expect this news item would have been featured on news sites .in the dataset there are 29,570,136 5-grams and we explore the number of 5-grams that were related to the three events . for example , consider the bird flu event .we explore the dataset to determine whether there are 5-grams related to the event by searching for relevant key words .figure [ figure8a ] and figure [ figure8a1 ] shows the frequency of the 5-grams related to the bird flu .the 2-gram , bird flu , is indicated while all the other tokens that surround the 2-gram are denoted using .for example , `   ' could be `   ' or  . there are 148,934 5-grams directly related to the event which is 0.5037% of the entire corpus .  to perform the above, the user provides as input the table name ( ` yahoo_n - grams ` ) , column names ( ` n - gram ` , ` frequency ` and the name of the module that for analysis ( ` event_analysis ` ) through the data browser .the ` query constructor ` then assembles the following query :  .... select transform(n - gram , frequency ) using ' ngram_analysis '  as distinct_n - gram , total_frequency from yahoo_n - grams ....  the ` ngram_analysis ` module is embedded from the ` module repository ` into the query by the ` analytics processor ` .the module includes the logic for searching the corpus for tokens related to the bird blu and then organising the data into various distinct ngrams and their frequencies .for example , the distinct n - grams related to bird flu are   ,   ,   , and   .the output is parsed by the ` response processor ` to display the information in a meaningful way , for example , to generate data that is represented in figure [ figure8a ] and figure [ figure8a1 ] .figure [ figure8b ] is the frequency of the 5-grams related to the world cup football event .the 5-grams related to the event are based on phrases for example , ` italy beat france ' and italy wins. there are 388 5-grams directly related to the event which is approximately 0.0013% of all the 5-grams contained in the corpus .figure [ figure8c ] is the frequency of the 5-grams related to the purchase of youtube by google .there are 56 5-grams related to the event .there are about four hundred times more 5-grams directly related to the bird flu , an event which evolved during the whole year , than the finals of the world cup football match .similarly , the world cup final has seven times more coverage than the takeover of youtube by google . using this analysis it may not be possible to make firm conclusions but it would seem that the bird flu generated nearly 30 times more news each week compared to the weeks when the world cup football was going on .  0.48   0.48   +  0.48   0.48this research identified three big data challenges , namely ( i ) the accessibility of big data technologies for non - computer scientists , ( ii ) the ad hoc exploration of large data sets with minimal efforts and ( iii ) the availability of lightweight frameworks for ad hoc analytics on the web .we developed the bigexcel framework to address the above challenges .the framework is three tiered and web - based that facilitates the management of user interactions with large data sets , the construction of queries to explore the data set and the management of the infrastructure .the feasibility of bigexcel was demonstrated using two yahoo sandbox datasets in which quantitative and qualitative analysis was performed .a video demonstration of the framework with the source code is available at http://bigdata.cs.st - andrews.ac.uk / projects / bigexcel - exploring - big - data - for - social - sciences/.  in the future , we aim to extend bigexcel to accommodate the analysis of unstructured data for fully exploiting the benefits of big data technologies .additionally , efforts need to be directed towards automating data management by taking minimum input from the user .this will make the framework more accessible to non - computer science communities .another direction for enhancing bigexcel is developing generic algorithms for the ` module repository ` and here machine learning approaches will need to be incorporated for dealing with numerous types of data and their analyses .further , the repository can benefit from the development and integration of analytical libraries that support mathematical and statistical functions .this research was pursued through an amazon web services education research grant and was facilitated by the availability of the yahoo sandbox datasets .the first author was the recipient of an erasmus mundus scholarship .d. boyd and k. crawford , `` critical questions for big data : provocations for a cultural , technological , and scholarly phenomenon , '' information , communication & society , vol .15 , issue 5 , 2012 , pp .662 - 679 .a. thusoo , j. s. sarma , n. jain , z. shao , p. chakka , s. anthony , h. liu , p. wyckoff and r. murthy , `` hive - a warehousing solution over a map - reduce framework , '' journal proceedings of the vldb endowment , vol .2 , issue 2 , 2009 , pp .1626 - 1629 .

Abstract: this paper argues that there are three fundamental challenges that need to be overcome in order to foster the adoption of big data technologies in non - computer science related disciplines : addressing issues of accessibility of such technologies for non - computer scientists , supporting the ad hoc exploration of large data sets with minimal effort and the availability of lightweight web - based frameworks for quick and easy analytics . in this paper , we address the above three challenges through the development of ` bigexcel ' , a three tier web - based framework for exploring big data to facilitate the management of user interactions with large data sets , the construction of queries to explore the data set and the management of the infrastructure . the feasibility of bigexcel is demonstrated through two yahoo sandbox datasets . the first dataset is the yahoo buzz score data set we use for quantitatively predicting trending technologies and the second is the yahoo n - gram corpus we use for qualitatively inferring the coverage of important events . a demonstration of the bigexcel framework and source code is available at http://bigdata.cs.st - andrews.ac.uk / projects / bigexcel - exploring - big - data - for - social - sciences/.  big data ; real - time processing ; hive ; hadoop ; web - based querying


optimization with pde constraints is nowadays a well - studied topic motivated by its relevance in industrial applications .we are interested in the numerical approximation of such optimization problems in an efficient and reliable way using surrogate models obtained with pod - mor .the surrogate models are built upon _snapshots _ of the system to provide information about the underlying problem .this stage is usually called the offline stage . for the snapshot pod approachwe refer the reader to .+ several works focus their attention on the choice of the snapshots , in order to approximate either dynamical systems or optimal control problems by suitable surrogate models . in , it is proposed to optimize the choice of the time instances such that the error between pod and the trajectory of the dynamical system is minimized .a recent approach proposes to choose the snapshots by an a - posteriori error estimator in order to equidistribute the state error on the time grid related to the snapshot locations ( see ) .we also mention an adaptive method , proposed in , where the aim is to reduce expensive offline costs selecting the snapshots according to an adaptive time - stepping algorithm using time error - control . for further referenceswe refer the interested reader to . + in optimal control problemsthe reduced model is usually built upon a forecast on the control .this approach does not guarantee a proper construction of the surrogate model since we do not know how far away the optimal solution is from the reference control .more sophisticated approaches select snapshots by solving an optimization problem in order to improve the selection of the snapshots according to the desired controlled dynamics . for this purpose optimality system for pod ( os - pod )is introduced in . in os - pod ,the computation of the basis functions is performed by means of the solution of an enlarged optimal control problem which involves the full problem , the reduced equation and the eigenvalue problem for the pod modes .+ the reduction of optimal control problems with particular focus on adaptive adjustments of the surrogate models can be found in .we should also mention another adaptive method for feedback control problems by means of the hamilton - jacobi - bellman equation , introduced in .+ recently , an a - posteriori error estimator was introduced in for optimal control problems . in theseworks the error between the unknown optimal and the computed pod suboptimal control is estimated for linear and nonlinear problems , and it is shown that increasing the number of basis functions leads to the desired convergence .os - pod and a - posteriori error estimation is combined in .+ all these works have in common that they compute basis functions for optimal control problems . in our paperwe address the question of an efficient and suitable selection of snapshot locations by means of an a - posteriori error control approach proposed in .we rewrite the optimality conditions as a second order in time and fourth order in space elliptic equation for the adjoint variable and we generalize this approach to control constraints .in particular , a time adaptive concept is used to build the snapshot grid which should be used to construct the pod surrogate model for the approximate solution of the optimal control problem . here the novelty for the reduced control problem is twofold : we directly obtain snapshots related to an approximation of the optimal control and , at the same time , we get information about the time grid .  we have proposed a similar approach based on a reformulation of the optimality system with respect to the state variable in .now , we focus our approach on the adjoint variable and generalize the idea presented in to time dependent control intensities with control shape functions including control constraints .furthermore , we certify our approach by means of several error bounds for the state , adjoint state and control variable .+ the outline of this paper is as follows . in section 2we present the optimal control problem together with the optimality conditions . in section 3we recall the main results of .proper orthogonal decomposition and its application to optimal control problems is presented in section 4 .the focus of section 5 lies in investigating our snapshot location strategy .finally , numerical tests are discussed in section 6 and conclusions are driven in section 7 .in this section we describe the optimal control problem .the governing equation is given by a linear parabolic pde : where is an open bounded domain with smooth boundary , ,  ] , and the state is denoted by . as control space we use , where , and define the control operator as , , where denote specified control actions . thus is linear and bounded . for the control variablewe require \ } \subset l^\infty(0,t;\mathbb{r}^m)\ ] ] with almost everywhere in .it is well - known ( see , for example ) that for a given initial condition and a forcing term the equation admits a unique solution , where if , higher regularity results can be derived according to .we also note that the unconstrained case is related to .the weak formulation of is given by : find with and   the cost functional we want to minimize is given by where is the desired state and the regularization parameter is a real positive constant .the optimal control problem then reads note that is a non - empty , bounded , convex and closed subset of .hence , it is easy to argue that admits a unique solution with associated state , see e.g. .the first order optimality system of the optimal control problem is given by the state equation , together with the adjoint equation and the variational inequality where is the dual operator of .in we have identified with and with , where we use that hilbert spaces are reflexive .the variational inequality is equivalent to the projection formula ,\ ] ] where denotes the orthogonal projection onto .it follows from the reflexivity of the involved spaces that the action of the adjoint operator is given as and   since our domain is smooth , the regularities of the optimal state , the optimal control and the associated adjoint state are limited through the regularities of the initial state , the right hand side , the control and the desired state + the numerical approximation of the optimality system -- with a standard finite element method ( fem ) in the spatial variable leads to a high - dimensional system of ordinary differential equations :    here \rightarrow{\mathbb r}^n ] .we define the time discrete space where the notation stands for the polynomials of first order on the interval .then , we consider the semi - discrete problem : using the arguments of e.g. ( * ? ? ?* th . 1.25 ) one can show that problem admits a unique solution .+ we note that with and we have the galerkin orthogonality thus , for it holds true the following theorem states a temporal residual type a - posteriori error estimate for , which transfers the estimation of ( * ? ? ?* theorem 3.5 ) to the control constrained optimal control problem :  [ thm : apost ] let and denote the solutions to and , respectively .then we obtain where and   _ proof ._ we start the proof showing a consequence of the monotonicity of the projector operator .we find that and hence   for easier notation , we set . + let and let denote the standard lagrange type temporal interpolation of . using the inequality for and from ( * ? ? ?* lemma 2.5 ) , the monotonicity and the galerkin orthogonality , we can estimate :   & \leq &  \left\| \displaystyle\frac{\partial ( p - p_k)}{\partial t } \right\|_{l^2(\omega_t)}^2 + \| \delta ( p - p_k ) \|_{l^2(\omega_t)}^2 \\  & \leq & \left\| \displaystyle\frac{\partial ( p - p_k)}{\partial t }  \right\|_{l^2(\omega_t)}^2 + \| \delta ( p - p_k )\|_{l^2(\omega_t)}^2  +  \displaystyle\int_{\omega_t } ( n(p)-n(p_k))(p - p_k ) \\[2ex ] & = & \displaystyle\int_{\omega_t } \displaystyle\frac{\partial ( p - p_k)}{\partial t } \frac{\partial e^p}{\partial t } + \int_{\omega_t } \delta ( p - p_k ) \delta e^p + \int_{\omega_t}(n(p ) - n(p_k))e^p \\[2ex ]  & = & \displaystyle\int_{\omega_t } \displaystyle\frac{\partial ( p - p_k)}{\partial t } \frac{\partial ( e^p-\pi_k e^p)}{\partial t } + \int_{\omega_t } \delta ( p - p_k ) \delta ( e^p-\pi_k e^p ) + \int_{\omega_t}(n(p ) - n(p_k))(e^p - \pi_k e^p ) \\[2ex ]  & = & \displaystyle\int_{\omega_t } ( -\frac{\partial y_d}{\partial t } + \delta y_d)(e^p - \pi_k e^p ) + \int_{\sigma_t } y_d \nabla(e^p - \pi_k e^p ) \cdot \hat{n } - \int_{\omega_t } \frac{\partial p_k}{\partial t } \frac{\partial ( e^p - \pi_k e^p)}{\partial t}\\[2ex ]  & & - \displaystyle\int_{\omega_t } \delta p_k \delta ( e^p - \pi_k e^p)- \int_{\omega_t}n(p_k)(e^p - \pi_k e^p )  \end{array}\ ] ] integration by parts on each time interval and green s formula lead to  & \leq & \displaystyle\sum_j \int_{i_j } \int_\omega ( - \frac{\partial y_d}{\partial t } + \delta y_d + \frac{\partial^2 p_k}{\partial t^2 } - \delta^2 p_k - n(p_k))(e^p - \pi_k e^p ) + \sum_j \int_{i_j } \int_{\partial \omega } ( y_d - \delta p_k ) \nabla ( e^p - \pi_k e^p ) \cdot \hat{n}.  \end{array}\ ] ] utilizing error estimates of the lagrange interpolation , the trace inequality and young s inequality , we find  & \leq & c_1 \displaystyle\sum_j \deltat_j^2 \int_{i_j } \left\| -  \frac{\partial y_d}{\partial t } + \delta y_d +  \frac{\partial^2 p_k}{\partial t^2 } - \delta^2 p_k + \mathcal{b }  \mathcal{p}_{u_{ad } } \left\{-\frac{1}{\alpha } \mathcal{b}^ * p_k \right\ } \right\|_{l^2(\omega)}^2 \\[2ex ]  & & + c_1 \displaystyle\sum_j \int_{i_j } \| y_d - \delta p_k \|_{l^2(\partial \omega)}^2 .\hspace{9.5 cm } \square  \end{array}\ ] ]  theorem [ thm : apost ] provides a tool to refine the time grid by means of the residual of the system . due to ,the time instances of this grid may be regarded as ideal snapshot locations for pod - mor applied to problem .in this section , we recall the pod method which we use in order to replace the original problem by a surrogate model .the main interest when applying the pod method is to reduce computation times and storage capacity while retaining a satisfying approximation quality .this is possible due to the key fact that pod basis functions ( unlike typical finite element ansatz functions ) contain information about the underlying model , since the pod modes are derived from snapshots of a solution data set .for this reason it is important to use rich snapshot ensembles reflecting the dynamics of the modeled system .usually , we are able to improve the accuracy of a pod suboptimal solution by enlarging the number of utilized pod basis functions or enriching the snapshot ensemble , for instance . the snapshot form of pod proposed by sirovich in works in the continuous version as follows .+ let us suppose that the continuous solution of and of belongs to a real separable hilbert space , where or , equipped with its inner product and associated norm .we set , where , , .note that the initial condition is included in .the aim is to determine a pod basis of rank with , by solving the following constrained minimization problem : where denotes the kronecker symbol , i.e. for and .+ it is well - known ( see ) that a solution to problem ( [ prb : pod ] ) is given by the first eigenvectors corresponding to the largest eigenvalues of the self - adjoint linear operator i.e. , where is defined as follows : moreover , we can quantify the pod approximation error by the neglected eigenvalues ( more details in ) as follows :   let us assume that we have computed pod basis functions .then , we define the pod galerkin ansatz of order for the state as :   where and the unknown coefficients are denoted by . if we plug this ansatz into the weak formulation of the state equation and use as the test space , we get the following reduced order model for of low dimension :   \text { a.e.},\\[2ex ] \displaystyle\int_\omega y^\ell(0 ) \psi dx & = & \displaystyle\int_\omega y_0 \psi dx\\[2ex ] \end{array}\ ] ]  choosing for and utilizing , we infer from that the coefficients satisfy , \quad m^\ell w(0 ) = y_0^\ell,\ ] ] where , , and .note that is the identity matrix , if we choose as inner product .the reduced order model surrogate ( rom ) for the optimal control problem is given by where is the reduced cost functional , i.e. .we recall that the discretization of the optimal solution to is determined by the relation between the adjoint state and control and refer to for more details about the variational discretization concept . + in order to solve the reduced optimal control problem , we consider the well - known first order optimality condition given by the variational inequality which is sufficient since the underlying problem is convex .+ the first order optimality conditions of also deliver that the adjoint pod scheme for the approximation of is given by : find with satisfying section 4 , the pod method in the continuous framework is recalled , where the pod basis functions are computed in such a way that the error between the trajectories of and of and its pod galerkin approximation is minimized in . in practice, we do not have the whole solution trajectories } , 1 \leq k \leq 4 ] with quadrature weights we later choose the weights for the trapezoidal rule , compare . in the present work ,we neglect the error introduced by quadrature weights .the minimization problem related to then becomes and obviously constitutes a strong dependence of the pod basis functions on the chosen snapshot locations .the related snapshots shall have the property to capture the main features of the dynamics of the truth solution as good as possible . hereit is important to select suitable time instances at which characteristic dynamical properties of the optimal state are located .a natural question is :  _ how to pick time instances that represent good locations for snapshots in pod - mor for ? _+  moreover , we face some difficulties since the reduction of optimal control problems is usually initialized with snapshots computed from a given input control . this problem is usually addressed in the offline stage for pod , which is the phase needed for snapshot generation , pod basis computation and building the reduced order model .mostly , we do not have any information about the optimal control , such that in pod - mor the input control is often chosen as .this circumstance raises the question about the quality of the pod basis and the quality of the pod suboptimal solution .the a - posteriori error estimator in section 3 motivates a suitable location of time instances for the pod adjoint state and at the same time we get an approximation of the optimal control which can be used as an input control in order to generate the snapshots .the use , in the offline - stage , of a time adaptive mesh refinement process allows to overcome the choice of an input control and the choice of the snapshot locations by solving equation .then , we take advantage of the a - posteriori error estimation presented in theorem [ thm : apost ] .equation provides the optimal adjoint state associated with , which does not require the explicit knowledge of a control input .we note that the ellipticity of equation play a crucial role in this approach .the same approach would not work , if one solves the optimality conditions directly .the numerical approximation of provides important information about the control input .in fact , thanks to the variational inequality we are first able to build an approximate control and finally compute the associated state . in this wayour snapshot set will contain information about the state corresponding to an approximation of the optimal control .thanks to this numerical approximation of the optimal control problem we can build the snapshot matrix and compute the pod basis functions where the number is chosen such that . + the approximation of equation is very useful in model order reduction since we overcome the choice of the initial input control to generate the snapshot set. moreover , we also gain information about a temporal grid , which allows us to better resolve with respect to time .the a - posteriori error estimation guarantees that the finite element approximation of in the time variable is below a certain tolerance .therefore , the reduced optimal control problem is set up and solved on the resulting adaptive time grid .now the question is :  _ how good is the quality of the computed time grid in terms of the error between + the optimal solution and the pod surrogate solution ? _+   let us motivate our approach by analyzing the error between the optimal adjoint solution of associated with the optimal control for , i.e. and the pod reduced approximation , which is the time discrete solution to the pod - rom for associated with the time discrete optimal control for , i.e. in .we denote by the space and by the space . by the triangular inequality we get the following estimates for the -norm : where is the time discrete adjoint solution of associated with the control and the time discrete adjoint solution to with respect to the suboptimal control , i.e. in .by we denote the orthogonal pod projection operator as follows : the term ( [ err : est_new].1 ) can be estimated by and concerns the snapshot generation .thus , we can decide on a certain tolerance in order to have a prescribed error . the second term ( [ err : est_new].2 ) in is the pod projection error and can be estimated by the sum of the neglected eigenvalues .then , we note that the third term ( [ err : est_new].3 ) can be estimated as follows : where and is the constant referring to the lipschitz continuity of independent of as in . in order to control the quantity make use of the a - posteriori error estimation of , which provides an upper bound for the error between the ( unknown ) optimal control and any arbitrary control ( here and ) by where is the regularization parameter in the cost functional and is chosen such that is satisfied . finally ,last term ( [ err : est_new].4 ) can be estimated according to and involves the sum of the eigenvalues not considered , the first derivative of the time discrete adjoint variable and the difference between the state and the pod state : for a constant .we note that the sum of the neglected eigenvalues is sufficiently small provided that is large enough . furthermore , error estimation depends on the time derivative . to avoid this dependence ,we include time derivative information concerning the adjoint variable into the snapshot set , see .to summarize , the error estimation reads :    finally , we note that estimation involves the state variable which is estimated in the following section 5.2 .   in this sectionwe address the problem of the certification of the quality for pod approximation for the state variable .it may happen that the time grid selected for the adjoint will not be accurate enough for the state variable .therefore a further refinement of the time grid might be useful in order to reduce the error between the pod state and the true state below a given threshold .this is not guarenteed if we use the time grid , which results from the use of the estimate . here, we consider the error between the full solution corresponding to the suboptimal control and the time discrete pod solution , where we assume to have the same temporal grid for snapshots and the solution of our pod reduced order problem .in this situation , the following estimate is proved in :  [ post_p ] & \hspace{1 cm } + \displaystyle\sum_{j=1}^n c_y\left(\sum_{i=\ell+1}^d\left(|\langle \psi_i , y_0\rangle_v|^2+\lambda_i\right)\right ) \label{post_p_b}\\[0.4 cm ] & \hspace{1 cm } + \displaystyle\sum_{j=1}^n \sum_{i=\ell+1}^d c_y \dfrac{\lambda_i}{\delta{t_j^2 } } \label{post_p_c}\end{aligned}\ ] ]  where is a constant depending on , but independent of the time grid .we note that is the continuous solution of at given time instances related to the suboptimal control .the temporal step size in the subinterval  ] .we set .the example is built in such a way that the exact optimal solution of problem with associated optimal adjoint state is known : with and the control shape function for the operator .this leads to the right hand side the desired state and the initial condition .we choose the regularization parameter to be . for small values of ( we use ) ,the adjoint state develops a layer towards , which can be seen in the left plots of figure [ fig1:sol ] and figure [ fig2:con ] .( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]  ( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]  ( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]   ( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]  ( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]  ( left ) , pod adjoint solution utilizing an equidistant time grid with ( middle ) , pod adjoint solution utilizing an adaptive time grid with dof=21 ( right).,title="fig : " ]  in this test run we focus on the influence of the time grid to approximate of the pod solution .therefore , we compare the use of two different types of time grids : an equidistant time grid characterized by the time increment and a non - equidistant ( adaptive ) time grid characterized by degrees of freedom ( dof ) .we build the pod - rom from the uncontrolled problem ; we create the snapshot ensemble by determining the associated state and adjoint state corresponding to the control function and we also include the initial condition and the time derivatives of the adjoint into our snapshot set , which is accomplished with time finite differences of the adjoint snapshots .we use pod basis function .although we would also have the possibility to use suboptimal snapshots corresponding to an approximation of the optimal control , here , we want to emphasize the importance of the time grid . nevertheless in this example , the quality of the pod solution does not really differ , if we consider suboptimal or uncontrolled snapshots .first , we leave out the post - processing step 4 of algorithm [ alg : optpod ] and discuss the inclusion of this part later .+ figure [ fig3:mesh ] visualizes the space - time mesh of the numerical solution of ( [ 2ordp ] ) utilizing the temporal residual type a - posteriori error estimate ( [ est - thm31 ] ) .the first grid in figure [ fig3:mesh ] corresponds to the choice of dof=21 and , whereas the grid in the middle refers to using dof = 21 and .both choices for spatial discretization lead to the exact same time grid , which displays fine time steps towards the end of the time horizon ( where the layer in the optimal adjoint state is located ) , whereas at the beginning and in the middle of the time interval the time steps are larger .this clearly indicates that the resulting time adaptive grid is very insensitive against changes in the spatial resolution . for the sake of completeness, the equidistant grid with the same number of degrees of freedom is shown in the right plot of figure [ fig3:mesh ] . +since the generation of the time adaptive grid as well as the approximation of the optimal solution is done in the offline computation part of pod - mor , this process shall be perfomred quickly , which is why we pick for step 1 in algorithm 1 . +   according to the strategy in and ( left ) and ( middle ) , respectively , and the equidistant grid with ( right),title="fig : " ]  according to the strategy in and ( left ) and ( middle ) , respectively , and the equidistant grid with ( right),title="fig : " ]  according to the strategy in and ( left ) and ( middle ) , respectively , and the equidistant grid with ( right),title="fig : " ]   ( top left ) , approximation of the optimal control gained by step 1 of algorithm [ alg : optpod ] ( top right ) ; pod control utilizing a uniform time grid with ( bottom left ) , pod control utilizing an adaptive time grid with dof=21 ( bottom right),title="fig : " ]  ( top left ) , approximation of the optimal control gained by step 1 of algorithm [ alg : optpod ] ( top right ) ; pod control utilizing a uniform time grid with ( bottom left ) , pod control utilizing an adaptive time grid with dof=21 ( bottom right),title="fig : " ] +  ( top left ) , approximation of the optimal control gained by step 1 of algorithm [ alg : optpod ] ( top right ) ; pod control utilizing a uniform time grid with ( bottom left ) , pod control utilizing an adaptive time grid with dof=21 ( bottom right),title="fig : " ]  ( top left ) , approximation of the optimal control gained by step 1 of algorithm [ alg : optpod ] ( top right ) ; pod control utilizing a uniform time grid with ( bottom left ) , pod control utilizing an adaptive time grid with dof=21 ( bottom right),title="fig : " ]  figures [ fig1:sol ] and [ fig2:con ] ( middle and right plots ) show the surface and contour lines of the pod adjoint state utilizing an equidistant time grid and utilizing the time adaptive grid , respectively .the analytical control intensity , the approximation of the optimal control computed in step 1 of algorithm [ alg : optpod ] as well as the pod controls utilizing a uniform and time adaptive grid , respectively , are shown in figure [ fig2b : control ] .+ table [ tab:1 ] summarizes the approximation quality of the pod solution depending on different time discretizations .the fineness of the time discretization ( characterized by and dof , respectively ) is chosen in such a way that the results of uniform and adaptive temporal discretization are comparable .the absolute errors between the analytical optimal state and the pod solution , defined by , are listed in columns 2 and 6 ; same applies for the errors in the control ( columns 3 and 7 ) and adjoint state ( columns 4 and 8) .if we compare the results , we note that we gain one order of accuracy for the adjoint and control variable with the time adaptive grid . in order to achieve an accuracy in the control variable of order utilizing an equidistant time grid , we need about time steps ( not listed in table [ tab:1 ] ) .this emphasizes that using an appropriate ( non - equidistant ) time grid for the adjoint variable is of particular importance in order to efficiently achieve pod controls of good quality ..test 1 : absolute errors between the analytical optimal solution and the pod solution depending on the time discretization ( equidistant : columns 1 - 4 , adaptive : columns 5 - 8 ) [ cols="^,^,^,^,^,^,^,^",options="header " , ]in this paper we investigated the problem of snapshot location in optimal control problems .we showed that the numerical pod solution is much more accurate if we use an adaptive time grid , especially when the solution of the problem presents steep gradients .the time grid was computed by means of an a - posteriori error estimation strategy of space - time approximation of a second order in time and fourth order in space elliptic equation which describes the optimal control problem and has the advantage that it is independent of an input control function .furthermore , a coarse approximation with respect to space of the latter equation gives information on the snapshots one can use to build the surrogate model .finally , we provided a certification of our surrogate model by means of an a - posteriori error estimation for the error between the optimal solution and the pod solution . + for future work , we are interested in transferring our approach to optimal control problems subject to nonlinear parabolic equations .   k. afanasiev and m. hinze . adaptive control of a wake flow using proper orthogonal decomposition ._ lecture notes in pure and applied mathematics 216 _ , 317 - 332 .shape optimization & optimal design , marcel dekker , 2001 .e. arian , m. fahl and e. sachs .trust - region proper orthogonal decomposition models by optimization methods . _ in proceedings of the 41st ieee conference on decision and control , las vegas , nevada _ , 2002 , 3300 - 3305 .

Abstract: in this paper we study the approximation of an optimal control problem for linear parabolic pdes with model order reduction based on proper orthogonal decomposition ( pod - mor ) . pod - mor is a galerkin approach where the basis functions are obtained upon information contained in time snapshots of the parabolic pde related to given input data . in the present work we show that for pod - mor in optimal control of parabolic equations it is important to have knowledge about the controlled system at the right time instances . we propose to determine the time instances ( snapshot locations ) by an a - posteriori error control concept . this method is based on a reformulation of the optimality system of the underlying optimal control problem as a second order in time and fourth order in space elliptic system which is approximated by a space - time finite element method . finally , we present numerical tests to illustrate our approach and to show the effectiveness of the method in comparison to existing approaches .