bistability , and multi - stationarity in general , is ubiquitous in biological systems ranging from biochemical networks to epidemiological and eco - systems .it is considered an important biological mechanism for controlling cellular and bacterial behaviour and developmental processes in organisms , and it is closely linked to the idea of the cell as a decision making unit , where a continuous input is converted to an on / off response corresponding to two distinct states of the cell .  the question of bistability therefore arises naturally in many contexts .many studies aim to demonstrate that in a given biochemical system , bistability can or can not occur .this has created some interest in formal methods that connect the network structure to the dynamic behaviour of the system , see e.g.  .one qualitative network feature has in particular been linked to multi - stationarity , namely the existence of a _ positive feedback loop_. a positive feedback loop consists of a sequence of species such that each species affects the production of another species , either positively or negatively , and such that the number of negative influences is even .the idea of associating positive feedback loops with bistability goes back to jacob and monod who introduced it in the context of gene regulatory networks .it was later formalised by thomas in the form of a conjecture , which was finally proved by soul , see also .soul considers dynamical systems of the form where , is the vector of species concentrations , is the derivative of with respect to time , and is the so - called species - formation rate function , which specifies the instantaneous change in the concentrations .  the work of soul is based on the so - called _ interaction graph _ .this graph encodes how the variation of one species concentration depends on the concentration of the other species .it is built from the jacobian matrix of evaluated at a point , such that the non - zero entries of corresponds to directed edges of the graph and the signs of the entries are edge labels .soul s analysis is mainly informative when the signs of the entries are independent of .he proved that the existence of a positive feedback loop in the interaction graph is a necessary condition for to have multiple zeros .in other words , it is a necessary condition for multi - stationarity to exist in the ode system .soul s approach usually works well when modelling gene networks , but it is often useless when modelling enzymatic signalling networks . in this case , the interaction graph rarely has constant labels .a related line of work , that remedies this short - coming and might be seen as a refinement of soul s work , is based on the so - called _ directed species - reaction graph _( dsr - graph ) .if in is obtained from a reaction network , then it decomposes in the form where is the stoichiometric matrix of the network and the vector of reaction rates .the dsr - graph uses this particular structure .the dsr - graph is a bipartite graph with nodes labeled by the species and the reactions of the reaction network .there is a signed directed edge from a species to a reaction if the species concentration contributes either positively ( positive edge label ) or negatively ( negative edge label ) to the rate of the reaction .there is an edge from a reaction to a species if the stoichiometric coefficient of the species in the reaction is non - zero . in this casethe edge is labelled with the sign of the stoichiometric coefficient .compared to the interaction graph , the dsr - graph makes use of the explicit form of .the dsr - graph exists in different versions depending on the labelling of the edges ( signs versus stoichiometric coefficients ) and whether two opposite directed edges between the same node pair are combined into one undirected edge or not .it has been shown that the existence of positive feedback loops in the dsr - graph is a necessary condition for the system to admit multi - stationarity .based on these results it has become standard to highlight positive feedback loops in multi - stationary reaction networks , eg .the loops are typically found using intuitive reasoning that might overlook the existence of other relevant positive feedback loops or might select positive feedback loops that are not related to the existence of multi - stationarity .here we provide a method , based on theoretical considerations , to classify _ all _ positive feedback loops of a multi - stationary network into those that are related to the observed multi - stationarity and those that are not .in other words , we determine the positive feedback loops that when broken , multi - stationarity disappears .the question needs to be understood in the context of the whole network and not in isolation : a particular positive feedback loop that is responsible for multi - stationarity in one network might appear in another network that can not have multiple steady states .we present an automated procedure to determine the positive feedback loops that contribute to multi - stationarity .the procedure is based on the _ injectivity _ property applied to an ode system of the form , as described in . in this context , we review the proof of the fact that positive feedback loops are necessary for multi - stationarity and how this relates to the dsr - graph .we illustrate the procedure with examples of multi - stationary reaction networks involved in cell signaling .we further consider the networks in the biomodels database and apply the procedure to all non - injective networks ( injective networks can not be multi - stationary , see below ) .this provides an overview of the landscape of relevant positive feedback loops occurring in documented reaction networks .we use the formalism of chemical reaction network theory ( crnt ) .an ode system is built from a set of reactions and reaction rates .   a _ reaction network _ , or simply a _ network _, consists of a set of species and a set of reactions of the form : where are nonnegative integers , called the stoichiometric coefficients . as a running example we use the network in figure [ fig : santos ] .it has three species , x , x , x , which are different forms of the cdk1-cyclin b1 complex , and four reactions .we denote the concentration of the species by lower - case letters .the evolution of the species concentrations with respect to time is modelled as an ode system in the following way .we let be the stoichiometric matrix of the network : that is , the -th entry encodes the net production of species in reaction .the vector is called the _ reaction vector _ of reaction .the rate of reaction is a function , where and is the set of possible species concentrations .a typical choice of is _ mass - action kinetics_. in this case with the convention that .putting the pieces together provides a model for the evolution of the species concentrations over time :   returning to figure [ fig : santos ] , we let be the concentrations of x , x , x , respectively .following , one model of the network is : where are parameters .it takes the form with  and .observe that the phosphorylation reaction has a reaction rate that depends on both the concentration of the reactant x and the concentration of the product x .we also consider an alternative model in which the rate of x phosphorylation depends on only : this alternative model is also consistent with the set of reactions in figure [ fig : santos ] , but the third reaction is now independent of the amount of x .   the specific form of implies that the trajectories of the ode system are confined to the so - called _ stoichiometric compatibility classes _ : where in is the initial condition .that is , the trajectories are restricted to the space spanned by the reaction vectors .of particular interest is the ( relative ) _ interior _ of , also called the _ positive stoichiometric compatibility class _ , given by .any trajectory that starts in , stays there , but might be attracted towards the boundary .a reaction network is said to be _ multi - stationary _ if there exist two distinct steady states in a positive stoichiometric compatibility class ( but not necessarily in all classes ) .equivalently , if there exist distinct positive such that and .a network with one positive steady state and one steady state at the boundary is therefore not multi - stationary in this terminology .the reaction network in figure [ fig : santos ] is multi - stationarity for some choice of parameters with the rate vector in , but not with the rate vector in ( which will be shown later ) .the concept of a positive feedback loop is associated with structural network properties and qualitative features of the reaction rates .therefore , we assume some regularity on the reaction rates that we encode into an abstract symbolic matrix , called the _ influence matrix_. a feedback loop does not depend on specific parameters or the specific functional form of the reaction rates .  to proceed , we assume that the function is strictly monotone in each variable and define the influence matrix as where are symbolic variables .  the influence matrices associated with the two reaction rate vectors in and are given by and respectively .all influences are zero or positive .we define the dsr - graph as a labelled bipartite directed graph with node set and such that :  a.  there is an edge from to with label if .b.  there is an edge from to with label if .we refer to the _ signed dsr - graph _ as the graph identical to the dsr - graph given by ( a)-(b ) , but with the labels replaced by their signs .the ( signed ) dsr - graph of the running example with as in and as in is shown in figure [ fig : santosgraph ] . the ( signed ) dsr - graph with as in is identical to that in figure [ fig : santosgraph ] , with the edge from x to removed .   a _ circuit _ in a graph is a sequence of distinct nodes such that there is a directed edge from to for all and one from to . a circuit must involve at least one edge .the label of a circuit , denoted , is the product of the labels of the edges in the circuit .two circuits are disjoint if they do not have any common nodes .a circuit with positive label is a _ positive feedback loop _ , and similarly , a circuit with negative label is a negative feedback loop .the three positive feedback loops of the running example are shaded in figure [ fig : santosgraph]b .they correspond to shuttling of the complex between the nucleus and the cytoplasm , activation and deactivation of x , and self - activation of x ( the rate of reaction increases with , that is , the production of x increases with the amount of x ) .a -nucleus is a collection of disjoint circuits which involves nodes .the label of a -nucleus is the product of the labels of the edges in the nucleus .let be the number of circuits in the nucleus that have odd ( resp .even ) number of species nodes and let .the sign of a -nucleus is defined as .that is , if is a disjoint union of circuits , then   in the dsr - graph , any circuit involves an equal number of species and reaction nodes and , hence , an even number of edges . consider a -nucleus of the dsr - graph .we show that if none of the circuits are positive feedback loops , then the sign of is . indeed, if all circuits have negative labels , that is , for all , then because is a -nucleus , it contains species nodes .let be the number of species nodes in circuit , such that .we have that is odd for of the circuits and even for of the circuits. therefore , and if there are no positive feedback loops in .this result is also in , where it is stated using a different terminology .   in this sectionwe study injectivity of the function , . in the next section we link this injectivity property for all positive stoichiometric compatibility classes to the non - existence of positive feedback loops in the dsr - graph . with other words ,if all feedback loops are negative then the function is injective on all positive stoichiometric compatibility classes .in particular , there can not exist two distinct points in the same stoichiometric compatibility class such that , that is , the network can not be multi - stationary .  to decide whether the function is injective on all positive stoichiometric compatibility classes for any with given influence matrix , we rely on a method previously developed by us .we will now explain this method .given a pair of matrices , we define a polynomial of degree , in as many variables as there are non - zero entries of . for this ,let and let be a basis of , which we assume to be gauss - reduced .further , let be the indices of the first non - zero entries of , respectively .we define a symbolic matrix , , by replacing the -th row of with ( cf .* section 5 ) ) .the polynomial is defined as which can be written as a sum of terms depending on the variables , by expanding the determinant .each non - zero term is of the form where is a coefficient and all , respectively , are distinct .it is a result of that if is not identically zero and all non - zero coefficients of have the same sign , then the function is injective on each positive stoichiometric compatibility class and , hence , the network can not be multi - stationary . as a consequence , has coefficients of opposite sign whenever the network is multi - stationary .if the coefficients do not have the same sign , then the network might be multi - stationary , but it can not be concluded from the test .consider the matrix given in and in .we choose as a basis of and obtain there are both positive and negative terms , hence multi - stationarity can not be excluded . for in , the polynomial obtained from by setting , in this case all terms have the same sign and thus , the network can not be multi - stationary .this holds for any choice of rate functions with influence matrix .   in this sectionwe link injectivity and the polynomial to positive feedback loops . it is shown in that each term of the polynomial can be identified with a _ collection _ of disjoint unions of circuits in the dsr - graph .specifically , given subsets of cardinality , let be the set of -nuclei of with node set . then where the sets in the sum have cardinality ( cf .* section 11 ) ) .when is derived from a `` reasonable '' reaction network , the predominant sign of the coefficients is .this is because , for most reactions , the species in the reactant complex will have positive influence on the reaction rate , and the species in the product complex will have negative influence ; if they influence the reaction at all .consequently , if the network is multi - stationary , then has _ some _ terms with the _ wrong _ sign , that is , with sign . since whenever does not contain positive feedback loops , we conclude that the network must contain positive feedback loops in order to be multi - stationary .  based on the above considerations , we develop a procedure to extract the positive feedback loops that correspond to terms with the wrong sign in .fix a non - zero term of the polynomial , say ( is positive ) and consider the following edges from species to reactions the -nuclei corresponding to the term must contain these edges .therefore , we add to these edges all possible choices of edges from reactions to species such that the resulting graph is a -nucleus . we keep only the nuclei which the sign of is .the positive feedback loops in these nuclei are those that _ do _ contribute to the existence of multiple steady states .indeed , if all these loops are broken , then the network can not be multi - stationarity .we find these loops in the signed dsr - graph .for example , consider the polynomial in and the dsr - graph shown in figure [ fig : santosgraph ] .in this case , the rank of is , and hence we focus on the negative terms since .these are and .the corresponding -nuclei are depicted in figure [ fig : santosgraph](a ) : there are two 4-nuclei obtained as the union of the red circuit and one of the two blue circuits .the only positive feedback loop that appears is therefore the self - activation positive feedback loop , and this is the only loop that is related to the observed multi - stationarity .the other two positive feedback loops ( termed the spatial and the activation loop , respectively in figure [ fig : santosgraph](b ) ) are therefore not relevant for the observed multi - stationarity .   the procedure to select positive feedback loops that contribute to multi - stationarity consists of the following steps .  1 .  for a network with stoichiometric matrix of rank and influence matrix , compute and select the terms with sign .  construct the dsr - graph .for each selected term of with the wrong sign , determine the corresponding -nuclei of the dsr - graph that have the wrong sign .3 .  for each of the nuclei , select the connected components that form positive feedback loops .these steps have been implemented in maple and the script is available upon request .the procedure might fail for practical reasons ( such as lack of computational memory ) if the number of species and reactions is too big . in our experience, this number depends heavily on the sparsity of the influence matrix .we have applied the procedure to several multi - stationary networks .these examples are also in the maple script , together with some other systems such as the three - site phosphorylation system .we consider an ode model of histone h2a ubiquitination that involves the e3 ligases ring1b and bmi1 . when degradation of species is not taken into account , the model has species and reactions .we let b and b denote the protein bmi1 in isolation and targeted for degradation by ubiquitination , respectively . the protein ring1b is denoted by r , and r , r ,r denote three different forms of self - ubiquitinated r , with r being the form targeted for degradation .bmi1 and ring1b form a complex z , that also might be ubiquitinated , z . finally , ring1b ( either alone or in the complex z )is responsible for the ubiquitination of the histone h2a , with states h , h .the reactions describing the mechanism are : [r_2]}\text{b}_{ub}^d  &  \text{r } & \cee{<=>[r_3][r_4 ] }  \text{r}_{ub}^d  &  \text{b}+\text{r } &  \cee{<=>[r_5][r_6 ] } \text{z }  \\\text{z } &  \cee{<=>[r_7][r_8 ] }  \text{z}_{ub }  &    \text{z}_{ub }  & \cee{<=>[r_9][r_{10 } ] } \text{b}+\text{r}_{ub}^a  & \text{r }  & \cee{<=>[r_{11}][r_{12 } ] } \text{r}_{ub } \\\text{r}_{ub}^a  & \cee{->[r_{13 } ] } \text{r }  & \text{h } &  \cee{<=>[r_{14}][r_{15 } ] } \text{h}_{ub}.\end{aligned}\ ] ] we let denote the concentrations of b , b , r , r , r , r , z , z , h , h , respectively .the associated reaction rates are : where are constants .self - ubiquitination of b is taken into account in the rate functions and for reactions and , respectively .these incorporate a positive influence from the reaction products . with these specific rate functions ,the system is multi - stationary .we apply the automated procedure to find positive feedback loops and obtain the circuits depicted in figure [ fig : examples](a ) . in , it is postulated that self - ubiquitination of z and r are crucial steps for the emergence of multiple steady states , and we confirm the statement here .we have analysed different networks of signal transmission based on phosphorylation .we consider models for sequentially distributed phosphorylation and dephosphorylation cycles and some modifications of these , see e.g.  .  we consider first a two - site sequential phosphorylation cycle for a substrate s , where phosphorylation of the two sites is catalysed distributively by a kinase e , and dephosphorylation of the two sites uses different phosphatases f , f . assuming a michaelis - menten mechanism , the reaction network consists of the following reactions : [r_{2 } ] } \text{es}_0  \cee{->[r_{3 } ] } \text{e } + \text{s}_1 \cee{<=>[r_{7}][r_{8 } ] }\text{es}_1  \cee{->[r_{9 } ] } \text{e } + \text{s}_2 \\ \text{f}_1 + \text{s}_1 \cee{<=>[r_{4}][r_{5 } ] } \text{f}_1\text{s}_1  \cee{->[r_{6 } ] } \text{f}_1 + \text{s}_0  \\ \text{f}_2 + \text{s}_2  \cee{<=>[r_{10}][r_{11 } ] } \text{f}_2\text{s}_2  \cee{->[r_{12 } ] } \text{f}_2 + \text{s}_1 \end{aligned}\ ] ] in s , s , s the subindex denotes the number of phosphorylated sites .with mass - action kinetics , this system is multi - stationary .however , the positive feedback loops that can account for the observed multi - stationarity are not trivial .we apply the automated procedure and obtain the positive feedback loops given in figure [ fig : examples](b ) .the first of the two loops has two edges with negative labels .it implies that s and s enhance their own production .indeed , because s and s both compete for the same kinase , an increase in s increases the rate of reaction , which in turn lowers the amount of available enzyme e. this implies that reaction slows down and hence s is consumed at a slower rate .the circuit closes through the reactions and , which shows that an increase in s implies an increase in s .these type of loops are recurrent in phosphorylation motifs .it is worth emphasising that the loops do not have independent meaning outside to network .another network with the same positive feedback loop might not be multi - stationarity .this is apparent from the second positive feedback loop in figure [ fig : examples](b ) , which is present whenever a michaelis - menten enzymatic mechanism is considered .we consider a 2-layer cascade with an explicit positive feedback .the first layer is a phosphorylation cycle with kinase e , phosphatase f , and phosphorylated and unphosphorylated substrate s , s .the second layer has kinase s , phosphatase f , and phosphorylated and unphosphorylated substrate p , p . assuming a michaelis - menten mechanism , the reaction network consists of the following reactions : [r_{2 } ] } \text{es}_0   \cee{->[r_{3 } ] } \text{e } + \text{s}_1 \\ \text{f}_1 + \text{s}_1 \cee{<=>[r_{4}][r_{5 } ] } \text{f}_1\text{s}_1   \cee{->[r_{6 } ] } \text{f}_1 + \text{s}_0 \\\text{s}_1 + \text{p}_0  \cee{<=>[r_{7}][r_{8 } ] } \text{s}_1\text{p}_0  \cee{->[r_{9 } ] } \text{s}_1 + \text{p}_1 \\\text{f}_2 + \text{p}_1 \cee{<=>[r_{10}][r_{11 } ] } \text{f}_2\text{p}_1  \cee{->[r_{12 } ] } \text{f}_2 + \text{p}_0 \\\text{p}_1   \cee{->[r_{13 } ] } \text{e}. \end{aligned}\ ] ] the automated procedure finds three positive feedback loops , as shown in figure [ fig : examples](c ) .the first loop is expected and appears because the product of the second layer , p , activates the kinase of the first layer , e. the other two loops are similar to those in figure [ fig : examples](c ) .we finally consider a basic model of caspase activation for apoptosis : } \text{c}_8^*+\text{c}_3^ *  &  \text{c}_8 & \cee{<=>[r_{6}][r_{7 } ] } 0\\ \text{c}_8+\text{c}_3^ * &  \cee{->[r_2 ] } \text{c}_8^*+\text{c}_3^ * & \text{c}_3 &  \cee{<=>[r_{8}][r_{9 } ] } 0\\  \text{c}_3^*+\text{iap } &  \cee{<=>[r_{3}][r_{14 } ] }  \text{y } \cee{->[r_4 ] } 0 & \text{iap } &  \cee{<=>[r_{10}][r_{11 } ] } 0 \\ \text{c}_3^*+\text{iap } & \cee{->[r_5 ] } \text{c}_3^*\cee{->[r_{13 } ] } 0  & \text{c}_8^ * & \cee{->[r_{12 } ] } 0.\end{aligned}\ ] ] with mass - action kinetics , this network is multi - stationary and has two relevant positive feedback loops , figure [ fig : examples](d ) .the second loop consists of two species , each with positive influence on a reaction rate , while at the same time , decreasing the amount of the other .we investigated the models in the database pocab , which consists of 365 models from the publicly available database biomodels ( see also the page http://www.ebi.ac.uk/biomodels-main/ ) .the database pocab contains pre - computed stoichiometric matrices , mass - action exponent matrices , and kinetic data from the selected models .  in a previous paper we extracted influence matrices of the reported kinetics .of the 365 networks , 323 have strictly monotone kinetics such that the influence matrix is well defined . on these 323 networks we ran the injectivity method and ended up with a total of 64 non - injective networks , excluding 27 very large networks where the injectivity method failed ( as described in ) .non - injectivity is a prerequisite for being multi - stationary .we applied the automated procedure on the 64 networks to determine the positive feedback loops .we obtained a total of 341 different positive feedback loops with size distribution shown in figure [ fig : biomodels](c ) .most loops involve only 2 or 3 species ( 112 and 108 loops out of 341 , respectively ) . in figure[ fig : biomodels](a - b ) , we show the positive feedback loops involving one or two species and conclude that all possible types occur in the database .however , for two species , their frequencies vary from 9 to 35 ( out of 108 ) , indicating that the motifs are not equally represented in the database ( pearson s chi - square test , ) . for one species, there appears to be no difference ( ) .we show in table [ table : biomodels ] the most frequent positive feedback loops for different number of species nodes .we have presented an automated procedure to find the positive feedback loops in a multi - stationary network .the procedure relies on structural and qualitative features of the network together with a kinetics , and it is insensitive to the specific form of the rate functions . only positive feedback loops that are contributing to the multi - stationarity of the networkare reported , excluding those positive feedback loops that are not relevant .  whether a loop is relevant or not , depends on the entire dsr - graph of the network ( that is , the reactions and the influence matrix ) and not just on the loop itself . in this sense , being a positive feedback loop related to an observed multi - stationarity , is a context or network dependent property .this fact has also been observed in . in these papers , it is shown that the existence of a positive feedback loop fulfilling a certain extra condition is a requirement for multi - stationarity to occur . specifically , the loop must either intersect another positive feedback loop in a specific way ( called an _s - to - r intersection _ ) or fulfil a certain condition on the labels ( called an _s - cycle _ ) .the first possibility is network dependent .it is worth mentioning that there can be positive feedback loops that are s - cycles or that intersect another positive feedback loop in an s - to - r intersection without being relevant for the observed multi - stationarity .this is the case for most of the examples presented here .for example , the apoptosis network has another positive feedback loop , y ( all edges are positive ) , which intersects the loop on the right side in figure [ fig : examples](d ) in an s - to - r intersection .the property of network dependence is further illustrated in figure [ fig : santosgraph](a)-(b ) , where the procedure is applied to the reaction network in figure [ fig : santos ] with influence matrix given by .the network models translocation and phosphorylation of a cyclin dependent kinase x in the onset of mitosis .only one of the three positive feedback loops shown in figure [ fig : santosgraph](b ) can be associated with multi - stationarity in the model , namely the self - activating loop that stimulates the creation of phosphorylated x in the nucleus . in , it is argued by different means than in this paper , that the spatial redistribution of the cyclin dependent kinase is important for creating the observed bistability .in contrast , our results suggest that the observed bistability is due to the self - activation loop of the phosphorylated complex in the nucleus.  the presented procedure can not establish whether a reaction network is multi - stationary or not .other means will here be required . if the procedure is applied to a reaction network that might or might not be multi - stationary , then the absence of positive feedback loops implies that the network can not be multi - stationary , whereas the presence of positive feedback loops leaves the question open .  whether a positive feedback loop found by the procedure is important in a biological or experimental context ,is not addressed in this paper , but must be established in other ways , for example by experimental verification .a reaction network might only be multi - stationary for very specific choices of reaction rates , which might not be relevant in a particular experimental setting .as the procedure is parameter independent , any such verification and subsequent interpretation is beyond the scope of the procedure .cw was supported by the danish research council , the lundbeck foundation ( denmark ) and the carlsberg foundation ( denmark ) .ef was supported by project mtm2012 - 38122-c03 - 01/feder from the ministerio de economa y competitividad ( spain ) , and the lundbeck foundation ( denmark ) .this paper was completed while the authors were visiting the isaac newton institute in cambridge , uk .l. k. nguyen , j. muoz - garca , h. maccario , a. ciechanover , w. kolch , and b. n. kholodenko .switches , excitable responses and oscillations in the ring1b / bmi1 ubiquitination system . , 7(12):e1002317 , 2011 .

Abstract: bistability is ubiquitous in biological systems . for example , bistability is found in many reaction networks that involve the control and execution of important biological functions , such as signalling processes . positive feedback loops , composed of species and reactions , are necessary for bistability , and generally for multi - stationarity , to occur . these loops are therefore often used to illustrate and pinpoint the parts of a multi - stationary network that are relevant ( ` responsible ' ) for the observed multi - stationarity . however positive feedback loops are generally abundant in reaction networks but not all of them are important for subsequent interpretation of the network s dynamics .   we present an automated procedure to determine the relevant positive feedback loops of a multi - stationary reaction network . the procedure only reports the loops that are relevant for multi - stationarity ( that is , when broken multi - stationarity disappears ) and not all positive feedback loops of the network . we show that the relevant positive feedback loops must be understood in the context of the network ( one loop might be relevant for one network , but can not create multi - stationarity in another ) . finally , we demonstrate the procedure by applying it to several examples of signalling processes , including a ubiquitination and an apoptosis network , and to models extracted from the biomodels database .  we have developed and implemented an automated procedure to find relevant positive feedback loops in reaction networks . the results of the procedure are useful for interpretation and summary of the network s dynamics .


for a long time microstrip antennas have been miniaturized using different material fillings between the radiating element and the ground plane .most traditionally , high permittivity dielectrics have been used to decrease the physical dimensions of the radiator .common problems encountered with high permittivity substrates include e.g. the excitation of surface waves leading to lowered radiation efficiency and pattern degradation , and difficulties in impedance matching of the antenna . during the first heyday of so called metamaterials , artificial high - permeability materials working in the microwave regimehave gained increasing attention . the possibility to create artificial magnetism at microwave frequencieshas heated the discussion on the possibility to enhance the impedance bandwidth properties of planar radiators using magneto - dielectric substrates or other electromagnetically exotic substrates .according to the work of hansen and burke , inductive ( magnetic ) loading leads to an efficient size miniaturization of a microstrip antenna . a transmission - line ( tl ) model for a normal half - wavelength patch antennapredicts that increase in the permeability of the antenna substrate does not reduce the impedance bandwidth of the miniaturized radiator ( when the material parameters are _ dispersion - free _ , and ) .edvardsson derived a condition for the radiation quality factor of a planar inverted f - antenna ( pifa ) , and concluded that increase in the permeability allows size miniaturization without increasing .however , to draw a general conclusion on the practical applicability of the magnetic loading e.g. with mobile phone antennas , there are three things which need to be clarified : i ) the effect of frequency dispersion of the substrate .practically realizable artificial magnetic medium operating at microwave frequencies is composed of electrically small resonating metal unit cells , thus , the magneto - dielectric substrate obeys strong frequency dispersion .even though the substrate can be considered as a paramagnetic medium when using plane wave excitation ( at the macroscopic level , over a certain frequency range ) , the performance is not so obvious to predict when the resonant substrate is coupled to the antenna element .ii ) the effect of high compared to over the matching band of the antenna . with practically realizable artificial magneto - dielectric substrates for microwave frequenciesre is known to be rather moderate , thus in practice it seems very difficult to achieve the condition ( outside the resonant region of the substrate ) .iii ) the effect of losses .realistic model representing the losses of the substrate is needed to correctly predict the behavior of the susceptance seen at the antenna terminal ( further , the quality factor ) . moreover, the numerical and experimental results found in the literature do not offer a complete , quantitative comparison scheme applicable for practical antenna design . according to the authors knowledge , the impedance bandwidths ( quality factors ) of reduced size antennas loaded with practically realizable , artificial , dispersive magneto - dielectricshave not been compared to the results obtained using high - permittivity dielectrics leading to the same size reduction .when thinking of practical antenna design , this quantitative comparison is clearly needed to make decisions whether or not the possibly enhanced impedance bandwidth outweights e.g. the increased manufacturing cost and substrate weight .  in real lifelarge permeabilities for microwave materials are achieved with a complex mixture of electrically small inhomogeneities ( resonating unit cells ) loaded into a substrate in a specific periodic arrangement . in the present paperwe shortly discuss the possibilities to enhance artificial magnetism at microwave frequencies , and construct a tl model taking into account realistic dispersive behavior of a practically realizable artificial magneto - dielectric substrate .we start the analysis by reproducing the known miniaturization results with static material parameters , and further extend the study to take into account frequency dispersion .it is shown that a substrate obeying the lorentzian type dispersion for magnetic permeability leads to a narrower impedance bandwidth than a high - permittivity substrate offering the same size reduction .physical reason for the phenomena is clarified .a prototype antenna is constructed and the observation is experimentally validated .the rest of the paper is organized in the following way : in section ii we briefly revise the numerical and experimental results found in the literature for magneto - dielectric loading of microstrip antennas .section iii presents a short discussion on practically realizable artificial magnetic media and the used tl - model is presented in detail in section iv .section v presents the calculated impedance bandwidth properties in different loading scenarios , and an experimental demonstration is presented in section vi .the work is concluded in section vii .mosallaei and sarabandi applied in a finite - difference time - domain ( fdtd ) simulation scheme and investigated antenna miniaturization using a band - gap substrate consisting of dielectric and magneto - dielectric ( z - type hexaferrite ) layers .constant scalar - permeability assumption was used with substrate material parameters , at 277 mhz .magneto - dielectric substrate was shown to lead to a significantly wider impedance bandwidth than a pure dielectric substrate offering the same size reduction .the same authors introduced in an embedded circuit metamaterial structure and utilized it under a patch antenna operating at 2.3 ghz ( fdtd simulations ) . a 1.5 percent fractional bandwidth was demonstrated for a 0.075 size patch antenna with lossless , dispersion - free substrate material parameters ( ) .the authors did not , however , present any result for high permittivity dielectrics leading to the same size reduction .al _ measured the performance of a patch antenna utilizing an embedded circuit metamaterial at 250 mhz , and reported that miniaturization factors as high as 6.4 ( with 0.83 percent ) could be achieved .however , the radiation efficiency was measured to be only 21.6 percents , and a comparative measurement with high - permittivity dielectrics was not conducted .yoon and ziolkowski simulated a microstrip antenna with different material fillings ( dispersion - free material parameters ) , and came to the conclusion that the optimal condition for the effective substrate material parameters is the same as predicted by hansen and burke , and edvardsson ( ) . in krkkinenal _ numerically studied a pifa with dispersive magnetic material filling .the authors pointed out the need to regard the loaded radiator as a system of two coupled resonators , rather than a radiator loaded with a static paramagnetic load .the conclusion was that if the resonance of the material is considerably higher than the resonance of the empty antenna , the material loading enables size reduction while approximately retaining the fractional bandwidth .the effective permittivity of the substrate was assumed to be unity , and the design utilizing dispersive magnetic filling was not challenged against a design utilizing dispersion - free dielectrics offering the same size reduction .ermutlu _ et .al _ loaded the volume under a half - wavelength patch antenna with a practically realizable artificial magneto - dielectric material , and concluded based on numerical and experimental results that significant miniaturization factors could be achieved while the bandwidth of the loaded antenna is approximately retained .the results were not , however , challenged against results obtained using high - permittivity substrates .authors of proposed two utilization schemes for a resonant magnetic media with planar radiators .the volume under a pifa was loaded with an array of metasolenoids and the impedance bandwidth properties were experimentally investigated .when utilizing the resonant region of the metasolenoids , the authors designed a multiresonant antenna with significantly enhanced impedance bandwidth . according to the second suggestion, the metasolenoid array would behave as a paramagnetic load enabling efficient antenna miniaturization , air filling leads to the best figure of merit . ] .authors of loaded the volume under a half - wavelength patch antenna with an array of metasolenoids and compared the impedance bandwidth to that obtained using high - permittivity dielectrics .the result indicated that there is practically no advantage when using the metasolenoid array .attempts have also been conducted to miniaturize the size of planar antennas by partially filling the volume under the radiating element with backward - wave materials . with patch antennas , in practise , filling the volume with backward - wave material corresponds to inserting bulk inductors and capacitors to the antenna , thus the technique is very similar to the well known technique of size miniaturization using reactive loads .moreover , as expected , material obeying realistic dispersive behavior results in significantly narrower impedance bandwidth than the hypothetical material with  .in the present paper we restrict the discussion only to artificial magnetic media composed of resonating metal unit cells . other ways to enhance the magnetic response include e.g. the utilization of ferrites , or composites filled with ferromagnetic inclusions . in general ,resonant artificial magnetic media can be utilized with planar radiators in two principal ways : if the resonance of the material lies inside the operational band of the ( loaded ) radiator , with a suitable coupling the resonance of the material can be combined with the antenna resonance , thus a multiresonant antenna is achieved . in other words , radiating particles are used instead of a real material filling . another way is to design the material to resonate at a considerably higher frequency than the operational frequency of the loaded radiator . in the latter caseit is important that the utilized material retains its effective magnetic properties over a wide frequency band , meaning that inside the operational band of the radiator the value of the real part of the effective permeability is larger than unity , re ( low losses are another desired feature ) .this allows one to consider the material as a paramagnetic load with ( almost ) constant real part for the effective permeability over the desired frequency range .a paramagnetic load will increase the inductance of the resonator , and many authors consider this as an advantage when trying to retain the impedance bandwidth characteristics in antenna miniaturization with material fillings , see e.g.  .the problem with most of the introduced designs for artificial magnetic media operating in the microwave regime is the fact that the effective magnetism rapidly vanishes as the frequency deviates from the particle resonance .usually the maximum predicted value for the real part of the effective permeability of artificial magnetic medium ( with realistic bulk concentration / volume filling ratio and loss factor ) is re at the resonance .this means that in practice re of the magneto - dielectric substrate approaches unity very quickly as the frequency deviates from the material resonance .so in practice one should utilize the particles rather close to their resonance to achieve a condition re .this further increases the effect of frequency dispersion in the impedance bandwidth characteristics .moreover , it might not be clear anymore if the loaded antenna should be considered as a system of two coupled resonators rather than a resonator filled with a homogenous , dispersion - free material .in addition to this , usually the real part of the effective permittivity of magneto - dielectric substrates is considerably higher than the real part of the effective permeability ( when the substrate is utilized well below its resonance ) .thus , at microwave frequencies the condition considered e.g. by hansen and burke ( ) seems unlikely to be achieved with a resonant media composed of metal unit cells .for example , with the structure presented in a realistic value for the real part of the effective permeability over the operational band of a patch antenna ( operating well enough below the magneto - dielectric substrate resonance ) is re .however , at the same time the real part of the effective permittivity can be as high as re , even if the host substrate has a low value for the relative permittivity .  to give a link between the presented effective medium model and practical design for artificial magnetic media , we need to choose a unit cell for the media . in the present work we consider the metasolenoid as an example unit cell for practically realizable artificial magnetic media .[ meta_meas ] . introduces the structural geometry of the metasolenoid .metasolenoid is chosen due to its simple structural geometry enabling rather straightforward , yet accurate theoretical analysis .moreover , experimental results can be found in the literature for the utilization of the metasolenoid under microstrip antennas , and the effective medium model has been experimentally validated and found to be accurate .in this section we will derive expressions for the input impedance and unloaded quality factor of an arbitrary size strip fed patch antenna ( lying on top of a large , non - resonant ground plane ) .the impedance level seen directly at the patch edge is usually very high causing difficulties in impedance matching of the antenna .the strip feeding allows one to conveniently tune the impedance level seen at the feeding point , thus it offers a possibility to tune the matching in different scenarios . a schematic illustration of the analyzed antenna structure and the equivalent tl - model are presented in fig . [ sche ] .the characteristic impedance of a wide microstrip line reads : z = , z where is the wave impedance in free space , is the height of the substrate ( assumed to be the same as the height of the patch from the ground plane ) , is the length of the patch , and and are the effective substrate material parameters .  in the present model we neglect the effect of the material filling to the radiation conductance and the shunt susceptance representing the open end extension of the tl .this is possible if the material filling leaves the ends of the patch free so that there is a short ( about the patch height ) section of the empty tl just near the radiating edges ( in the present analysis the length of the empty section ) . with this assumption the radiation conductance and shunt susceptance read :  g_r= , w>2_0 .[ gr ] b = 2 .[ b ] above is the width of the patch , and is the wavelength in free space . in eq .( [ b ] ) is the open end extension of the tl ( with a wide microstrip ) .the propagation factor in the antenna part of the tl is of the following form = k_0 , [ beta ] where is the free space wave number .for the input impedance of the antenna we can write z_in = z_f , [ zin ] where is the impedance seen from the reference plane of the first radiating slot and is the length of the feed strip .the impedance of the ( narrow ) feed line is z_f = 1 + + , [ zfl ] where is the width of the feed strip .the impedance level of the feeding probe ( connected to the end of the strip ) is assumed to be , and the reflection coefficient is defined as = .[ roo ] to relate the unloaded quality factor to the return loss characteristics , we regard the antenna as a parallel - resonant rlc circuit in the vicinity of the fundamental resonant frequency .the coupling coefficient describes the quality of the matching between the source and the resonator and is defined as t = , [ t ] where is the admittance of the feed line , and is the conductance of the parallel resonator .let us assume that the criterion for the maximum vswr inside of the band of interest is s. [ vswr ] in this case the fractional bandwidth can be expressed as bw = , [ bwr ] where is the unloaded quality factor of the parallel resonator . knowing the dispersive behavior of the input return loss , can be solved from ( [ bwr ] ) .a half - wavelength patch antenna fed directly at the patch edge operates as a parallel resonant circuit when .the feed strip can be interpreted as an additional inductance placed in series with the rest of the circuit .this inductance will create a series resonance above the parallel resonance , and usually the antenna operates in between the series and the parallel resonance .the dispersive behavior of the metasolenoid array is of the following form ( see fig .[ meta_meas ] ) : _ eff = 1 - v_r , [ permeability ] where is the cross - section area of the metasolenoid , and is a coefficient taking into account realistic bulk concentration .the total impedance of the metasolenoid reads : z_tot = j + + r_eff .[ ztot ] the effective permittivity of the metasolenoid array does not equal to that of the host substrate relative permittivity ( ). the electric resonant behavior of the metasolenoid is very weak ( compared to the magnetic resonant behavior ) , thus a good approximation for the permittivity is a frequency independent expression _eff = m(v_r)_eff^s , [ eps_eff ] where is a multiplication factor depending on the bulk concentration ( ) , and is the effective permittivity of a host substrate having relative permittivity .in the first case we load the volume between the patch element and the ground plane with hypothetical , lossless substrates having static material parameters .the following loading scenarios are considered ( values are for the substrate material parameters ) : 1 ) , 2 ) , , 3 ) , , 4 ) .the empty antenna has dimensions ( see fig .[ sche ] ) mm , mm , mm , mm . with the aforementioned dimensions the empty antenna resonates at 3.0 ghz .the dimensions of the patch loaded with high- , high- , and magneto - dielectric substrates are the following : mm , mm , mm , mm ( high- substrate ) , mm ( high- substrate and magneto - dielectric substrate ) .feed strip width has been left as a free parameter and is used to tune the quality of coupling in different loading scenarios . with all the loading scenarios has been tuned to  by varying .this allows us to neglect the square root term in ( [ bwr ] ) when comparing unloaded quality factors ( note that coupling fine tuning is a common procedure in practical antenna design where the goal usually is to maximize the impedance bandwidth ) .[ s11_hansen ] shows the calculated reflection coefficient for a patch filled with different substrates .the main calculated parameters are gathered in table [ hansen_t ] . the unloaded quality factor obtained from eq .( [ bwr ] ) has been calculated using 6 db matching criterion .we can observe from fig .[ s11_hansen ] and table [ hansen_t ] that the impedance bandwidth properties behave similarly as predicted by hansen and burke : dispersion - free high- materials allow size miniaturization while practically retaining the impedance bandwidth whereas when using high- materials the impedance bandwidth suffers significantly .we can also observe from fig .[ s11_hansen ] that increase in permittivity rather quickly increases the unloaded quality factor and narrows the impedance bandwidth ( case `` magneto - dielectric '' ) ..the calculated impedance bandwidth properties . dispersion - free . [ cols="<,^,^,^ " , ]inspiring discussions with prof .pertti vainikainen , dr . jani ollikainen , and dr .murat ermutlu are warmly acknowledged .s. i. maslovski , p. ikonen , i. a. kolmakov , s. a. tretyakov , m. kaunisto , `` artificial magnetic materials based on the new magnetic particle : metasolenoid , '' _ progress in electromagnetics research _ , vol .54 , pp .6181 , 2005 .a. n. lagarkov , v. n. semenenko , v. n. kisel , v.a .chistyaev , `` development and simulation of microwave artificial magnetic composites utilizing nonmagnetic inclusions , '' _ j. magnetism and magnetic materials _ ,258259 , pp . 161166 , 2003 .c. r. simovski , a. a. sochava , s. a. tretyakov , `` new compact and wide - band high impedance surface , '' _ ieee antennas propagat .symposium _ , monterey , california , vol . 1 , pp . 297300 , june 2025 , 2004 .k. buell , h. mosallaei , k. sarabandi , `` embedded - circuit magnetic metamaterial substrate performance for patch antennas , '' _ ieee antennas propagat .symposium _ , monterey , california , pp . 14151418 , june 2025 , 2004 .s. yoon and r. w. ziolkowski , `` bandwidth of a microstrip patch antenna on a magneto - dielectric substrate , '' _ ieee antennas propagat .symposium _ , columbus , ohio , pp . 297300 , june 22 - 27 , 2003 .m. e. ermutlu , c. r. simovski , m. k. krkkinen , p. ikonen , s. a. tretyakov , a. a. sochava , `` miniaturization of patch antennas with new artificial magnetic layers , '' _ 2005 ieee int .workshop on antenna technology _ ,singapore , pp .8790 , march 79 , 2005 ; + also : m. e. ermutlu , c. r. simovski , m. k. krkkinen , p. ikonen , a. a. sochava , s. a. tretyakov , `` patch antennas with new artificial magnetic layers , '' submitted to _ ieee wireless component lett ._ preprint available at http://arxiv.org/abs/physics/0504075 .p. ikonen , s. maslovski , s. tretyakov , `` pifa loaded with artificial magnetic material : practical example for two utilization strategies , '' _ microwave opt . technol . lett ._ , vol .46 , no . 3 , pp .205210 , 2005 .d. sievenpiper , l. zhang , r. f. jimenez broas , n. g. alexpolous , e. yablonovitch , `` high - impedance electromagnetic surfaces with a forbidden frequency band , '' _ ieee trans .microwave theory tech ._ , vol .47 , no . 11 , pp . 20592074 , 1999 .o. edvardsson , `` on the influence of capacitive and inductive loading on different types of small patch / pifa structures for use on mobile phones , '' _ icap int .antennas propagat ._ , manchester , uk , pp . 762765 april 1720 , 2001 .s. f. mahmoud , `` a new miniaturized annular ring patch resonator partially loaded by a metamaterial ring with negative permeability and permittivity , '' _ ieee antennas wireless propagat ._ , vol . 3 , pp .1922 , 2004 .a. n. lagarkov , a. v. osipov , k. n. rozanov , s. n. starostenko , `` microwave composites filled with thin ferromagnetic films .part i. theory , '' _ proc .symposium r electromagnetic materials _ ,singapore , pp . 7477 , july 38 , 2005 .i. t. iakubov , a. n. lagarkov , s. a. maklakov , a. v. osipov , k. n. rozanov , i. a. ryzhikov , `` microwave composites filled with thin ferromagnetic films .experiment , '' _ proc .symposium r electromagnetic materials _ ,singapore , pp .7881 , july 38 , 2005 .

Abstract: in the present paper we discuss the effect of artificial magneto - dielectric substrates on the impedance bandwidth properties of microstrip antennas . the results found in the literature for antenna miniaturization using magnetic or magneto - dielectric substrates are revised , and discussion is addressed to the practically realizable artificial magnetic media operating in the microwave regime . using a transmission - line model we , first , reproduce the known results for antenna miniaturization with non - dispersive material fillings . next , a realistic dispersive behavior of a practically realizable artificial substrate is embedded into the model , and we show that frequency dispersion of the substrate plays a very important role in the impedance bandwidth characteristics of the loaded antenna . the impedance bandwidths of reduced size patch antennas loaded with dispersive magneto - dielectric substrates and high - permittivity substrates are compared . it is shown that unlike substrates with dispersion - free permeability , practically realizable artificial substrates with dispersive magnetic permeability are not advantageous in antenna miniaturization . this conclusion is experimentally validated .   * key words * : patch antenna , high - permeability materials , magneto - dielectric substrate , frequency dispersion , quality factor , miniaturization .


most complex systems , living systems in particular , are characterized by remarkable degrees of stability and at the same time by a tremendous potential of flexibility and adaptability .this has led some authors to define complex and living systems as living at the edge of chaos , meaning in a somewhat picturesque way that it takes only tiny changes in the system to move it from a stable and regular mode into a chaotic phase where large portions of phase space can get sampled .the concept is that systems at the edge of chaos are especially well suited for adaptation and information processing in the sense that adaptability is associated by the possibility of finding adequate new states in possibly changed environments at very fast rates .it has been argued that living systems at the edge of chaos would get favored by natural selection , and that life has evolved towards such a special region in parameter space . in many dynamical systemsthe edge of chaos is a very special set of points in parameter space , often of measure zero , characterized by the system s maximal lyapunov exponent passing through zero .it is not clear how systems can get regulated towards ( or have evolved towards ) such a limited set of critical points , even though some interesting ideas have been proposed in this direction . even in the simplest maps like the logistic map, the dynamics exactly at these special points can become highly non - trivial .it is evident that living systems have evolved towards stable systems in stationary disequilibrium .various authors argue that a key principle of living systems is their ability to replicate ; corresponding rate equations for molecular replicators have been proposed for a long time , beginning with . as such , basic molecular reactions in living systems (e.g. protein production or degradation ) have to be autocatalytic .if autocatalytic reactions are not balanced by degradation and/or thermostatic net - flow of substance to and fro the system ( like in a flow reactor ) , concentrations of molecular products will diverge in the replicator . a stationary state can be established when production and decay ( flow ) rates of inter cellular molecules effectively balance each other , . in this sense stability ( stationarity )provides a natural selection criterion .catalytic reactions are simply described by reaction networks , which contain production and degradation rates . given current developments in genomics- and proteomics technology some facts about these networks become known .by now there is some evidence that these ( directed ) networks show scale - free ( sf ) topological organization .on the basis of a given molecular reaction topology several gene network models have been proposed . in principletwo different approaches have been pursued : discrete approaches , using boolean networks and continuous approaches , using ordinary or stochastic differential equations .combinations of both have also been reported .recently the importance of noise in molecular reaction networks has been stressed and its relevance has been experimentally demonstrated .for example the level of noise can determine whether cells in drosophila become epidermal or neural cells .further it was shown that low reproduction rates of dna and important regulatory molecules forbid to neglect stochastic effects .intrinsic noise , microscopic events within the cell , and extrinsic noise , such as cell to cell variations , are experimentally distinguishable . in this contexta stochastic differential equation model has been proposed for regulatory transcription networks .  in this workwe study a simple linear , noise driven dissipative model for catalytic molecular reactions .we impose a non - linearity to this system by assuming that concentrations can not become negative .we demonstrate that this non - linearity changes the the edge of chaos from a point where , to extended regions of vanishing lyapunov exponents .the model offers a way to understand how systems naturally evolve and adapt towards a widened edge of chaos.we assume that gene to gene interactions can be modeled as chemical reactions between proteins , mrna and other nucleic material .let us denote the concentrations of proteins at time by the vector and of rna molecules by .for convenience let us combine all types of concentrations into a single vector .the size of the vector ( number of products ) we denote by . the simplest linear model to capture all possible interactionsis given by where is the matrix of reaction rates .even though this model is clearly an over - simplification of reality it has been frequently used recently .let us assume that these rates are not perfect constants but fluctuate according to , for example through thermal noise . for simplicity let be an iid process with zero mean . replacing by we get regardless of the distribution of , and assuming that will converge to a reasonably stationary distribution , according to the central limit theorem , the sum of the right hand side will yield a random number from a gaussian distribution , which we shall denote by .for simplicity we shall assume also gaussian , i.e. the same variance  .we need a final addition of our model eq .( [ model_raw ] ) , to incorporate a further experimental observation .many gene - products ( e.g. mrna levels ) fluctuate around some characteristic value , , across the cell cycle . in fig .[ tsallisfig]a we show the expression levels for mrna levels of 10 randomly picked genes from the yeast genome ( _ s.cerevisae _ ) over 2 cell cycles at 17 time points taken at 10 minute intervals .the number of measured genes in the budding yeast genome was . to incorporate these characteristic levels we chose values from some distribution . in fig[ tsallisfig]b we show the experimental distribution of mrna expression levels of the yeast genome , defined as the time - average over cell cycles . . in the following taken from uniform , gaussian or the experimental distribution .this fixes our model to be with and the multiplicative and additive noise components respectively .multiplicative and additive noise can be interpreted as intrinsic and extrinsic noise as used e.g. in .both intrinsic and extrinsic noise are present in gene networks .  to be able to interpret as concentrations we have to introduce the constraint which means that regardless of of ( [ model ] ) , can never be below zero .this imposes a non - linearity onto the system and makes it non - trivial .   before solving the system we have to specify the interaction network , i.e. the matrix elements ( chemical rates ) of .it is obvious that the network will be directed and weighted .diagonal elements are decay rates of the products , off - diagonal rates can be both positive or negative , corresponding to activation or inhibition .further , it is clear that first not all products can interact with each other , i.e. a large number of matrix elements will be zero , and second most rates are not available from experiments .we are thus led to model as a random matrix in the following way .using terminology from network theory the degree of product is defined as the number of products that can be regulated by product .the class of interaction networks can now be specified by the degree distribution. there is evidence that protein networks and metabolic networks are scale - free networks , characterized by a degree distribution with average degree , and an exponent . in the following we generate such networks andcontrast them to classical random networks , i.e. erds - renyi graphs ( er ) with the same average degree .if the number of non - zero rates in is denoted by , the average connectivity is .once it is decided which products interact with each other , i.e. , the actual rates have to be fixed .we assume these being gaussian , .this is supported experimentally e.g. by , where a least squares fit of synthetic gene network models to real data indicates that the normal distribution of interaction weights gives the best results .the ( negative ) decay rates we take constant and equal in the following .note that the diagonal component of eq .( [ model ] ) ignoring the positivity constraint immediately reminds at the stochastic differential equation , with and .this langevin process has been exactly solved , the solution ( probability distribution of ) being a -exponential , ^{1/(1-q ) }  \quad ,  \label{tsallis}\ ] ] with , where is the asymptotic power exponent . in fig .1c we show experimental data confirming the power law aspect of mrna concentrations in yeast in the same data . is the difference in gene expression levels between two consecutive measurements ; is the cumulative distribution , for all and . in the same plotwe show results of a numerical simulation of the model eq .( [ model ] ) with the er topology for at , for the two cases : first , and ( gaussian noise model ) , and second and ( multiplicative noise model ) .further a -exponential fit ) is again a -exponential , with a different value , . ] to the data is shown ( broken line ) , with an effective .c +  +  +   biological systems are sufficiently stable , i.e. products are not produced ad infinitum , and at the same time are sufficiently dynamical .if our model is reasonable we have to find the regions in parameter - space where such non - trivial stability is ensured .if we ignore for a moment the positivity condition , eq .( [ poshalfsp ] ) , and the stochastic terms in eq .( [ model ] ) , the stability of the system is dominated by the largest real part of the eigenvalues of the interaction matrix . if there are no non - negative real parts of the eigenvalues , the system will be asymptotically stable .if the distribution of off - diagonal elements in is normal with variance , the eigenvalue spectrum is according to a powerful result from random matrix theory a circle in the complex plane ( girko s circular law ) , .for a fully connected matrix , with non - zero entries in the radius of this circle is equal to the product of the standard deviation and square root of the system size . fornon - fully connected networks , , the radius is given by ( see e.g. ) if the diagonal elements of the random matrix are from a zero - mean distribution , girko s circle is centered at the origin of the complex plane . in our casewe have and the center of the circle will be shifted to the position , see e.g. .now , including the positivity constraint and the stochastic dynamics , it is obvious that the eigenvalue spectrum of will only be a part of the story . to define a measure for system stability that captures these aspects ,maybe the simplest choice is the maximal lyapunov exponent where , is the difference between two trajectories , where results from a small perturbation in the initial condition , .  for the system without the positivity condition the can now be related to in the following way where the is the eigenvalue spectrum shift discussed above .  for the case of the full model i.e. with the positivity condition , we hypothesize the following scenario : with strong noise levels after some time several trajectories diffuse to hit zero . for long enough timesthe expected number of these trajectories will be half of the total number , .this amounts to a reduction of system size by one half ( with fixed connectedness ) , i.e. and . for connectivitythis means , .we would thus expect the asymptotic ( large ) behavior of of the _ full _ model as a function of connectivity for small where no ( or few ) trajectories hit zero , of course , we expect eq .( [ maxlyap ] ) to hold .more generally , it is reasonable to assume that for given connectivity and noise levels , there will be an effective connectivity , where is the number of trajectories not at zero .let us finish with commenting on a potential stabilizing role of multiplicative noise .consider the one dimensional case of our model with and .the evolution of a perturbation thus follows with the solution the lyapunov exponent is proportional to showing that the system can be stable even for positive .we numerically solve the model eq .( [ model ] ) , now with positivity condition and compare with the above predictions . for numerical simulations we generated scale - free ( sf ) and er networks of sizes of . to vary adjusted the number of non - zero rates in the matrix . for scale - free networks we fixed the scaling exponent .all the following results are averages over 20 random realizations of networks for a given parameter set .the lyapunov exponents were computed from datasets of 1000 timepoints , after discarding the initial 200 timesteps . was chosen from the experimental distribution of fig .[ tsallisfig]b .we do not observe noteworthy changes of results when using uniform or gaussian distributions .  in fig .[ wopc ] we show the solution for for the er network , as a function of , with ( triangles ) and without ( circles ) positivity condition .the corresponding theoretical predictions eqs .( [ maxlyap ] ) and ( [ maxlyap2 ] ) , are drawn as broken and solid lines , respectively . the case without positivity conditionis completely explained by theory over the entire range of , eq .( [ maxlyap ] ) . in the case with the constraintthe asymptote follows eq .( [ maxlyap2 ] ) , as expected .it is also seen that for small , eq .( [ maxlyap ] ) is valid .the main finding of this paper is that within a -window between about 10 and 30 a plateau forms where practically vanishes .the constraint dynamics allows a full range of connectivities to support a life at the edge of chaos.  the stability of the system for different network topologies , sizes and various noise components is shown in fig .[ lyapfig ] .+  +  +  figure [ lyapfig]a indicates that both , network size and degree distribution are slightly influencing the width of the plateau .while in the region there is no significant difference in system stability , the low connectivity region shows a size effect on the plateau .the effect of network topology is relatively small , the curve pertaining to sf always being slightly below the er networks , see fig .[ lyapfig]a .while the width of the plateau is always wider for the random distribution of links , in the region , the system is more stable ( smaller ) for sf networks . for higher connectivity regions ( ) the difference between random and scale - free networks becomes indistinguishable due to numerical inaccuracy .figure [ lyapfig]b shows the influence of pure multiplicative ( ) and pure additive noise ( ) on the plateau width , compared to the deterministic process , . with multiplicative noise the plateau becomes significantly wider , while additive noise hardly shows any effect when compared to the deterministic process .plateau widths are collected in tab . 1.  .zero- plateau widths for an er network , with . the width is defined as the region of connectivity where .for the situation of variable multiplicative noise the additive noise was fixed to , for the variable additive noise , .cases for different are shown . [cols="^,^,^,^,^,^,^,^,^,^",options="header " , ]   to understand the formation of the plateau ( ) , one could naively expect eq .( [ maxlyap ] ) to hold at the plateau with the modification that is replaced by from eq .( [ maxlyap3 ] ) .this then would imply in fig .[ lyapfig]c we show that the tail of and does _ not _ follow the naive expectation from eq .( [ expe ] ) , .the key to understanding the formation of the plateau lies in the fact that is not a constant .when , and nodes start becoming inactive , this amounts to effectively reducing the interaction matrix . for each product hitting zero, this means that row and column of matrix can be dropped .( in biological terms , once the concentration of some product type reaches zero , these molecules stop playing a role in the regulation of other products , the effective regulation network consisting just of active nodes ( ) , gets smaller . ) the key observation is that this does not leave the variance of the matrix elements unchanged , but systematically reduces the variance the more nodes become inactive .this can be seen as a selection mechanism in which the most active reactions ( largest reaction rates in ) will hit the boundary first , and will be the first ones to be removed on average .this mechanism drives the system to a self - organized critical state at .we show the situation in fig .[ lyapfig]d ; declines with the number of inactive nodes , as expected . with further increasing number of inactive products saturates at half the network size and eq .( [ maxlyap2 ] ) holds .we have studied the stability of a simple stochastic model of catalytic reaction equations for cellular products such as mrna molecules or proteins .the system is driven by intrinsic molecular noise ( multiplicative ) and external ( additive ) noise .we show that the model captures some essential experimental features , such as the fat tail distribution of concentration changes .imposing an intuitively natural constraint on the system , ( non - negativity of concentrations ) we observe the forming of a plateau of vanishing lyapunov exponents .the dynamical stability of concentrations in catalytic regulatory networks , defined with eq .( [ model ] ) has three extended phases in parameter space ( here connectivity ) . in the first phase the system is asymptotically stable, is negative . after being exposed to a random perturbation in this phase the system always relaxes to its steady state .the main finding of this work is the existence of a second phase , where extends over extended regions .this is in marked contrast to the dynamics of many other non - linear systems , which have only at a set of points .the emergence of this phase can be fully explained within the model . at higher connectivitiessome products will start to reach the boundary .those products with the largest reaction rates will on average hit the boundary first .these are then removed from the system .this means that the variance of the effective reaction rates will get driven downward as a function of connectivity .the variance of rates and the number of active nodes balance each other to exactly arrive at the critical point .this is a self - organized critical effect .a physical analogy is the evaporation at boiling temperature , where molecules of higher - than - average energy leave the liquid first , keeping the ( critical ) temperature at the boiling point .the third phase is defined by where system is dynamically unstable and concentration levels are diverging .we studied the dependence of the plateau on two network topologies , er and sf , where a remarkably small effect was found . we found that with multiplicative noise the size of the plateau can be varied while additive noise showed relatively little effect . for strong enough noise levels of either typethe plateau breaks down .  in was noted that neural networks can perform most complex computations if the dynamics of random threshold gate networks is _ _ a__t the critical boundary between the ordered and chaotic regime .if we interpret gene - regulatory networks as computing devices performing hundreds of optimization problems simultaneously , it is plausible that evolution would have selected among the most efficient variations working at the edge of chaos .

Abstract: as a model for gene and protein interactions we study a set for molecular catalytic reactions . the model is based on experimentally motivated interaction network topologies , and is designed to capture some key statistics of gene expression statistics . we impose a non - linearity to the system by a boundary condition which guarantees non - negative concentrations of chemical concentrations and study the system stability quantified by maximum lyapunov exponents . we find that the non - negativity constraint leads to a drastic inflation of those regions in parameter space where the lyapunov exponent exactly vanishes . we explain the finding as a self - organized critical phenomenon . the robustness of this finding with respect to different network topologies and the role of intrinsic molecular- and external noise is discussed . we argue that systems with inflated edges of chaos could be much more easily favored by natural selection than systems where the lyapunov exponent vanishes only on a parameter set of measure zero .


the use of high - dimensional data has become very popular these days , especially in search , natural language processing ( nlp ) , and computer vision .for example , winner of 2009 pascal image classification challenge used 4 million ( non - binary ) features  . discussed datasets with billions or even trillions of features .  for text data ,the use of extremely high - dimensional representations ( e.g. , -grams ) is the standard practice , for example , .binary representations could be sufficient if the order of -grams is high enough .on the other hand , in current practice of computer vision , it is still more common to use non - binary feature representations , for example , _ local coordinate coding ( lcc ) _it is often the case that in practice high - dimensional non - binary visual features might be appropriately sparsified without hurting the performance .however simply binarizing the features will often incur loss of accuracies , sometimes significantly so .see table [ tab_linearsvm ] for an illustration .our contribution in this paper is the proposal of two types of ( nonlinear ) `` core '' kernels , where `` core '' stands for `` correlation - resemblance '' , for non - binary sparse data .interestingly , using core kernels leads to improvement in classification accuracies ( in some cases significantly so ) on a variety of datasets .  forpractical large - scale applications , naive implementations of nonlinear kernels may be too costly ( time and/or memory ) , while linear learning methods ( e.g. , linear svm or logistic regression ) are extremely popular in industry .the proposed core kernels would be facing the same challenge . to address this critical issue, we also develop efficient hashing algorithms which approximate the core kernels by linear kernels .these new hashing algorithms allow us to take advantage of highly efficient ( batch or stochastic ) linear algorithms , e.g. ,  .in the rest of this section , we first review the definitions of correlation and resemblance , then we provide an experimental study to illustrate the loss of classification accuracies when sparse data are binarized .we assume a data matrix of size , i.e. , observations in dimensions .consider , without loss of generality , two data vectors .the correlation is simply the normalized inner product : it is well - known that constitutes a positive definite and linear kernel , which is one of the reasons why correlation is very popular in practice .   for binary data ,the resemblance is commonly used : it was shown in  that the resemblance defines a type of positive definite kernel . in this study, we will combine correlation and resemblance to define two new types of nonlinear kernels .table [ tab_linearsvm ] lists the datasets , which are non - binary and sparse .the table also presents the test classification accuracies using linear svm on both the original ( non - binary ) data and binarized data .the results in the table illustrate the noticeable drop of accuracies by using only binary data ..classification accuracies ( in % ) using linear svm ( liblinear  ) on sparse non - binary datasets .as we always normalize the data ( to unit norm ) , the correlation kernel is naturally used here .we experiment with the -regularized linear svm and report the best test accuracies from a wide range of `` '' values ( where is the parameter in linear svm ) . using binarized data ( i.e. , the last column ) ,the test accuracies drop quite noticeably in most cases .+ available at the uci repository , * youtube * is a multi - view dataset , and we choose the largest set of features ( audio ) for our experiment .* m - basic * , * m - rotate * , and * mnist10k * were used in  for testing _abc - logitboost _ and _ abc - mart _  ( and comparisons with deep learning  ) . for * rcv1* , we use a subset of the original testing examples ( to facilitate efficient kernel computation later needed in the paper ) . [ cols="<,>,>,>,>,>",options="header " , ]   [ tab_linearsvm ]  figure [ fig_linearsvm ] provides more detailed results for a wide range of values , where is the usual -regularization parameter in svm .  while linear svm is extremely popular in industrial practice , it is often not as accurate .our proposed core kernels will be able to produce noticeably more accurate results than linear svm .we propose two types of core kernels , which combine resemblance with correlation , for sparse non - binary data .both kernels are positive definite. we will demonstrate the effectiveness of the two core kernels using the same datasets in table [ tab_linearsvm ] and figure [ fig_linearsvm ] .the first type of core kernel is basically the product of correlation and the resemblance . later we will express as an ( expectation of ) inner product , i.e. , is obviously positive definite .  if the data are fully dense , then and . on the other hand , if the data are binary , then and .recall the definitions of in ( [ eqn_r ] ) .the second type of core kernel perhaps appears less intuitive than the first type : if the data are binary , then . we will , later in the paper , also write as an expectation of inner product to confirm it is also positive definite.   figure [ fig_kernelsvm ] presents the classification accuracies on the same six datasets as in figure [ fig_linearsvm ] , using nonlinear kernel svm with three different kernels : core type 1 , core type 2 , and resemblance .we can see that resemblance ( which only uses binary information of the data ) does not perform as well as core kernels .[ tab_kernelsvm ]  the best results are summarized in table [ tab_kernelsvm ] .it is interesting to compare them with linear svm results in table [ fig_linearsvm ] and figure [ fig_linearsvm ] .we can see that core kernels perform very well , without using additional tuning parameters .in fact , if we compare the best results in  ( e.g. , rbf svm , abc - boosting , or deep learning ) on mnist10k , m - rotate , and m - basic , we will see that core kernels ( with no tuning parameters ) can achieve the same ( or similar ) performance .  we should mention that our experiments can be fairly easily reproduced because all datasets are public and we use standard svm packages ( libsvm and liblinear ) without any modifications .we also provide the results for wide range of values in figure [ fig_linearsvm ] and figure [ fig_kernelsvm ] .because we use pre - computed kernel functionality of libsvm ( which consumes very substantial amount of memory to store the kernel matrices ) , we only experiment with training data of moderate sizes , to ensure repeatability .* section 1.4.3 ) mentioned three main computational issues of kernels summarized as follows :  1 ._ computing kernels is very expensive _ + computing kernels can account for more than half of the total computing timecomputing the full kernel matrix is wasteful _ + this is because not all pairwise kernel values will be used during training .the kernel matrix does not fit in memory _+ the cost of storing the full kernel matrix in the memory is , which is not realistic for most pcs even for merely , while the industry has used training data with billions of examples .thus , kernel evaluations are often conducted _ on the fly _ , which means the computational cost is dominated by kernel evaluations .  in fact , evaluating kernels on - demand would encounter another serious ( and often common ) issue if the datasets themselves are too big for the memory .all these crucial issues motivate us to develop hashing algorithms to approximate core kernels .our goal is to develop good hashing algorithms to ( approximately ) transform nonlinear kernels into linear kernels .once we have the new data representations ( i.e. , the hashed data ) , we can use highly efficient batch or stochastic linear methods for training svm ( or logistic regression )  .another benefit would be in the context of approximate near neighbor search because probabilistic hashing provides a ( often good ) strategy for space partitioning which will help reduce the search time ( i.e. , no need to scan all data points ) .our proposed hashing methods can be modified to become an instance of _ locality sensitive hashing ( lsh ) _ in the space of core kernels .  in this study , we will present hashing algorithms for core kernels based on standard random projection and minwise hashing methods .we first provide a review of these two methods .consider two vectors .the idea of random projection is simple .we first generate a random vector of i.i.d .entries , to , and then compute the inner products as the hashed values : for the convenience of theoretical analysis , we adopt , which is a typical choice in the literature .several variants of random projections like  are essentially equivalent , as analyzed in  .we always assume the data are normalized , i.e. , .note that computing the norms of all the data points only requires scanning the data once which is anyway needed during data collection / processing . for normalized data , it is known that = \rho ] and .then the permuted vector becomes  ] .this way , we can write as an inner product of two -dimensional sparse vectors .  note that the input data format of standard svm packages is the sparse format . for linear svm, the cost is essentially determined by the number of nonzeros ( in this case , ) , not much to do with the dimensionality ( unless it is too high ) .if is too high , then we can adopt the standard trick of _ -bit minwise hashing _ by only using the lowest bits of .our second proposal is recall that we always assume the data ( , ) are normalized .for example , if the data are binary , then we have , .hence the values and are small ( and we need the term ) .  [ thm_kc2 ]  * proof : * see appendix [ app_thm_kc2].  once we understand how to express as an inner product , it should be easy to see that can also be written as an inner product . again , suppose , , and .we can consider an equivalent vector  ] if , or  ] if , or  ] if or  ] .therefore , the variance is   this completes the proof .we need the first two moments of the estimator   because \\\notag = & e\left[v_j(u ) v_j(v)1\{l_j(u ) = l_j(v)\}|l_j(u)=l_j(v)\right]\\\notag & \times\mathbf{pr}\left(l_j(u)=l_j(v)\right)\\\notag = & \frac{\sum_{i=1}^d u_iv_i}{a}r = \rho \frac{1}{f_1+f_2-a}\end{aligned}\ ] ] we know and \\\notag = & e\left[v_j^2(u ) v_j^2(v)\right]\mathbf{pr}\left(l_j(u)=l_j(v)\right)\\\notag = & \frac{\sum_{i=1}^d u_i^2v_i^2}{a}r = \frac{\sum_{i=1}^d u_i^2v_i^2}{f_1+f_2-a}\end{aligned}\ ] ] therefore ,                    h. larochelle , d. erhan , a. c. courville , j. bergstra , and y. bengio . an empirical evaluation of deep architectures on problems with many factors of variation . in _icml _ , pages 473480 , corvalis , oregon , 2007 .

Abstract: the term `` core kernel '' stands for _ correlation - resemblance kernel_. in many applications ( e.g. , vision ) , the data are often high - dimensional , sparse , and non - binary . we propose two types of ( nonlinear ) core kernels for non - binary sparse data and demonstrate the effectiveness of the new kernels through a classification experiment . core kernels are simple with no tuning parameters . however , training nonlinear kernel svm can be ( very ) costly in time and memory and may not be suitable for truly large - scale industrial applications ( e.g. search ) . in order to make the proposed core kernels more practical , we develop basic probabilistic hashing algorithms which transform nonlinear kernels into linear kernels .


it is expected that stellar core collapses of supernova will radiate the burst gravitational waves , which frequency is suitable to detect with ground - based gravitational wave detectors .the predicted burst gravitational waves have a some peculiar characteristics : type i ) big narrow spike of msec and following 10 - 15 msec decay ringing , type ii ) a few msec spikes iterate a few times with msec interval , type iii ) broad excess of msec with fine structure of msec .the predicted burst gravitational waves have a duration and tail of signals .since the tail structure and frequency spread is not same as popular spike noises which caused by the detector instruments , it will be possible to differentiate the gravitational waves from noises .however , it is hard to predict the gravitational wave forms in analytically or hard to fit with a few parameters .  to identify such a signal ,we have developed a new method by examining clustering of signal powers in time - frequency domain .we study using tama observation data and numerical predicted gravitational waveforms by dimmelmeier at al.  in some other time - frequency domain analysis , which especially use cluster , were studied as reference , to find efficient power filter window .our practical issue is employment much more parameters and find efficient combinations of parameters .tama300 is a 300 m baseline laser interferometric gravitational wave detector in mitaka , tokyo , japan .the interferometer is fabry - perot - michelson type with a power recycling technique .the strain sensitivity of the detector is } ] +  +  + & distance from center to the to the peak & + & standard deviation around & + & kurtosis around power weighted center & +  +  + & distance from weighted center to the to the peak & + & standard deviation around & +  +  + & & +  + & & +  [ default ]   scatter plot of cluster parameters - + red cross points is noises in real data .others are injected gravitational waves ( dfm waveforms ) in simulation data.,width=377 ]   scatter plot of cluster parameters - + cluster due to gaussian noise fluctuation is spread uniformly in this figure.,width=377 ]   with the criteria in the table [ table : criteria ] , we estimate the efficiency by using monte - carlo simulation with random injection of the dfm gravitational waves in tama data . the figure fig : distvseff shows the efficiency for tf - clusters of source distance from the earth .the efficiency depend on the type of burst waveform .also the figure fig : hrssvseff shows the efficiency for : root square sum of strain amplitude . the selection criteria for clusters which peak snr achieved 50% average efficiency for injected dfm waveforms at 350 pc ( ) for type i burst , and at 170 pc for all average .we take note that the efficiency looks higher than previous our analysis , because we set lower snr threshold for the comparison of clusters .if we wish to derive better upper limit of event search , we must set more higher threshold of snr to reject small spurious noise .tf - cluster selection efficiency for injected dfm waveforms ( source distance),width=377 ]   tf - cluster selection efficiency for injected dfm waveforms ( ),width=377 ]   tf - cluster selection criteria can reject typical noises efficiently in real data . using a part of tama dt9 data ( ) , we compare the snr histogram of peak before and after cluster parameter selection . the reduction for spurious event clusters by spike noisesare more than one order improvement for the snr .the reduction is efficient for higher snr clusters for snr . because that tf - cluster method is efficient when the waveform is clear .histogram of snr for real data + histogram of cluster snr is shown for a part of tama dt9 data .green dashed line is all found cluster in tf domain .red line is after the selection by criteria in the table [ table : criteria].,width=377 ]we recently practice the newer search algorithm as tf - cluster using many cluster parameters .the method is efficient to select gravitational event and reject noise fake , especially in larger snr case .since the tf - cluster method is not require the exact template of wave form , it is useful to identify the burst gravitational waves which we believe only the typical characteristics is knows , or which predicted numerically without analytic waveform .  in the sonogram of this analysis, we employed sfft , which has uniform time and frequency resolution .however , this is not optimal time - frequency resolution in an uncertainty principle of short wave packet . according to the complemental relation between the time and the frequency, we can improve time resolution in high frequency band , can improve frequency resolution in lower frequency band instead of time resolution .we began to employ with wavelet transform to achieve such resolution and to make more clear appearance of burst gravitational waves .this research is supported in part by a grant - in - aid for scientific research on priority areas ( 415 ) of the ministry of education , culture , sports , science and technology .9 h.dimmelmeier , j.a.font and e.muller , astron .astrophys . , 388 , 917 - 935 ( 2002 ) h.dimmelmeier , j.a.font and e.muller , astron . astrophys . , 393 , 523 - 542 ( 2002 ) r.honda , master thesis , 2007 julien sylvestre , phys .d 66 ( 2002 ) 102004 m.ando , et al.,physical review letters , volume 86 , 3950 - 3954 , ( 2001 ) t.akutsu et al ., physical review d 74 , 122002 ( 2006 ) m.ando , et al . , physical review d 71 , 082002 ( 2005 ) http://lappweb.in2p3.fr/virgo/framel/ tomomi akutsu , et al .quantum grav .23 ( 2006 ) s715

Abstract: we have developed a method time - frequency ( tf ) clustering to find the burst gravitational waves for tama data analysis . tf clustering method on sonogram ( spectrogram ) shows some characteristics of short duration signal . using parameters which represent the cluster shape , we can efficiently identify some predicted gravitational wave forms and can exclude typical unstable spike like noises due to detector instruments . the requirement of some parameters of cluster achieved roughly 50% average efficiency for injected dfm waveforms of for type i burst . also the reduction for signal by spike noises are more than one order improvement for the snr .