Takens' Embedding Theorem remarkably established that concatenating M previous outputs of a dynamical system into a vector (called a delay coordinate map) can be a one-to-one mapping of a low-dimensional attractor from the system state space. However, Takens' theorem is fragile in the sense that even small imperfections can induce arbitrarily large errors in this attractor representation. We extend Takens' result to establish deterministic, explicit and non-asymptotic sufficient conditions for a delay coordinate map to form a stable embedding in the restricted case of linear dynamical systems and observation functions. Our work is inspired by the field of Compressive Sensing (CS), where results guarantee that low-dimensional signal families can be robustly reconstructed if they are stably embedded by a measurement operator. However, in contrast to typical CS results, i) our sufficient conditions are independent of the size of the ambient state space, and ii) some system and measurement pairs have fundamental limits on the conditioning of the embedding (i.e., how close it is to an isometry), meaning that further measurements beyond some point add no further significant value. We use several simple simulations to explore the conditions of the main results, including the tightness of the bounds and the convergence speed of the stable embedding. We also present an example task of estimating the attractor dimension from time-series data to highlight the value of stable embeddings over traditional Takens' embeddings.
Given the system matrix @math and the definition of a dynamical system , knowing the state at some fixed time @math is equivalent to knowing the path that the system takes to and from that state (called the ). Classic results in linear systems theory @cite_19 show that the explicit solution for this path is given by a matrix multiplication: @math , where @math is the . Note that this solution is valid for positive or negative values of @math , describing the flow both forward and backward from time @math .
@cite_19
2133776435 2045107949 2130154921 2750924312
Abstract We establish a number of properties associated with the dynamical system H = [H,[H, N]] , where H and N are symmetric n by n matrices and [ A , B ] = AB − BA . The most important of these come from the fact that this equation is equivalent to a certain gradient flow on the space of orthogonal matrices. We are especially interested in the role of this equation as an analog computer. For example, we show how to map the data associated with a linear programming problem into H (0) and N in such a way as to have H = [H[H, N]] evolve to a solution of the linear programming problem. This result can be applied to find systems which solve a variety of genetic combinatorial optimization problems, and it even provides an algorithm for diagonalizing symmetric matrices. We present algorithms for solving symmetric, diagonally-dominant linear systems to accuracy e in time linear in their number of non-zeros and log (κ f (A) e), where κ f (A) is the condition number of the matrix defining the linear system. Our algorithm applies the preconditioned Chebyshev iteration with preconditioners designed using nearly-linear time algorithms for graph sparsification and graph partitioning. We deal with nonlinear dynamical systems, consisting of a linear nominal part perturbed by model uncertainties, nonlinearities and both additive and multiplicative random noise, modeled as a Wiener process. In particular, we study the problem of finding suitable measurement feedback control laws such that the resulting closed-loop system is stable in some probabilistic sense. To this aim, we introduce a new notion of stabilization in probability, which is the natural counterpart of the classical concept of regional stabilization for deterministic nonlinear dynamical systems and stands as an intermediate notion between local and global stabilization in probability. This notion requires that, given a target set, a trajectory, starting from some compact region of the state space containing the target, remains forever inside some larger compact set, eventually enters any given neighborhood of the target in finite time and remains thereinafter, all these events being guaranteed with some probability. We give a Lyapunov-based sufficient condition for achieving stability in probability and a separation result which splits the control design into a state feedback problem and a filtering problem. Finally, we point out constructive procedures for solving the state feedback and filtering problem with arbitrarily large region of attraction and arbitrarily small target for a wide class of nonlinear systems, which at least include feedback linearizable systems. The generality of the result is promising for applications to other classes of stochastic nonlinear systems. In the deterministic case, our results recover classical stabilization results for nonlinear systems. In this paper, we use dynamical system to analyze the nonlinear weight dynamics of two-layered bias-free networks in the form of @math , where @math is ReLU nonlinearity. We assume that the input @math follow Gaussian distribution. The network is trained using gradient descent to mimic the output of a teacher network of the same size with fixed parameters @math using @math loss. We first show that when @math , the nonlinear dynamics can be written in close form, and converges to @math with at least @math probability, if random weight initializations of proper standard derivation ( @math ) is used, verifying empirical practice. For networks with many ReLU nodes ( @math ), we apply our close form dynamics and prove that when the teacher parameters @math forms orthonormal bases, (1) a symmetric weight initialization yields a convergence to a saddle point and (2) a certain symmetry-breaking weight initialization yields global convergence to @math without local minima. To our knowledge, this is the first proof that shows global convergence in nonlinear neural network without unrealistic assumptions on the independence of ReLU activations. In addition, we also give a concise gradient update formulation for a multilayer ReLU network when it follows a teacher of the same size with @math loss. Simulations verify our theoretical analysis.