Dynamical systems are those whose evolution can be described by a rule, evolves with time and is deterministic. In this context can I say that Neural networks have a rule of evolution which is the activation function $f(\text{sum of product of weights and features})$ ? Are neural networks dynamical systems, linear or nonlinear dynamical systems? Can somebody please shed some light on this?
Are neural networks dynamical systems?
terminology;artificial intelligence;neural networks
A particular neural network does not evolve with time.  Its weights are fixed, so it defines a fixed, deterministic function from the input space to the output space.The weights are typically derived through a training process (e.g., backpropagation).  One could imagine building a system that periodically re-applies the training process to generate new weights every so often.  Such a system would indeed evolve over time.  However, it would be more accurate to call this a system that includes a neural network as one component of it.Anyway, at this point we are probably descending into quibbling over terminology, which might not be very productive.  This site format is a better fit for objectively answerable questions with some substantive technical content.