Autonomous optimization refers to the design of feedback controllers that steer a physical system to a steady state that solves a predefined, possibly constrained, optimization problem. As such, no exogenous control inputs such as setpoints or trajectories are required. Instead, these controllers are modeled after optimization algorithms that take the form of dynamical systems. The interconnection of this type of optimization dynamics with a physical system is however not guaranteed to be stable unless both dynamics act on sufficiently different timescales. In this paper, we quantify the required timescale separation and give prescriptions that can be directly used in the design of this type of feedback controllers. Using ideas from singular perturbation analysis we derive stability bounds for different feedback optimization schemes that are based on common continuous-time optimization schemes. In particular, we consider gradient descent and its variations, including projected gradient, and Newton gradient. We further give stability bounds for momentum methods and saddle-point flows interconnected with dynamical systems. Finally, we discuss how optimization algorithms like subgradient and accelerated gradient descent, while well-behaved in offline settings, are unsuitable for autonomous optimization due to their general lack of robustness.
 Extremum seeking feedback is a powerful method to steer a dynamical system to an extremum of a partially or completely unknown map. It often requires advanced system-theoretic tools to understand the qualitative behavior of extremum seeking systems. In this paper, a novel interpretation of extremum seeking is introduced. We show that the trajectories of an extremum seeking system can be approximated by the trajectories of a system which involves certain Lie brackets of the vector fields of the extremum seeking system. It turns out that the Lie bracket system directly reveals the optimizing behavior of the extremum seeking system. Furthermore, we establish a theoretical foundation and prove that uniform asymptotic stability of the Lie bracket system implies practical uniform asymptotic stability of the corresponding extremum seeking system. We use the established results in order to prove local and semi-global practical uniform asymptotic stability of the extrema of a certain map for multi-agent extremum seeking systems. A novel class of derivative-free optimization algorithms is developed. The main idea is to utilize certain non-commutative maps in order to approximate the gradient of the objective function. Convergence properties of the novel algorithms are established and simulation examples are presented.