Adaptive Dynamic Programming-Regulated Extremum Seeking for Distributed Feedback Optimization

This note studies the distributed feedback optimization for linear multi-agent systems without precise knowledge of cost functions and agent dynamics. The goal is to regulate the outputs of the agents toward an unknown minimizer of a sum of local costs. To achieve this, distributed reference signals are combined with an extremum seeking mechanism to search for the minimizer. Meanwhile, each agent steers its output toward the designed reference signal using a learning-based adaptive optimal tracker. The entire process relies only on measurements of local costs and input-state data along the agents' trajectories. Moreover, the overall feedback loop has three time scales: tracking and consensus of the reference signals are the fastest, periodic sinusoidal perturbation is the medium, and optimization of the global cost is the slowest. Through this time-scale separation, the closed-loop system is guaranteed to be practically exponentially stable at an equilibrium of interest, along with the convergence of the output of each agent to a small neighborhood of the desired minimizer. A numerical example of robotic networks demonstrates the efficacy of the proposed method.