Almost-Sure Convergence of Adaptive Algorithms by Projections

In this note we offer a new convergence proof for the normed least mean square algorithm for ergodic inputs. Our approach is based on interpreting the algorithm as a sequence of relaxed projection operators1by which the key contraction property is derived. Although the result is not new, the proof technique is strongly motivated by physical intuition, and our hope is that it will provide additional insight into the LMS type algorithms under ergodic inputs. Embedded in the development is a slight generalization to a random time-varying gain parameter. This allows the incorporation of variations such as the LMS and signed LMS algorithms.