Nonsmooth optimization via quasi-Newton methods

Adrian S. Lewis, Michael L. Overton

Research output: Contribution to journalArticlepeer-review

Abstract

We investigate the behavior of quasi-Newton algorithms applied to minimize a nonsmooth function f, not necessarily convex. We introduce an inexact line search that generates a sequence of nested intervals containing a set of points of nonzero measure that satisfy the Armijo and Wolfe conditions if f is absolutely continuous along the line. Furthermore, the line search is guaranteed to terminate if f is semi-algebraic. It seems quite difficult to establish a convergence theorem for quasi-Newton methods applied to such general classes of functions, so we give a careful analysis of a special but illuminating case, the Euclidean norm, in one variable using the inexact line search and in two variables assuming that the line search is exact. In practice, we find that when f is locally Lipschitz and semi-algebraic with bounded sublevel sets, the BFGS (Broyden-Fletcher-Goldfarb-Shanno) method with the inexact line search almost always generates sequences whose cluster points are Clarke stationary and with function values converging R-linearly to a Clarke stationary value. We give references documenting the successful use of BFGS in a variety of nonsmooth applications, particularly the design of low-order controllers for linear dynamical systems. We conclude with a challenging open question.

Original languageEnglish (US)
Pages (from-to)135-163
Number of pages29
JournalMathematical Programming
Volume141
Issue number1-2
DOIs
StatePublished - Oct 2013

Keywords

  • BFGS
  • Clarke stationary
  • Line search
  • Nonconvex
  • Partly smooth
  • R-linear convergence

ASJC Scopus subject areas

  • Software
  • Mathematics(all)

Fingerprint

Dive into the research topics of 'Nonsmooth optimization via quasi-Newton methods'. Together they form a unique fingerprint.

Cite this