A robust gradient sampling algorithm for nonsmooth, nonconvex optimization

James V. Burke, Adrian S. Lewis, Michael L. Overton

Research output: Contribution to journalArticlepeer-review

Abstract

Let f be a continuous function on R n, and suppose f is continuously differentiable on an open dense subset. Such functions arise in many applications, and very often minimizers are points at which f is not differentiable. Of particular interest is the case where f is not convex, and perhaps not even locally Lipschitz, but is a function whose gradient is easily computed where it is defined. We present a practical, robust algorithm to locally minimize such functions, based on gradient sampling. No subgradient information is required by the algorithm. When f is locally Lipschitz and has bounded level sets, and the sampling radius ε is fixed, we show that, with probability 1, the algorithm generates a sequence with a cluster point that is Clarke ε-stationary. Furthermore, we show that if f has a unique Clarke stationary point x̄, then the set of all cluster points generated by the algorithm converges to x̄ as ε is reduced to zero. Numerical results are presented demonstrating the robustness of the algorithm and its applicability in a wide variety of contexts, including cases where f is not locally Lipschitz at minimizers. We report approximate local minimizers for functions in the applications literature which have not, to our knowledge, been obtained previously. When the termination criteria of the algorithm are satisfied, a precise statement about nearness to Clarke ε-stationarity is available. A MATLAB implementation of the algorithm is posted at http://www.cs.nyu.edu/overton/papers/gradsamp/alg.

Original languageEnglish (US)
Pages (from-to)751-779
Number of pages29
JournalSIAM Journal on Optimization
Volume15
Issue number3
DOIs
StatePublished - 2005

Keywords

  • Generalized gradient
  • Gradient sampling
  • Nonconvex
  • Nonsmooth optimization
  • Subgradient

ASJC Scopus subject areas

  • Software
  • Theoretical Computer Science

Fingerprint

Dive into the research topics of 'A robust gradient sampling algorithm for nonsmooth, nonconvex optimization'. Together they form a unique fingerprint.

Cite this