Issues in large-scale optimization with expensive functions

Continuous optimization problems of all sizes involve many of the same generic algorithmic paradigms, such as replacement of a difficult problem by a sequence of related but easier sub-problems. As optimization problems increase in size, however, certain standard techniques for smaller problems become inappropriate because of excessive requirements of time and/or storage. Furthermore, some approaches regarded as uncompetitive for small problems re-emerge as useful alternatives when problems become sufficiently large. Specialized high-performance computer architectures complicate the picture even more with respect to both viable linear algebraic sub-problems and algorithmic strategies. This paper will briefly discuss selected topics in large-scale optimization, with particular stress on issues associated with very expensive function evaluations.