Greedy Criterion in Orthogonal Greedy Learning

Lin Xu, Shaobo Lin, Jinshan Zeng, Xia Liu, Yi Fang, Zongben Xu

Research output: Contribution to journalArticlepeer-review

Abstract

Orthogonal greedy learning (OGL) is a stepwise learning scheme that starts with selecting a new atom from a specified dictionary via the steepest gradient descent (SGD) and then builds the estimator through orthogonal projection. In this paper, we found that SGD is not the unique greedy criterion and introduced a new greedy criterion, called as ' δ-greedy threshold' for learning. Based on this new greedy criterion, we derived a straightforward termination rule for OGL. Our theoretical study shows that the new learning scheme can achieve the existing (almost) optimal learning rate of OGL. Numerical experiments are also provided to support that this new scheme can achieve almost optimal generalization performance while requiring less computation than OGL.

Original languageEnglish (US)
Article number7862802
Pages (from-to)955-966
Number of pages12
JournalIEEE Transactions on Cybernetics
Volume48
Issue number3
DOIs
StatePublished - Mar 2018

Keywords

  • Generalization performance
  • greedy algorithms
  • greedy criterion
  • orthogonal greedy learning (OGL)
  • supervised learning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Greedy Criterion in Orthogonal Greedy Learning'. Together they form a unique fingerprint.

Cite this