Generalizing Nesterov’s Acceleration Framework by Embedding Momentum Into Estimating Sequences: New Algorithm and Bounds

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

2 Citations (Scopus)

Abstract

We present a new type of heavy-ball momentum term, which is used to construct a class of generalized estimating sequences. These allow for accelerating the minimization process by exploiting the information accumulated in the previous iterates. Combining a newly introduced momentum term with the estimating sequences framework, we devise, as an example, a new black-box accelerated first-order method for solving smooth unconstrained optimization problems. We prove that the proposed method exhibits an improvement over the rate of the celebrated fast gradient method by at least a factor of 1/√ 2, and establish that lower bound on the number of iterations carried through until convergence is O (√ κ/2). Finally, the practical performance benefits of the proposed method are demonstrated by numerical experiments.

Original languageEnglish
Title of host publication2022 IEEE International Symposium on Information Theory (ISIT)
PublisherIEEE
Pages1506-1511
Number of pages6
ISBN (Electronic)978-1-6654-2159-1
DOIs
Publication statusPublished - 2022
MoE publication typeA4 Conference publication
EventIEEE International Symposium on Information Theory - Espoo, Finland
Duration: 26 Jun 20221 Jul 2022

Publication series

NameIEEE International Symposium on Information Theory
ISSN (Electronic)2157-8117

Conference

ConferenceIEEE International Symposium on Information Theory
Abbreviated titleISIT
Country/TerritoryFinland
CityEspoo
Period26/06/202201/07/2022

Keywords

  • Gradient methods
  • Minimization
  • Optimization
  • Information theory
  • Convergence
  • estimating sequences
  • black-box methods
  • complexity analysis
  • optimization

Fingerprint

Dive into the research topics of 'Generalizing Nesterov’s Acceleration Framework by Embedding Momentum Into Estimating Sequences: New Algorithm and Bounds'. Together they form a unique fingerprint.

Cite this