Skip to main navigation Skip to search Skip to main content

SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives

  • Aaron Defazio
  • , Francis Bach
  • , Simon Lacoste-Julien

    Research output: Contribution to journal › Conference article › peer-review

    1431 Citations (Scopus)

    Abstract

    In this work we introduce a new optimisation method called SAGA in the spirit of SAG, SDCA, MISO and SVRG, a set of recently proposed incremental gradient algorithms with fast linear convergence rates. SAGA improves on the theory behind SAG and SVRG, with better theoretical convergence rates, and has support for composite objectives where a proximal operator is used on the regulariser. Unlike SDCA, SAGA supports non-strongly convex problems directly, and is adaptive to any inherent strong convexity of the problem. We give experimental results showing the effectiveness of our method.

    Original languageEnglish
    Pages (from-to)1646-1654
    Number of pages9
    JournalAdvances in Neural Information Processing Systems
    Volume2
    Issue numberJanuary
    Publication statusPublished - 2014
    Event28th Annual Conference on Neural Information Processing Systems 2014, NIPS 2014 - Montreal, Canada
    Duration: 8 Dec 2014 → 13 Dec 2014

    Fingerprint

    Dive into the research topics of 'SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives'. Together they form a unique fingerprint.

    Cite this