Short course on first-order optimization methods

Description

This is a short course on introducing first-order optimization methods, including proximal gradient descent, primal-dual splitting methods, alternating direction method of multipliers and Douglas-Rachford splitting methods. Some well-known accelerated algorithms are also introduced.

Lecture slides

  1. Mathematical background
  2. Motivations
  3. Proximal Gradient method
  4. Primal-Dual splitting method
  5. ADMM and Douglas-Rachford splitting method
  6. Acceleration

References

  • R. T. Rockafellar. Convex analysis. Princeton university press, 2015.
  • A. Beck. First-order methods in optimization. Vol. 25. SIAM, 2017.
  • H. H. Bauschke and P. L. Combettes. Convex analysis and monotone operator theory in Hilbert spaces. Vol. 408. New York: Springer, 2011.
  • B. Polyak. Introduction to optimization. Optimization Software, 1987.
  • Y. Nesterov. Introductory lectures on convex optimization: A basic course. Vol. 87. Springer Science & Business Media, 2013.