Course description

This is a short course on introducing first-order optimization methods, including proximal gradient descent, primal-dual splitting methods, alternating direction method of multipliers and Douglas-Rachford splitting methods. Some well-known accelerated algorithms are also introduced.

Lecture slides

1. Mathematical background
2. Motivations
3. Proximal gradient method
4. Primal-Dual splitting method
5. ADMM and Douglas-Rachford splitting method
6. Acceleration

References

• R. T. Rockafellar. Convex Analysis. Princeton University Press, 2015.
• A. Beck. First-order Methods in Optimization. Vol. 25. SIAM, 2017.
• H. H. Bauschke and P. L. Combettes. Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Vol. 408. New York: Springer, 2011.
• B. Polyak. Introduction to Optimization. Optimization Software, 1987.
• Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Vol. 87. Springer Science Business Media, 2013.