Short course on first-order optimization methods
This is a short course on introducing first-order optimization methods, including proximal gradient descent, primal-dual splitting methods, alternating direction method of multipliers and Douglas-Rachford splitting methods. Some well-known accelerated algorithms are also introduced.
- Mathematical background
- Proximal Gradient method
- Primal-Dual splitting method
- ADMM and Douglas-Rachford splitting method
- R. T. Rockafellar. Convex analysis. Princeton university press, 2015.
- A. Beck. First-order methods in optimization. Vol. 25. SIAM, 2017.
- H. H. Bauschke and P. L. Combettes. Convex analysis and monotone operator theory in Hilbert spaces. Vol. 408. New York: Springer, 2011.
- B. Polyak. Introduction to optimization. Optimization Software, 1987.
- Y. Nesterov. Introductory lectures on convex optimization: A basic course. Vol. 87. Springer Science & Business Media, 2013.