STAT424 | Optimization for Statistical Learning
Semester 2/2025 [Syllabus]
Time: Tu/F 1:00-2:30pm at STB205
Instructor
-
Donlapark Ponnoprat
Office: STB304
Grading
- Homework 30%
- Midterm 35%. TBA
- Final 35%. FRI MAR 20, 2026 15:30 - 18:30
References
- Amir Ali Ahmadi, Computing and Optimization (ORF 363) Lecture Notes.
- Boyd and Vandenberghe, Convex Optimization (2014) book.
- David Childers, Forecasting for Economics and Business (73-423) Lecture Notes.
- Julian McAuley, Personalized Machine Learning (2022) book draft.
We will follow my Lecture Notes (PDF) throughout the course.
| Date | Topic | Colab | Homework |
|---|---|---|---|
| Week 1 | Calculus Review & Matrix Calculus for Optimization |
numpy and jax gradient and Hessian |
|
| Week 2 | Introduction to Optimization & Linear Programming | ||
| Week 3 | Duality and Its Interpretations & Introduction to Convexity | ||
| Week 4 | Convex Optimization Problems in Statistics | ||
| Week 5 | The Lagrangian and KKT Conditions | ||
| Week 6 | Mixed-Integer Programming | ||
| Week 7 | Break Week (No Class) | ||
| Week 8 | Midterm Exam | ||
| Week 9 | Gradient Descent | ||
| Week 10 | Momentum, Accelerated, and Adaptive Gradient Methods | ||
| Week 11 | Projected, Stochastic, and Mirror Descent | ||
| Week 12 | Second-Order Optimization & Newton's Method | ||
| Week 13 | Application: Matrix Factorization and Recommender Systems | ||
| Week 14 | Variational Inference & Optimal Transport | ||
| Week 15 | Sampling as Optimization & Generative Modeling | ||
| Week 16 | Final Exam | ||