Reading list

This list is horribly out of date — will be updated soon 🙂

  1. Amir Beck, /First-Order Methods in Optimization/. A really well-written text bringing together a wealth of material on fundamental optimization theory and first-order convex optimization algorithms.
  2. Nemirovski's recent update to his classical /Lectures on Modern Convex Optimization/ (PDF). The first three chapters are the natural next step after working through the MOSEK modeling cookbook.
  3. An Introduction to Optimization on Smooth Manifolds by Nicolas Boumal (current maintainer of PyManOPT).
  4. Computational Optimal Transport — a well-written introduction to OT for mathematically inclined readers, co-authored by Marco Cuturi.
  5. Semidefinite approximations of the matrix logarithm — Fawzi, Saunderson, Parrilo. My evolving commentary lives on the blog.
  6. Distributional Reinforcement Learning with Quantile Regression. For a comprehensive introduction to distributional RL, see here.
  7. Practical Near Neighbour Search via Group Testing — clever use of Distance-Sensitive Bloom Filters coupled with ideas from group testing. Outperforms FAISS by large factors.
  8. /Numerical Linear Algebra/ by Trefethen and Bau. The best self-contained introduction to NLA there is — a good way to consolidate before graduating to Matrix Computations as a reference.
  9. Introduction to Online Convex Optimization — Elad Hazan's text. Online learning is fascinating and I want to dig in properly.
  10. Convex Optimization: Algorithms and Complexity — a beautiful monograph on the algorithmics of convex optimization.
  11. Non-Convex Optimization for Machine Learning — Prateek Jain's monograph on broad ideas in non-convex optimization.
  12. Tengyu Ma's StatML notes — great companion to Shai Shalev-Shwartz's learning-theory text, with material on NTK ideas.
  13. /A User's Guide to Measure-Theoretic Probability Theory/ by David Pollard. An amazing self-contained tour of MTPT — probably sufficient for a wannabe applied mathematician.