Planning in MDPs

---

Table of contents

• 1. Introductions
• 2. The Fundamental Theorem
• 3. Value Iteration and Our First Lower Bound
• 4. Policy Iteration
• 5. Local Planning - Part I.
• 6. Local Planning - Part II.
• 7. Function Approximation
• 8. Approximate Policy Iteration
• 9. Limits of query-efficient planning
• 10. Planning under $q^*$ realizability
• 11. Planning under $v^*$ realizability (TensorPlan I.)
• 12. TensorPlan and eluder sequences
• 13. From API to Politex
• 14. Politex
• 15. From policy search to policy gradients
• 16. Policy gradients

---