Tarun Rambha
×
About Me
Introduction
Contact
Research
Projects
Group
Publications
Join Us
Teaching
Traffic Network Equilibrium
Markov Decision Processes
Mathematics for Engineers
Traffic Engineering
Transportation Logistics
Past Courses
Open-Source Software
Under Construction
Miscellaneous
Under Construction

Markov Decision Processes

Course Syllabus
October – January 2020
Timings: Tue and Thu 9:30 AM – 11:00 AM
Location: Online

This page contains lecture slides and assignments for ‘CE 273 Markov Decision Processes’. Codes for the programming project and LaTeX resources will also be posted here.

Lectures

  1. Introduction

  2. Transient Behavior and Classification of States

  3. Limiting Behavior of DTMCs

  4. Finite Horizon MDPs

  5. Applications of Finite Horizon MDPs - Part I

  6. Applications of Finite Horizon MDPs - Part II

  7. Infinite Horizon Discounted MDPs

  8. Value Iteration

  9. Policy Iteration

  10. Linear Programming Methods

  11. Infinite Horizon Total Cost MDPs

  12. Solution Methods for Total Cost MDPs

  13. Infinite Horizon Average Cost MDPs

  14. Optimality Conditions and Classification of Average Cost MDPs

  15. Value Iteration for Average Cost MDPs

  16. Policy Iteration and Linear Programming for Average Cost MDPs

  17. Introduction to Approximate Dynamic Programming

  18. Approximation in Value Space - Part I

  19. Approximation in Value Space - Part II

  20. Q-learning and Approximate Linear Programming

  21. Approximation in Policy Space

  22. Dynamic Discrete Choice Models

  23. Inverse Reinforcement Learning

  24. Semi-Markov Decision Processes

  25. POMDPs and Risk-Sensitive MDPs

  26. Continuous-Time Control

Resources

LaTeX

https://en.wikibooks.org/wiki/LaTeX
https://www.overleaf.com/

Python Toolbox

http://pymdptoolbox.readthedocs.io/en/latest/api/mdptoolbox.html

Last edited on February 11th 2025 10:43PM (Time Zone: IST).
Powered by jemdoc + new design.