MS&E351 Tentative Syllabus

  1. Markov decision processes

    1. Total cost

    2. Discounted cost

    3. Average cost

  2. Dynamic programming algorithms

    1. Value iteration

    2. Parallel/asynchronous variants

    3. Policy iteration

    4. Linear programming

  3. Problem-specific ideas

    1. Linear systems with quadratic cost

    2. Inventory control

    3. Portfolio management

    4. Interchange argument

    5. Queueing systems

    6. Multi-armed bandits

  4. Imperfect state information

    1. Reduction to the basic problem

    2. Sufficient statistics

    3. Separation principle

    4. POMDPs

  5. Further directions

    1. Approximate dynamic programming

    2. Reinforcement learning