Week - 1 |
Dynamic Programming Algorithm: Introduction to Sequential Decision Making |
Week - 2 |
Problem Formulation, Deterministic and Stochastic Dynamic Programming |
Week - 3 |
Deterministic Dynamic Programming: Forward and Backward Recursion Algorithms, Shortest Path Problems |
Week - 4 |
Markov Decision Processes |
Week - 5 |
Problems with Perfect State Information |
Week - 6 |
Structural Analysis for the Optimal Solutions of Dynamic Programming |
Week - 7 |
Dynamic Programming Applications: Inventory Control, Dynamic Portfolio Analysis, Optimal Stopping Problems |
Week - 8 |
Problems with Imperfect State Information |
Week - 9 |
Reduction to Perfect State Info, Sufficient Statistics, Bayesian Models |
Week - 10 |
Infinite Horizon Problems Stochastic Shortest Path, Bellman's Equation |
Week - 11 |
Discounted Cost and Average Cost Analysis, Value and Policy Iteration Methods |
Week - 12 |
Continuous Time Problems |
Week - 13 |
Uniformization, Queueing Examples |
Week - 14 |
Presentations of term projects |