All the lecture notes are linked to this web page, in PDF and postscript format. For convenience of printing, slides are also available in a 4 per page format (the links labeled PS4 and PDF4). The reader for PDF files is available free from Adobe for UNIX, Apple Macintosh, and Windows. You can view both postscript and PDF files using ghostview.
| Introduction | Pearl, Chapter 1 Slides: PS, PDF, PS4, PDF4 |
|||
| Bayesian Inference | Pearl, Sections 2.1,2.3.1 Slides: PS, PDF,PS4, PDF4 Answers to puzzles: PS, PDF, PS4, PDF4 |
PS, PDF |
||
| Bayesian Networks: Introduction, I-maps | Pearl, Sections 3.1, 3.3 Slides: PS, PDF,PS4, PDF4 | |||
| Bayesian Networks: d-separation, construction | Friedman and Koller's notes on Bayesian networks Slides: PS, PDF, PS4, PDF4 |
|||
| Inference: Variable Elimination | Friedman and Koller's notes on Variable Elimination Slides: PS, PDF, PS4, PDF4 |
PS, PDF |
||
| Inference: Clique tree construction, Cutset conditioning | Pearl, Sections 4.4.1, 4.4.2 Friedman and Koller's notes on Clique tree inference Slides: PS, PDF, PS4, PDF4 |
|||
| Inference: Sampling | Pearl, Section 4.4.3; Friedman and Koller's notes on Approximate Inference Slides: PS, PDF, PS4, PDF4 |
|||
| Learning: Parameter Estimation | Koller and Friedman's notes on Parameter Estimation Heckerman's tutorial on Learning with Bayesian Networks Slides: PS, PDF, PS4, PDF4 |
|||
| Learning: Structure | Koller and Friedman's notes on Learning Structure Heckerman's tutorial on Learning with Bayesian Networks Slides: |
|||
| No class | |
|||
| Learning: Structure, Hidden Variables | TBA Slides: |
PS, PDF |
||
| Expectation Maximization (EM) | TBA Slides: |
|||
| Dynamic Bayesian Networks and Hidden Markov Models | TBA Slides: | Solution 2: PS, PDF | ||
| Pearl, Chapter 6 Slides: |
||||
| Spring break | |
|||
| Spring break | |
|||
| Introduction to Decision Making | Pearl, Sections 6.1, 6.3, 6.4 Slides: PS, PDF, PS4, PDF4 |
Reading 1 Posted |
||
| Markov Decision Processes | Slides: PS, PDF, PS4, PDF4 R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, Chapters 3-4 C. Boutilier, T. Dean and S. Hanks. Decision-Theoretic Planning: Structural Assumptions and Computational Leverage, Sections 1-3. |
|||
| Monte Carlo Methods | No slides R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, Chapter 5, Chapter 2 FYI |
|
||
| Temporal Difference Learning | No slides R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, Sections 6.1-6.5 |
|
||
| Eligibility traces | No slides R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction Sections 7.1-7.6, 7.8-7.11 |
|
||
| Approximation methods for MDPs | Slides: PS, PDF, PS4, PDF4 R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction, Chapter 8 |
PS, PDF |
||
| Structured state spaces | |
|||
| Hierarchical actions and temporal abstraction | |
|||
| No class (Easter) | |
|||
| Partially Observable Markov Decision Processes | No Slides L. Kaelbling, M. Littman, and A. Cassandra. |
|||
| Memory-based methods for POMDPs | |
Reading 2 Posted |
||
| Finite-state controllers for POMDPs
Class evaluations |
|
|||
| Wrap-up | |