Syllabus for Probabilistic Reasoning in AI (COMP-526)

Winter 2004

General Information

Location:	Burnside Hall, Room 1B24.
Times:	Monday, Wednesday, Friday, 1:35-2:25pm.
Instructor:	Professor Doina Precup, School of Computer Science.
Office:	McConnell 326.
Phone:	398-6443.
Email:	dprecup@cs.mcgill.ca
Office hours:	Wednesday, 2:30-3:30pm; Friday, 2:30-3:00pm. Meetings at other times by appointment only! IMPORTANT: Email is the easiest way to reach me!
Class web page:	http://www.cs.mcgill.ca/~dprecup/courses/prob.html IMPORTANT: This is where class notes, announcements and homeworks are posted!
Teaching Assistant:	Rohan Shah Email: rshah3@cs.mcgill.ca Office hours: TBA

Course Description

One of the primary goals of AI is the design, control and analysis of agents or systems that behave appropriately in a variety of circumstances. Good decision making often requires the existence of knowledge or beliefs about the agent's environment, as well as about its own abilities to observe and change the environment, and about its own goals and preferences. In this course we will examine computational approaches for modeling the environment and solving decision problems. We will focus mainly on probabilistic models of reasoning, and on sequential decision making.

The course is intended for advanced undergraduate students and for graduate students, and will provide an introduction to the on-going research in the field of reasoning under uncertainty, which has been very active during the last decade. We will cover the following topics:

The principles of Bayesian inference
Belief networks
- Syntax and semantics
- Exact and approximate inference
- Learning methods
Temporal models
- Hidden Markov Models
- Dynamic Bayes nets
- Kalman filters
Basics of utility theory
Markov Decision Processes
- Dynamic programming methods
- Structured state and action spaces
- Temporal-difference learning algorithms
- Generalization and function approximation
Partially Observable Markov Decision Processes
- Exact solution methods
- Approximate methods

A tentative class schedule is posted on the course web page.

Prerequisites

Basic knowledge of a programming language is required. Basic knowledge of probabilities and statistics is highly recommended. Example courses at McGill providing sufficient background are MATH-323 or ECE-305. Some AI background is recommended, as provided, for instance by COMP-424 or ECE-526. If you have doubts regarding your background, please contact me to discuss it. Should you need to find more on course descriptions and pre-requisites, please refer to the Undergraduate Course Home Pages.

Reference Materials

The first part of the course is based mainly on the textbook An Introduction to Probabilistic Graphical Models by Michael Jordan (in preparation). The second part of the course is based mostly on the textbook Reinforcement learning: An introduction by Richard Sutton and Andrew Barto (MIT Press, 1998). The full text of this book is available on-line. Since the books do not cover many exciting developments in the field, we will supplement them with research papers and other notes. These will be distributed in class or posted on the web page, as appropriate. The class slides will be posted on the web page.

Class Requirements

The class grade will be based on the following components:

Approximately 10 written assignments, worth 70% of the grade. The assignments will contain a mixture of problem sets, programming questions, and reading summaries. The lowest grade on the homeworks will be discarded.
Two written examinations, each worth 15% of the grade. Each exam will cover roughly half of the class material.
Participation in class discussions - up to 5% extra credit.

Minor changes to the evaluation scheme (if any) will be announced in class by Monday, January 12 (pending in-class discussion and the estimated total enrollment).

The problem sets include questions related to the material discussed in class. The programming assignments will involve experimenting with algorithms that we discuss in class and reporting results produced by these algorithms. The ability to program in a high-level language, such as C/C++ or Java, under the UNIX operating system is assumed. We may also use programs written by other researchers, as needed. The reading assignments will involve summarizing and/or critiquing research papers in the field. When reading, always think about questions that you may have, and bring them to class.

Homework Policy

Assignments should be submitted in class on the day when they are due. If you cannot come to class, please contact me to make different arrangements for submitting the assignment. In addition, the answers to programming problems must also be submitted electronically, using the handin system. Assignments submitted until 1:30pm on the next day will be penalized by 10 points. Assignments submitted before the next class are penalized 25 points. No credit is given for assignments submitted at a later time, unless you have a medical problem.

All assignments are INDIVIDUAL! You may discuss the problems with your colleagues, but you must submit individual homeworks. Please acknowledge all sources you use in the homeworks (papers, code or ideas from someone else).

Additional information

CORRECTION TO CALENDAR: Thursday, April 8, 2004, is the last day of class for lectures that follow the Tuesday-Thursday class schedule. Tuesday, April 13, 2004, will follow the Monday class schedule, and is the last day of class for lectures that follow the Monday-Wednesday-Friday schedule.

Prof. Doina PRECUP

Last modified: Wed Jan 14 12:20:32 EST 2004