Syllabus for Probabilistic Reasoning in AI (COMP-526)

Winter 2008

General Information

Location:	Trottier, Room 0060
Times:	Monday, Wednesday, Friday, 10:35-11:25am.
Instructor:	Doina Precup, School of Computer Science.
Office:	McConnell 111N.
Phone:	(514) 398-6443.
Email:	dprecup@cs.mcgill.ca
Office hours:	Monday 11:30-12:00 and 5:00-6:00 and Friday 2:30-3:30. Meetings at other times by appointment only!
Class web page:	http://www.cs.mcgill.ca/~dprecup/courses/prob.html IMPORTANT: This is where class notes, announcements and homeworks are posted!
Teaching Assistant:	Pablo Samuel Castro Office Hours: Tuesday 3:30-5:30, McConnell 111 E-mail: pcastr at cs dot mcgill dot ca

Course Description

One of the primary goals of AI is the design, control and analysis of agents or systems that behave appropriately in a variety of circumstances. Good decision making often requires the existence of knowledge or beliefs about the agent's environment, as well as about its own abilities to observe and change the environment, and about its own goals and preferences. In this course we will examine computational approaches for modeling uncertainty and solving decision problems. We will focus mainly on probabilistic models of reasoning, and on sequential decision making.

The course is intended for advanced undergraduate students and for graduate students, and will provide an introduction to the on-going research in the field of reasoning under uncertainty, which has been very active during the last decade. We will cover the following topics:

The principles of Bayesian inference
Belief networks and Markovian networks
- Syntax and semantics
- Exact and approximate inference
- Learning methods
Temporal models
- Hidden Markov Models
- Dynamic Bayes nets
- Kalman filters
Basics of utility theory
Markov Decision Processes
- Exact and approximate planning methods
- Learning algorithms
Partially Observable Markov Decision Processes
- Exact and approximate planning methods
- Learning algorithms
Alternative representations for decision making under unncertainty

A tentative class schedule is posted on the course web page.

Prerequisites

Basic knowledge of a programming language is required. Basic knowledge of probabilities and statistics is highly recommended. Example courses at McGill providing sufficient background are MATH-323 or ECE-305. Some AI background is recommended, as provided, for instance by COMP-424 or ECE-526. If you have doubts regarding your background, please contact me to discuss it.

Reference Materials

The slides used in the lectures are posted on the schedule web page. In addition, we will use selected chapters from the following textbooks:

An Introduction to Probabilistic Graphical Models by Michael Jordan. Copies of the relevant chapters will be made available to registered students (permission pending).
Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto. MIT Press, 1998.
Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig, Second Edition. Part V and VI. Prentice Hall, 2002.
Information Theory, Inference and Learning Algorithms by David MacKay. Cambridge University Press, 2003.
Pattern Recognition and Machine Learning by Christopher Bishop. Springer, 2006.

The relevant sections will be indicated on the schedule web page as needed. We will supplement these as needed with research papers, which will be posted on the schedule web page.

Class Requirements

The class grade will be based on the following components:

Seven assignments, worth 70% of the grade. The assignments will contain a mixture of problem sets and programming questions.
One written examination, worth 15% of the grade. The exam will cover lectures 1-22 and will be scheduled in the evening during the week of March 3-7. The precise date and time will be determined after consultation with the class. More information about the exam content and sample problems will be posted on the course web page.
One class project, worth 15% of the grade. The project will be chosen from a list of pre-defined topics and will cover the second part of the course. More information on the project will be posted on the class web page at the beginning of March.
Participation in class discussions - up to 5% extra credit.

Minor changes to the evaluation scheme (if any) will be announced in class by Friday, January 7 (pending in-class discussion and the estimated total enrollment).

The problem sets include questions related to the material discussed in class. The programming assignments will involve experimenting with algorithms that we discuss in class and reporting results produced by these algorithms. We will be using Matlab for most assignments, but prior knowledge is not required.

Homework Policy

Assignments should be submitted before class or in class on the day when they are due. Submission is either done in hard copy or electronically, via e-mail to the TA. Assignments submitted late will be penalized by 10 points per day, up to 5 days. No credit is given for assignments submitted at a later time, unless you have a medical problem.

All assignments are INDIVIDUAL! You may discuss the problems with your colleagues, but you must submit individual homeworks. Please acknowledge all sources you use in the homeworks (papers, code or ideas from someone else).

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offences under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/integrity for more information).