General Information

Location:Trottier (ENGTR) room 0100
Times:Monday and Wednesday, 8:35-9:55am
Instructor:Doina Precup, School of Computer Science
Office:McConnell Engineering building, room 111N
Phone:(514) 398-6443 (Doina)
Office hours: See course home page
Class web page:
IMPORTANT: This is where class notes, announcements and homeworks are posted!

Course Description

The goal of this class is to provide an introduction to reinforcement learning, a very active research sub-field of machine learning. Reinforcement learning is concerned with building programs that learn how to predict and act in a stochastic environment, based on past experience. Applications of reinforcement learning range from classical control problems, such as powerplant optimization or dynamical system control, to game playing, inventory control, and many other fields. Notably, reinforcement learning has also produced very compelling models of animal and human learning. During this course, we will study theoretical properties and practical applications of reinforcement learning. We will follow the second edition of the classic textbook by Sutton & Barto (available online for free, or from MIT Press), and supplement it as needed with papers and other materials.


Knowledge of programming in Python is required. Knowledge of probabilities/statistics, calculus and linear algebra is required. Example courses at McGill providing sufficient background in probability are MATH-323 or ECSE-305. Machine learning background, as provided for example by COMP-551 or COMP-652 is required. If you have doubts regarding your background, please contact Doina to discuss it.

Reference Materials

Required textbook: Additional textbooks: Lecture notes and other relevant materials are linked to the lectures web page.

MyCourses will be used only for bulletin board, discussion groups and assignment submission and grading.

Class Requirements

The class grade will be based on the following components:

  1. Three assignments - 45%. Assignments will consist of working either individually or in teams of 2 students. Assignments will contain a mix of theoretical questions and programming/experimentation questions. Any student is expected to be able to answer questions about all parts of the assignment.
  2. A midterm exam - 25%. The exam is tentatively scheduled on March 11. It is an in-class exam, concerning the material covered until March break, and you are permitted one double-sided crib sheet.
  3. A final project - 30%. For the final project, students can work individually or in groups of up to 3 students, on a topic of their choice. Students must send, by email, to the course instructor a brief description of their topic by March 16. Projects will be presented in a poster session on April 8. Project reports can be handed in without penalty until April 20.
  4. Participation in class discussions is highly encouraged
Minor changes to the evaluation scheme (if any) will be announced in class by Wednesday, January 15 (pending in-class discussion and the estimated total enrollment).

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures (see for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.