Course Description

The goal of this class is to provide an introduction to reinforcement learning, a very active part of machine learning. Reinforcement learning is concerned with building computer agents which learn how to predict and act in a stochastic environment, based on past experience. Applications of reinforcement learning range from classical control problems, such as power plant optimization or dynamical system control, to game playing, inventory control, and many other fields. Notably, reinforcement learning has also produced very compelling models of animal and human learning. During this course, we will study theoretical properties and practical applications of reinforcement learning. We will follow the second edition of the classic textbook by Sutton & Barto (available online for free, or from MIT Press), and supplement it as needed with papers and other materials.


Prerequisites

Knowledge of a programming language is required (ideally Python). Knowledge of probability/statistics, multivariate calculus and linear algebra is required. Example courses at McGill providing sufficient background in probability are MATH-323 or ECSE-305. Machine learning background, as provided for example by COMP-451, COMP-551 or COMP-652 is required. If you have doubts regarding your background, please contact Doina or Isabeau to discuss it.


Reference Materials

Required textbook: Lecture notes and other relevant materials will be linked to the lectures web page.

MyCourses will be used for bulletin board, access to Ed discussion groups, and assignment submission and grading.


Class Requirements

The class grade will be based on the following components:

  1. Three assignments - 60%. Assignments will consist of a mixture of theoretical questions and programming work.
  2. A final project - 40%. For the final project, students can work individually or in groups of up to 3 students. More information about the project will be provided in February.
Minor changes to the evaluation scheme (if any) will be announced in class by Thurssday January 11 (pending in-class discussion and the estimated total enrollment).


McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/students/srr/honest for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.