Syllabus
General Information
| Location: | McConnell Engineering room 103 |
| Times: | Friday, 10:05am-12:55pm |
| Instructor: | Doina Precup and Pierre-Luc Bacon, School of Computer Science |
| Office: | McConnell Engineering building, room 111N and 107 |
| Phone: | (514) 398-6443 (Doina) |
| Email: | dprecup@cs.mcgill.ca and pbacon@cs.mcgill.ca |
| Office hours:
|
TBA (see course home page)
Meetings at other times by appointment only! |
| Class web page: | http://www.cs.mcgill.ca/~dprecup/courses/rl.html
IMPORTANT: This is where class notes, announcements and homeworks are posted! |
Course Description
The goal of this class is to provide an introduction to reinforcement learning, a very active part of machine learning. Reinforcement learning is concerned with building programs which learn how to predict and act in a stochastic environment, based on past experience. Applications of reinforcement learning range from classical control problems, such as powerplant optimization or dynamical system control, to game playing, inventory control, and many other fields. Notably, reinforcement learning has also produced very compelling models of animal and human learnign. During this course, we will study theoretical properties and practical applications of reinforcement leanring. We will follwo the second edition of the classic textbook by Sutton & Barto (available online), and supplement it as needed with papers and other materials.
Prerequisites
Basic knowledge of a programming language is required. Knowledge of probabilities/statistics, calculus and linear algebra is required. Example courses at McGill providing sufficient background in probability are MATH-323 or ECSE-305. Machine learning background, as provided for example by COMP-551 or COMP-652 is required. If you have doubts regarding your background, please contact Doina to discuss it.
Reference Materials
Required textbook:- Richard S. Sutton and Andrew G. Barto, "Reinforcement learning: An introduction", Second Edition, MIT Press, in preparation
- Csaba Szepesvari, "Algorithms for Reinforcement Learning", Morgan and Claypool, 2010.
- Dimitri Bertsekas and John Tsitsiklis, "Neuro-dynamic programming", Athena Scientific, 1997.
MyCourses will be used only for bulletin board, discussion groups and assignment submission and grading.
Class Requirements
The class grade will be based on the following components:
- Five assignments - 60%. Each week, part of the course will be devoted to student presentations and demonstrations. Each week, on Friday evening, a set of papers and possible experimental demonstration topics will be posted. Students can choose a topic from this list and be prepared to do a short presentation in the following week on this topic. Every student MUST subscribe to doing this 5 times during the term. Students can work individually or in teams of 2. Due to time constraints, not all students or teams who volunteered in a given week may be picked to present during that week, but they have to post their material on a shared site. This material will be graded.
- A final project - 40%. For the final project, students can work individually or in groups of up to 3 students, on a topic of their choice. Students must send by email to the course instructors a brief description of their topic by March 10. Project presntation will be scheduled outside of class hours, and project reports wil be due after the presentations.
- Participation in class discussions - up to 1% extra credit.
McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures (see www.mcgill.ca/students/srr/honest for more information).
In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.
In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.