COMP 599 HOME PAGE
Statistical Learning Theory
- A new lecture schedule has been posted. Please check below.
- I have posted notes below for the lecture
on VC dimension on the 16th of September.
- I gave the lecture live on the 14th of
September and it was recorded successfully. I have posted some notes
below but everything in my notes is also in the book.
- The lecture by Prof. Oberman is recorded and is available from
myCourses. To see it look under Content and not under
- Assignment 1 has been posted both here (see below) and on myCourses.
To find it in myCourses look under Content and not under
Assignments. The Assignments tab on myCourses is for submission of your
solutions to assignment 1. Assignments can be submitted until 11:00
- The course will be taught on campus in Trottier 100 from 11:35
to 12:55 Mondays and Wednesdays. The lectures will be
- This course is co-taught with Prof. Adam Oberman of the Department of
Mathematics and Statistics and is cross listed as MATH 597.
- Here is the lecture schedule.
- Here is a course outline.
- The recommended textbooks for the course are Understanding Machine
Learning by Shai Shalev-Schwartz and Shai Ben-David and Foundations
of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh and Ameet
Talwalkar; this book is available at Mohri's website. These books
have not been ordered through the bookstore.
- Lecture Times: MW 11:35 - 12:55
- Lecture Place: Trottier 100
- Office Hours: MW 1:30 - 2:30 by Zoom
- Office: McConnell (North Wing) 105N
- TA and office hours:
- Vincent Luczkow Wed 2:30 - 3:30 by Zoom
Video recording of lectures
These are also available in myCourses.
Here are some pointers to possible projects. Your project is not expected
to include any original new work. I expect a critical review of
some work. By "critical" I mean that you should formulate some reaction to
what you are reading and not just summarize. The expected size is 10 pages
with a variance of 2 pages. This is still being updated with more
modern references and some of the broken links have to be fixed.
Below are some suggestions for topics that could be explored in more
detail for the final project, and a couple of examples of relevant papers
for each topic. All of these would make good literature synthesis topic,
though you should be prepared to find additional relevant papers on your
own! (Google Scholar is a good
place to start.) This list might also inspire ideas for research projects.
Of course you are welcome to choose a topic that is not on the list.
Probabilistic bisimulation and metrics:
- Josée Desharnais, Abbas Edalat, and Prakash Panangaden.
labelled Markov processes. Information and Computation 179.2 (2002):
- Josee Desharnais, Vineet Gupta, Radha Jagadeesan and Prakash
Metrics for Labelled Markov Processes, by
Josee Desharnais, Vineet Gupta, Radhakrishnan Jagadeesan and Prakash
Panangaden, Theoretical Computer Science, 318(3), pp. 323-354, June
for Markov Decision Processes with Infinite State Spaces by Norm
Ferns, Prakash Panangaden and Doina Precup. In UAI 2005.
The Multi-Armed Bandit Problem:
- Abernethy, Hazan, and Rakhlin. Competing
in the Dark: An Efficient Algorithm for Bandit Linear Optimization.
- Even-Dar, Mannor, and Mansour. Action
Elimination and Stopping Conditions for the Multi-Armed Bandit and
Reinforcement Learning Problems. JMLR, 2006.
- Auer, Cesa-Bianchi, Freund, and Schapire. The
nonstochastic multiarmed bandit problem. SICOMP, 2002.
- Auer, Cesa-Bianchi, and Fischer. Finite
time analysis of the multiarmed bandit problem. MLJ, 2002.
Domain Adaptation and/or Multi-Source Learning:
- Ben-David, Blitzer, Crammer, Kulesza, Pereira, and Vaughan. A
Theory of Learning from Different Domains. MLJ, 2010.
- Mansour, Mohri, and Rostamizadeh. Domain adaptation:
Learning bounds and algorithms. COLT, 2009.
- Crammer, Kearns, and Wortman. Learning
from Multiple Sources. JMLR, 2008.
Privacy-Preserving Machine Learning:
- Chaudhuri, Monteleoni, Sarwate. Differentially
Private Empirical Risk Minimization. JMLR, 2010.
- Kasiviswanathan, Lee, Nissim, Raskhodnikova, and Smith. What Can We Learn
Privately? SICOMP, 2011.
- Blum, Ligett, and Roth. A
Learning Theory Approach to Non-Interactive Database Privacy. STOC,
Learning Bounds for Reinforcement Learning:
Online Convex Optimization:
GANs and metrics
McGill University values academic integrity. Therefore all students
must understand the meaning and consequences of cheating, plagiarism and
other academic offenses under the Code of Student Conduct and Disciplinary
Procedures (see http://www.mcgill.ca/integrity for more information). Most
importantly, work submitted for this course must represent your own
efforts. Copying assignments or tests from any source, completely or
partially, or allowing others to copy your work, will not be tolerated.
Every student has the right to submit written
work that is to be graded, in English or in French.
Chaque étudiant a le droit de soumettre en français ou en
anglais tout travail écrit.