Machine Learning (COMP-652)
Fall 2009


General Information

Location:McConnell Engineering building, room 103
Times:Monday and Wednesday, 2:35-3:55pm
Instructor:Professor Doina Precup, School of Computer Science.
Office:McConnell Engineering building, room 111N (left from the elevators).
Phone:(514) 398-6443.
Office hours:
Monday 4:00-4:30pmand Friday, 1:30-3:30pm
Meetings at other times by appointment only!
Teaching assistants:
Jordan Frank
Office Hours: Tuesday, 1:30-2:30pm, McConnell Engineering building room 108 (right from the elevators)
E-mail: jordan dot frank at cs dot mcgill dot ca
  Mahdi Milani Fard
Office Hours: Thursday 2:30-3:30pm, McConnell Engineering building room 112 (right from the elevators)
E-mail: mahdi dot milanifard at mail dot mcgill dot ca
Class web page:
IMPORTANT: This is where class notes, announcements and homeworks are posted!

Course Description

The field of machine learning is concerned with the question of how to construct computer programs that improve automatically with experience. In recent years, many successful applications of machine learning have been developed, ranging from data-mining programs that learn to detect fraudulent credit card transactions, to autonomous vehicles that learn to drive on public highways. At the same time, there have been important advances in the theory and algorithms that form the foundation of this field. The goal of this class is to provide an overview of the state-of-art algorithms used in machine learning. We will discuss both the theoretical properties of these algorithms and their practical applications.


Basic knowledge of a programming language is required. Basic knowledge of probabilities and statistics is required. Example courses at McGill providing sufficient background are MATH-323 or ECSE-305. Some AI background is recommended, as provided, for instance by COMP-424 or ECSE-526, but not required. If you have doubts regarding your background, please contact me to discuss it.

Reference Materials

The recommended references are: Alternative textbooks: Lecture notes and other relevant materials are available on this web page. Other reference materials will be distributed in class as needed.

Class Requirements

The class grade will be based on the following components:

  1. Up to seven assignments, of which the lowest score will be dropped - 60%.
    The assignments will contain a mix of theoretical and experimental questions. Experimentation will mainly be carried out in Matlab, but prior knowledge of Matlab is NOT required. Matlab is available through a site licence at McGill. You can also use Octave, which is a free software very similar to Matlab.
  2. A midterm in-class examination - 20%.
  3. A final project - 20%.
    The final project will involve choosing a topic, either from a list of suggestions that will be provided or based on your own interests. A typical project will require you to read papers related to the topic, implement or find available implementations of the algorithms discussed, and perform experimental work comparing these algorithms. At the end of the semester, you will write a final report and you will prepare a 10-minute class presentation of the topic you investigated. The presentations will be scheduled outside of our lecture time, during December.
  4. Participation in class discussions - up to 1% extra credit.
Minor changes to the evaluation scheme (if any) will be announced in class by Wednesday, September 10 (pending in-class discussion and the estimated total enrollment).

Homework Policy

Assignments should be submitted electronically on the day when they are due. You should e-mail the assignment to cs652 at cs dot mcgill dot ca. For late assignments, 10 points will be deducted from the grade for every late day, for the first 5 days. No credit is given for assignments submitted more than 5 days late, unless you have a medical problem.

All assignments are INDIVIDUAL! You may discuss the problems with your colleagues, but you must submit individual homeworks. Please acknowledge all sources you use in the homeworks (papers, code or ideas from someone else).

McGill University values academic integrity. Therefore all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures (see for more information).

In accord with McGill University's Charter of Students' Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

In the event of extraordinary circumstances beyond the University's control, the content and/or evaluation scheme in this course is subject to change.

Tentative Course Content

  1. Introduction (1 lecture)
  2. Linear models (3 lectures): linear an polynomial regression, overfitting, model selection, logistic regression, Naive Bayes
  3. Non-linear models (5 lectures): decision trees, instance-based learning, boosting, neural networks
  4. Support vector machines and kernels (2 lectures)
  5. Computational learning theory (2 lectures)
  6. Experimental methodology, sources of error (1 lecture)
  7. Midterm (1 lecture)
  8. Structured models (2 lectures): graphical models, deep belief networks
  9. Unsupervised learning (3 lectures): K-means, expectation maximization, PCA and other dimensionality reduction methods
  10. Learning in dynamical systems (2 lectures): Hidden Markov Models and other types of temporal/sequence models
  11. Reinforcement learning (3 lectures)
  12. Wrap-up (1 lecture)
IMPORTANT: This outline is subject to change. Up-to-date information about the course content and assigned readings will be posted on the schedule web page.