Machine Learning (COMP 652)

Fall 2008

Course web page:
Location: Wong 1030
Time: 2:35pm - 3:55pm, Mon & Wed

Taught by: Ted Perkins
Office: McGill Centre for Bioinformatics
Phone: 398-5018
Office hours: MW 3:55-4:15pm, TR 12:00-12:40pm

TA 1: Cosmin Paduraro
Email: cosmin dot paduraru at gmail dot com
Office hours: Tue 11:00am-12:00pm, McConnell 111

TA 2: Jordan Frank
Email: jordan dot frank at cs dot mcgill dot ca
Office hours: Wed 4:00-5:00pm, McConnell 108

Credits: 4
Prereqs: COMP 424, COMP 526 or ECSE 526, COMP 360, MATH 323 or ECSE 305


An overview of state-of-the-art algorithms used in machine learning, including theoretical properties and practical applications of these algorithms.


  • Preliminary syllabus: [HTML] [PDF]
  • Homeworks

    HW Files Assigned Due Late Sample solutions Grader
    1 COMP652_HW1.pdf COMP652_HW1_Q2Data.txt Sep 10 Sep 17 Sep 22 COMP652_HW1Solns.pdf Cosmin
    2 COMP652_HW2.pdf wpbc_x_normalized.txt wpbc_yrecur.txt Sep 22 Sep 29 Oct 1 COMP652_HW2Solns.pdf Jordan
    3 COMP652_HW3.pdf TTR.txt wpbc_x_normalized.txt wpbc_yrecur.txt Oct 8 Oct 20 N/A COMP652_HW3Solns.pdf Cosmin
    4 COMP652_HW4.pdf HW4_Q1_X.txt HW4_Q1_Y.txt HW4_Q2_X.txt HW4_Q2_Y.txt HW4_Q3_X.txt HW4_Q3_Y.txt Oct 22 Nov 3 N/A COMP652_HW4Solns.pdf Jordan
    5 COMP652_HW5.pdf HW5_Q1_X.txt HW5_Q3_X.txt HW5_Q3_Y.txt Nov 4 Nov 12 Nov 17 COMP652_HW5Solns.pdf Cosmin
    6 COMP652_HW6.pdf HW6_Q1_Traj.txt HW6_Q1_StateCoords.txt HW6_Q2_Data.txt Nov 19 Nov 26 Nov 28 COMP652_HW6Solns.pdf Jordan

    Anonymized homework gradesheet: HWGradesAnon.txt


    Lec Date Topic Notes
    1 Sep 3 Introduction to machine learning. Course administration. Beginning discussion of supervised learning, and linear regression in particular. Lecture01a.pdf
    2 Sep 8 Continuing linear regression. Polnomial regression. Overfitting. Cross-validation. Deriving the sum-of-squares error criteria via maximum likelihood. Lecture02.pdf
    3 Sep 10 Guest lecture: Javad Sadri
    4 Sep 15 Logistic regression. Gradient descent. Lecture04.pdf
    5 Sep 17 Artificial neural networks. Backprop. Lecture05.pdf
    6 Sep 22 Tricks to speed backprop. Second order methods for training ANNs. Network structure: overfitting & overtraining. A few "destructive" means to simplify network structure. Lecture06.pdf
    7 Sep 24 Finishing ANNs: Constructive methods for determining ANN architecture. Special ANN architectures. Beginning generative learning. Lecture07.pdf Lecture07b.pdf
    8 Sep 28 Continuing generative learning: Maximum likelihood estimation of univariate and multivariate Gaussian densities. Gaussian discriminant analysis. Discrete distribution estimation. Lecture08.pdf
    9 Oct 6 Continuing generative learning: Max likelihood estimation of discrete distributions. The problem of zeros. Application to text classification. Naive Bayes. Lecture09.pdf
    10 Oct 8 Instance based learning: k-nearest enighbor, weighted nearest neighbor, nonparametric density estimation, locally-weighted regression. Lecture10.pdf
    11 Oct 15 Decision trees Lecture11.pdf
    12 Oct 20 Perceptrons and linear support vector machines. The margin, and derivation of the linear SVM optimization problem. Lecture12.pdf
    13 Oct 22 Support vector machines - the nonseparable case, feature expansions, the Kernel trick, and Mercer's theorem. Lecture13.pdf
    14 Oct 27 Intro to unsupervised learning. K-means clustering. EM for mixtures of Gaussians. Lecture14.pdf
    15 Oct 29 Hierarchical clustering. Principal components analysis (PCA). Lecture15.pdf HCExample1.pdf HCExample2.pdf
    16 Nov 3 Kernel PCA. Multidimensional scaling. Self-organizing maps. Lecture16.pdf
    17 Nov 5 Beginning reinforcement learning. Bandit problems. Return. Lecture17.pdf
    18 Nov 10 More reinforcement learning. Value functions, Monte Carlo, Policy iteration. Lecture18.pdf
    19 Nov 12 More reinforcement learning. Approximate policy iteration. Dynamic programming. Model-based and Model-free methods. Lecture19.pdf
    20 & 21 Nov 17 & 19 Hidden Markov models: inference and parameter fitting Lecture20.pdf Plus, look on the web for Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition"
    21 & 22 Nov 19 & 24 Partially Observable Markov Decision Processes Lecture21.pdf
    22 & 23 Nov 24 & 26 The bias-variance-noise decomposition. Bagging and boosting. Lecture22.pdf