Machine Learning (COMP 652)

Fall 2008

Course web page: www.cs.mcgill.ca/~perkins/COMP652_Fall2008/index.html
Location: Wong 1030
Time: 2:35pm - 3:55pm, Mon & Wed

Taught by: Ted Perkins
Office: McGill Centre for Bioinformatics
Email: perkins@mcb.mcgill.ca
Phone: 398-5018
Office hours: MW 3:55-4:15pm, TR 12:00-12:40pm

TA 1: Cosmin Paduraro
Email: cosmin dot paduraru at gmail dot com
Office hours: Tue 11:00am-12:00pm, McConnell 111

TA 2: Jordan Frank
Email: jordan dot frank at cs dot mcgill dot ca
Office hours: Wed 4:00-5:00pm, McConnell 108

Credits: 4
Prereqs: COMP 424, COMP 526 or ECSE 526, COMP 360, MATH 323 or ECSE 305

Summary

An overview of state-of-the-art algorithms used in machine learning, including theoretical properties and practical applications of these algorithms.

Handouts

  • Preliminary syllabus: [HTML] [PDF]
  • Homeworks

    HW Files Assigned Due Late Sample solutions Grader
    1 COMP652_HW1.pdf COMP652_HW1_Q2Data.txt Sep 10 Sep 17 Sep 22 COMP652_HW1Solns.pdf Cosmin
    2 COMP652_HW2.pdf wpbc_x_normalized.txt wpbc_yrecur.txt Sep 22 Sep 29 Oct 1 COMP652_HW2Solns.pdf Jordan
    3 COMP652_HW3.pdf TTR.txt wpbc_x_normalized.txt wpbc_yrecur.txt Oct 8 Oct 20 N/A COMP652_HW3Solns.pdf Cosmin
    4 COMP652_HW4.pdf HW4_Q1_X.txt HW4_Q1_Y.txt HW4_Q2_X.txt HW4_Q2_Y.txt HW4_Q3_X.txt HW4_Q3_Y.txt Oct 22 Nov 3 N/A COMP652_HW4Solns.pdf Jordan
    5 COMP652_HW5.pdf HW5_Q1_X.txt HW5_Q3_X.txt HW5_Q3_Y.txt Nov 4 Nov 12 Nov 17 COMP652_HW5Solns.pdf Cosmin
    6 COMP652_HW6.pdf HW6_Q1_Traj.txt HW6_Q1_StateCoords.txt HW6_Q2_Data.txt Nov 19 Nov 26 Nov 28 COMP652_HW6Solns.pdf Jordan

    Anonymized homework gradesheet: HWGradesAnon.txt

    Schedule

    Lec Date Topic Notes
    1 Sep 3 Introduction to machine learning. Course administration. Beginning discussion of supervised learning, and linear regression in particular. Lecture01a.pdf
    2 Sep 8 Continuing linear regression. Polnomial regression. Overfitting. Cross-validation. Deriving the sum-of-squares error criteria via maximum likelihood. Lecture02.pdf Lecture02.ps
    3 Sep 10 Guest lecture: Javad Sadri
    4 Sep 15 Logistic regression. Gradient descent. Lecture04.pdf Lecture04.ps
    5 Sep 17 Artificial neural networks. Backprop. Lecture05.pdf Lecture05.ps
    6 Sep 22 Tricks to speed backprop. Second order methods for training ANNs. Network structure: overfitting & overtraining. A few "destructive" means to simplify network structure. Lecture06.pdf Lecture06.ps
    7 Sep 24 Finishing ANNs: Constructive methods for determining ANN architecture. Special ANN architectures. Beginning generative learning. Lecture07.pdf Lecture07.ps Lecture07b.pdf Lecture07b.ps
    8 Sep 28 Continuing generative learning: Maximum likelihood estimation of univariate and multivariate Gaussian densities. Gaussian discriminant analysis. Discrete distribution estimation. Lecture08.pdf Lecture08.ps
    9 Oct 6 Continuing generative learning: Max likelihood estimation of discrete distributions. The problem of zeros. Application to text classification. Naive Bayes. Lecture09.pdf Lecture09.ps
    10 Oct 8 Instance based learning: k-nearest enighbor, weighted nearest neighbor, nonparametric density estimation, locally-weighted regression. Lecture10.pdf Lecture10.ps
    11 Oct 15 Decision trees Lecture11.pdf Lecture11.ps
    12 Oct 20 Perceptrons and linear support vector machines. The margin, and derivation of the linear SVM optimization problem. Lecture12.pdf Lecture12.ps
    13 Oct 22 Support vector machines - the nonseparable case, feature expansions, the Kernel trick, and Mercer's theorem. Lecture13.pdf Lecture13.ps
    14 Oct 27 Intro to unsupervised learning. K-means clustering. EM for mixtures of Gaussians. Lecture14.pdf Lecture14.ps
    15 Oct 29 Hierarchical clustering. Principal components analysis (PCA). Lecture15.pdf Lecture15.ps HCExample1.pdf HCExample2.pdf
    16 Nov 3 Kernel PCA. Multidimensional scaling. Self-organizing maps. Lecture16.pdf Lecture16.ps
    17 Nov 5 Beginning reinforcement learning. Bandit problems. Return. Lecture17.pdf Lecture17.ps
    18 Nov 10 More reinforcement learning. Value functions, Monte Carlo, Policy iteration. Lecture18.pdf Lecture18.ps
    19 Nov 12 More reinforcement learning. Approximate policy iteration. Dynamic programming. Model-based and Model-free methods. Lecture19.pdf Lecture19.ps
    20 & 21 Nov 17 & 19 Hidden Markov models: inference and parameter fitting Lecture20.pdf Lecture20.ps Plus, look on the web for Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition"
    21 & 22 Nov 19 & 24 Partially Observable Markov Decision Processes Lecture21.pdf Lecture21.ps
    22 & 23 Nov 24 & 26 The bias-variance-noise decomposition. Bagging and boosting. Lecture22.pdf Lecture22.ps