Machine Learning (COMP-652)
Fall 2009


Date Topic Materials Homework
Sep.2 Introduction. Types of machine learning. Linear regression. Overfitting and cross-validation Lecture 1 slides
Bishop, Sec. 1.1
NY Times Article on Statistics
Sep.9 Overfitting and bias-variance error decomposition. Linear models with basis functions. Gradient descent Lecture 2 slides
Bishop, Sec. 3.1, 3.2
For a review of probability: Bishop, Sec. 1.2, 2.1-2.4
Homework 1 posted
Due September 16
Sep.14 More on linear methods for regression. Analysis of least-squares as maximum-likelihood learning. L2 and L1 regularization. Bayesian learning Lecture 3 slides
Bishop, Sec. 3.1, 3.3
Sep.16 Classification. Generative vs. discriminative learning. Naive Bayes. Gaussian discriminant analysis. Lecture 4 slides
Bishop, Sec. 4.2
Sep.21 More on classification. Logistic regression. Feed-forward neural networks and backpropagation Lecture 5 slides
Bishop, Sec. 4.3
Sep.23 Instance-based learning. Nearest-neighbor, locally weighted regression. Lecture 6 slides
Bishop, Sec. 2.5.2
Homework 2 posted
Data: hw2x.dat,, wpbcx.dat,
Due October 2
Sep.28 Decision trees Lecture 7 slides
Bishop, Sec. 14.4
Sep.30 Class cancelled  
Oct. 5 Ensemble methods. Bagging. Boosting Lecture 8 slides
Bishop, Sec. 14.2, 14.3
Homework 2 due!
Oct. 7 Discriminative learning. Perceptrons. Support vector machines Lecture 9 slides
Bishop, Sec. 4.1, 6.1
Oct.14 More on support vector machines. The kernel trick Lecture 10 slides
Bishop, Sec. 6.2, 7.1,
Oct. 19 Computational learning theory. Sample complexity. PAC bounds Lecture 11 slides
Oct. 21 Unsupervised learning: Clustering. K-means. Hierarchical clustering. Lecture 12 slides
Bishop, Sec. 9.1
Oct. 26 Active learning Lecture 13 slides
Oct.28 In-class midterm Covering lectures 1-11. You are allowed one double-sided cheat sheet.
See sample exams from 2007, 2006, 2005.
Nov. 2 Unsupervised learning: Density estimation. Mixture models. Gaussian mixture models and EM. Lecture 14 slides
Bishop, Sec. 9.2, 9.3
Nov. 4 Unsupervised learning: Bayes nets, learning parameters Lecture 15 slides
Nov. 9 No class  
Nov. 11 More on Bayes nets. Exact inference. Lecture 16 slides
Bishop, Chapter 8
Nov.16 Dimensionality reduction: PCA, kernel PCA Lecture 17 slides
Nov.18 Time series data: Hidden Markov models Lecture 18 slides
Nov.23 No class  
Nov.25 Time series: Linear dynamical system, particle filters, importance sampling Lecture 19 slides (2-hour lecture)
Sutton & Barto,
Nov.30 Approximate inference: Gibbs sampling. Other types of graphical models. Lecture 20 slides (2-hour lecture)
Sutton & Barto
Dec.2 Reinforcement learning: Policy evaluation Lecture 21 slides (2-hour lecture)
Sutton & Barto
Dec.3 Reinforcement learning: Control algorithms Lecture 22 slides
Dec.4 Make-up class: Wrap-up Lecture 23 slides
Dec.17 Project presentations Location and time TBA