Schedule
Date  Topic  Materials  Homework 
Sep.2  Introduction. Types of machine learning. Linear regression. Overfitting and crossvalidation  Lecture 1 slides Bishop, Sec. 1.1 NY Times Article on Statistics  
Sep.9  Overfitting and biasvariance error decomposition. Linear models with basis functions. Gradient descent  Lecture 2 slides Bishop, Sec. 3.1, 3.2 For a review of probability: Bishop, Sec. 1.2, 2.12.4 
Homework 1 posted Due September 16 
Sep.14  More on linear methods for regression. Analysis of leastsquares as maximumlikelihood learning. L2 and L1 regularization. Bayesian learning  Lecture 3 slides Bishop, Sec. 3.1, 3.3 

Sep.16  Classification. Generative vs. discriminative learning. Naive Bayes. Gaussian discriminant analysis.  Lecture 4 slides Bishop, Sec. 4.2 

Sep.21  More on classification. Logistic regression. Feedforward neural networks and backpropagation  Lecture 5 slides Bishop, Sec. 4.3 

Sep.23  Instancebased learning. Nearestneighbor, locally weighted regression.  Lecture 6 slides Bishop, Sec. 2.5.2 
Homework 2 posted
Data: hw2x.dat, hw2y.day, wpbcx.dat, wpbcy.day Due October 2 
Sep.28  Decision trees  Lecture 7 slides Bishop, Sec. 14.4 

Sep.30  Class cancelled  
Oct. 5  Ensemble methods. Bagging. Boosting  Lecture 8 slides Bishop, Sec. 14.2, 14.3 
Homework 2 due! 
Oct. 7  Discriminative learning. Perceptrons. Support vector machines  Lecture 9 slides Bishop, Sec. 4.1, 6.1 

Oct.14  More on support vector machines. The kernel trick  Lecture 10 slides Bishop, Sec. 6.2, 7.1, 

Oct. 19  Computational learning theory. Sample complexity. PAC bounds  Lecture 11 slides TBA 

Oct. 21  Unsupervised learning: Clustering. Kmeans. Hierarchical clustering.  Lecture 12 slides Bishop, Sec. 9.1 

Oct. 26  Active learning  Lecture 13 slides  
Oct.28  Inclass midterm 
Covering lectures 111. You are allowed one doublesided cheat sheet. See sample exams from 2007, 2006, 2005. 

Nov. 2  Unsupervised learning: Density estimation. Mixture models. Gaussian mixture models and EM.  Lecture 14 slides Bishop, Sec. 9.2, 9.3 

Nov. 4  Unsupervised learning: Bayes nets, learning parameters  Lecture 15 slides Bishop, 

Nov. 9  No class  
Nov. 11  More on Bayes nets. Exact inference.  Lecture 16 slides Bishop, Chapter 8 

Nov.16  Dimensionality reduction: PCA, kernel PCA  Lecture 17 slides &nssp; 

Nov.18  Time series data: Hidden Markov models  Lecture 18 slides Bishop, 

Nov.23  No class  
Nov.25  Time series: Linear dynamical system, particle filters, importance sampling  Lecture 19 slides (2hour lecture) Sutton & Barto, 

Nov.30  Approximate inference: Gibbs sampling. Other types of graphical models.  Lecture 20 slides (2hour lecture) Sutton & Barto 

Dec.2  Reinforcement learning: Policy evaluation  Lecture 21 slides (2hour lecture) Sutton & Barto 

Dec.3  Reinforcement learning: Control algorithms  Lecture 22 slides TBA 

Dec.4  Makeup class: Wrapup  Lecture 23 slides TBA 

Dec.17  Project presentations  Location and time TBA 