Schedule
| Date | Topic | Materials | Homework |
| Sep.2 | Introduction. Types of machine learning. Linear regression. Overfitting and cross-validation | Lecture 1 slides Bishop, Sec. 1.1 NY Times Article on Statistics | |
| Sep.9 | Overfitting and bias-variance error decomposition. Linear models with basis functions. Gradient descent | Lecture 2 slides Bishop, Sec. 3.1, 3.2 For a review of probability: Bishop, Sec. 1.2, 2.1-2.4 |
Homework 1 posted Due September 16 |
Sep.14 | More on linear methods for regression. Analysis of least-squares as maximum-likelihood learning. L2 and L1 regularization. Bayesian learning | Lecture 3 slides Bishop, Sec. 3.1, 3.3 |
| Sep.16 | Classification. Generative vs. discriminative learning. Naive Bayes. Gaussian discriminant analysis. | Lecture 4 slides Bishop, Sec. 4.2 |
|
| Sep.21 | More on classification. Logistic regression. Feed-forward neural networks and backpropagation | Lecture 5 slides Bishop, Sec. 4.3 |
|
| Sep.23 | Instance-based learning. Nearest-neighbor, locally weighted regression. | Lecture 6 slides Bishop, Sec. 2.5.2 |
Homework 2 posted
Data: hw2x.dat, hw2y.day, wpbcx.dat, wpbcy.day Due October 2 |
| Sep.28 | Decision trees | Lecture 7 slides Bishop, Sec. 14.4 |
|
| Sep.30 | Class cancelled | ||
| Oct. 5 | Ensemble methods. Bagging. Boosting | Lecture 8 slides Bishop, Sec. 14.2, 14.3 |
Homework 2 due! |
| Oct. 7 | Discriminative learning. Perceptrons. Support vector machines | Lecture 9 slides Bishop, Sec. 4.1, 6.1 |
|
| Oct.14 | More on support vector machines. The kernel trick | Lecture 10 slides Bishop, Sec. 6.2, 7.1, |
|
| Oct. 19 | Computational learning theory. Sample complexity. PAC bounds | Lecture 11 slides TBA |
|
| Oct. 21 | Unsupervised learning: Clustering. K-means. Hierarchical clustering. | Lecture 12 slides Bishop, Sec. 9.1 |
|
| Oct. 26 | Active learning | Lecture 13 slides | |
| Oct.28 | In-class midterm |
Covering lectures 1-11. You are allowed one double-sided cheat sheet. See sample exams from 2007, 2006, 2005. |
|
| Nov. 2 | Unsupervised learning: Density estimation. Mixture models. Gaussian mixture models and EM. | Lecture 14 slides Bishop, Sec. 9.2, 9.3 |
|
| Nov. 4 | Unsupervised learning: Bayes nets, learning parameters | Lecture 15 slides Bishop, |
|
| Nov. 9 | No class | ||
| Nov. 11 | More on Bayes nets. Exact inference. | Lecture 16 slides Bishop, Chapter 8 |
|
| Nov.16 | Dimensionality reduction: PCA, kernel PCA | Lecture 17 slides &nssp; |
|
| Nov.18 | Time series data: Hidden Markov models | Lecture 18 slides Bishop, |
|
| Nov.23 | No class | ||
| Nov.25 | Time series: Linear dynamical system, particle filters, importance sampling | Lecture 19 slides (2-hour lecture) Sutton & Barto, |
|
| Nov.30 | Approximate inference: Gibbs sampling. Other types of graphical models. | Lecture 20 slides (2-hour lecture) Sutton & Barto |
|
| Dec.2 | Reinforcement learning: Policy evaluation | Lecture 21 slides (2-hour lecture) Sutton & Barto |
|
| Dec.3 | Reinforcement learning: Control algorithms | Lecture 22 slides TBA |
|
| Dec.4 | Make-up class: Wrap-up | Lecture 23 slides TBA |
|
| Dec.17 | Project presentations | Location and time TBA |