Location: Wong 1030

Time: 2:35pm - 3:55pm, Mon & Wed

Taught by: Ted Perkins

Office: McGill Centre for Bioinformatics

Email: perkins@mcb.mcgill.ca

Phone: 398-5018

Office hours: MW 3:55-4:15pm, TR 12:00-12:40pm

TA 1: Cosmin Paduraro

Email: cosmin dot paduraru at gmail dot com

Office hours: Tue 11:00am-12:00pm, McConnell 111

TA 2: Jordan Frank

Email: jordan dot frank at cs dot mcgill dot ca

Office hours: Wed 4:00-5:00pm, McConnell 108

Credits: 4

Prereqs: COMP 424, COMP 526 or ECSE 526, COMP 360, MATH 323 or ECSE 305

HW | Files | Assigned | Due | Late | Sample solutions | Grader |

1 | COMP652_HW1.pdf COMP652_HW1_Q2Data.txt | Sep 10 | Sep 17 | Sep 22 | COMP652_HW1Solns.pdf | Cosmin |

2 | COMP652_HW2.pdf wpbc_x_normalized.txt wpbc_yrecur.txt | Sep 22 | Sep 29 | Oct 1 | COMP652_HW2Solns.pdf | Jordan |

3 | COMP652_HW3.pdf TTR.txt wpbc_x_normalized.txt wpbc_yrecur.txt | Oct 8 | Oct 20 | N/A | COMP652_HW3Solns.pdf | Cosmin |

4 | COMP652_HW4.pdf HW4_Q1_X.txt HW4_Q1_Y.txt HW4_Q2_X.txt HW4_Q2_Y.txt HW4_Q3_X.txt HW4_Q3_Y.txt | Oct 22 | Nov 3 | N/A | COMP652_HW4Solns.pdf | Jordan |

5 | COMP652_HW5.pdf HW5_Q1_X.txt HW5_Q3_X.txt HW5_Q3_Y.txt | Nov 4 | Nov 12 | Nov 17 | COMP652_HW5Solns.pdf | Cosmin |

6 | COMP652_HW6.pdf HW6_Q1_Traj.txt HW6_Q1_StateCoords.txt HW6_Q2_Data.txt | Nov 19 | Nov 26 | Nov 28 | COMP652_HW6Solns.pdf | Jordan |

Anonymized homework gradesheet: HWGradesAnon.txt

Lec | Date | Topic | Notes |

1 | Sep 3 | Introduction to machine learning. Course administration. Beginning discussion of supervised learning, and linear regression in particular. | Lecture01a.pdf |

2 | Sep 8 | Continuing linear regression. Polnomial regression. Overfitting. Cross-validation. Deriving the sum-of-squares error criteria via maximum likelihood. | Lecture02.pdf Lecture02.ps |

3 | Sep 10 | Guest lecture: Javad Sadri | |

4 | Sep 15 | Logistic regression. Gradient descent. | Lecture04.pdf Lecture04.ps |

5 | Sep 17 | Artificial neural networks. Backprop. | Lecture05.pdf Lecture05.ps |

6 | Sep 22 | Tricks to speed backprop. Second order methods for training ANNs. Network structure: overfitting & overtraining. A few "destructive" means to simplify network structure. | Lecture06.pdf Lecture06.ps |

7 | Sep 24 | Finishing ANNs: Constructive methods for determining ANN architecture. Special ANN architectures. Beginning generative learning. | Lecture07.pdf Lecture07.ps Lecture07b.pdf Lecture07b.ps |

8 | Sep 28 | Continuing generative learning: Maximum likelihood estimation of univariate and multivariate Gaussian densities. Gaussian discriminant analysis. Discrete distribution estimation. | Lecture08.pdf Lecture08.ps |

9 | Oct 6 | Continuing generative learning: Max likelihood estimation of discrete distributions. The problem of zeros. Application to text classification. Naive Bayes. | Lecture09.pdf Lecture09.ps |

10 | Oct 8 | Instance based learning: k-nearest enighbor, weighted nearest neighbor, nonparametric density estimation, locally-weighted regression. | Lecture10.pdf Lecture10.ps |

11 | Oct 15 | Decision trees | Lecture11.pdf Lecture11.ps |

12 | Oct 20 | Perceptrons and linear support vector machines. The margin, and derivation of the linear SVM optimization problem. | Lecture12.pdf Lecture12.ps |

13 | Oct 22 | Support vector machines - the nonseparable case, feature expansions, the Kernel trick, and Mercer's theorem. | Lecture13.pdf Lecture13.ps |

14 | Oct 27 | Intro to unsupervised learning. K-means clustering. EM for mixtures of Gaussians. | Lecture14.pdf Lecture14.ps |

15 | Oct 29 | Hierarchical clustering. Principal components analysis (PCA). | Lecture15.pdf Lecture15.ps HCExample1.pdf HCExample2.pdf |

16 | Nov 3 | Kernel PCA. Multidimensional scaling. Self-organizing maps. | Lecture16.pdf Lecture16.ps |

17 | Nov 5 | Beginning reinforcement learning. Bandit problems. Return. | Lecture17.pdf Lecture17.ps |

18 | Nov 10 | More reinforcement learning. Value functions, Monte Carlo, Policy iteration. | Lecture18.pdf Lecture18.ps |

19 | Nov 12 | More reinforcement learning. Approximate policy iteration. Dynamic programming. Model-based and Model-free methods. | Lecture19.pdf Lecture19.ps |

20 & 21 | Nov 17 & 19 | Hidden Markov models: inference and parameter fitting | Lecture20.pdf Lecture20.ps Plus, look on the web for Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition" |

21 & 22 | Nov 19 & 24 | Partially Observable Markov Decision Processes | Lecture21.pdf Lecture21.ps |

22 & 23 | Nov 24 & 26 | The bias-variance-noise decomposition. Bagging and boosting. | Lecture22.pdf Lecture22.ps |