Goals

The main goal of the project is to allow you to study more in-depth machine learning algorithms of your choice. A secondary goal is to help you develop your research skills (reviewing research literature, formulating questions, deciding on a theoretical or empirical approach to answer them, writing up you findings and presenting them to your peers). Ideally, the project topic you choose should be related to your research interests.

There are three main categories of projects:

• Applying machine learning to interesting problem domains, thereby allowing you to gain in-depth experience with specific algorithms.
• Studying machine learning algorithms not covered in class.
• Investigating theoretical issues.

Ideally, the projects would be individual. However, if the topic is very big, teams of two people would be accepted too. If you choose to team up, there should be a clear delimitation between the work of that each person does. Both students should turn in separate write-ups and each should present a part of the work.

I strongly encourage you to perform some experiments as part of your project, in order to gain practical experience with machine learning algorithms. Many algorithms are available free from universities and research labs, so you would not have to code them. The resources web page should help you find both papers and software as needed.

Project themes

If you are already have a research topic in mind, feel free to propose it. Otherwise, you can choose a topic from the list below. If no topic suits your interest, please make an appointment to discuss alternatives.

Proposed topics:

• Machine learning for bioinformatics tasks
  A good starting point for problems is Baldi and Brunak's book, "Bioinformatics: The machine learning approach". The exact topic depends on the data you have available here at McGill.
• Machine learning for compiler optimization
  Possible applications here include function inlining, instruction scheduling, register allocation.
• Comparing machine learning methods for face/object recognition
  This is a topic that has been explored a lot. The main idea would be to run an empirical study, mainly involving new algorithms (e.g. ensembles, support vector machines).
• Learning in active vision
  The problem of active vision investigates where the focus of attention should be in order to gather as much information as possible about the task at hand. Reinforcement learning and Bayesian methods are adequate for this task.
• Links between machine learning and data compression
  Machine learning algorithms can be viewed as compressing data into a more compact representation. A theoretical project on this topic would look at the relationship between Kolmogorov complexity and the difficulty of machine learning tasks. An empirical project could look into similarities between compression and learning algorithms.
• Recurrent neural networks and their applications
  Recurrent neural networks are typically used for processing time series data. They use the outputs of units at time t as inputs for other units at time t+1. They are briefly mentioned in Tom Mitchell's book. A project could involve comparing different learning algorithms for recurrent nets, and/or applying one such algorithm to "interesting" data.
• Hidden Markov Models and applications
  HMMs are stochastic models for time series data, widely used in speech recognition, text processing, bioinformatics. I have a couple of great tutorials by Rabiner on learning for HMMs. The project involves studying these, then working on an application.
• Effect of exploration strategies in reinforcement learning.
  There is a great variety of exploration strategies in RL, varying from very straightforward methods (such as Boltzmann exploration or epsilon-greedy) to methods very well-motivated, such as Singh and Kearns' E3 algorithm. The project would involve surveying and comparing existing methods.
• Effect of eligibility traces in reinforcement learning
  Eligibility traces can significantly speed up reinforcement learning. In class we discuss only one version (accumulating traces) but several have been proposed. The project will especially involve looking a variable eligibility trace parameters.
• Actor-critic methods for reinforcement learning
  Our RL lectures discuss value-based RL only. Actor-critic methods use an explicit representation of a policy and a value function. They have better theoretical guarantees when used with function approximation. The purpose of the project will be to study and experiment 3with these methods.
• Learning orderings and preferences
  There are many applications in which it is desirable to order rather than classify instances. Of course, the desired ranks can be treated as class labels, but that can make learning much harder. Several special-purpose algorithms have been proposed for learning an ordering directly.
• Models of on-line learning.
  When we studied computational learning theory, we assumed all the data was available at the beginning. On-line learning studies the scenario in which data keeps accumulating over time. Avrim Blum has a nice survey paper on on-line learning, that you can use as a starting point. The project would also involve experimenting with an on-line learning algorithm.
• Blind source separation
  This is a good project for people interested in auditory perception. The problem considers a signal obtained by the superposition of several different sources (e.g. an audio signal with several people talking at once). The issue is to identify each source. Bayesian methods help in solving this problem.

Format and dates

The first step is to complete a project proposal, which is due in class on Tuesday, November 5. This should be a short document (max. 2 pages) stating the title of your intended project, the reason why you are interested in this topic, a rough plan (e.g. what machine learning algorithms you would use, where you would get the data), and a list of 5 papers that you think will be relevant to your topic. You do not have to read the papers before the proposal, nor do you have to include them in your final bibliography, if they end up not being relevant. The main purpose of this document is to inform me of your intentions, so I can give you feedback on the scope of the work.

The project itself will include two major components:

• Project report.
  The report should be approximately 10 pages long (this requirement is just to get you oriented; do not take it as a hard restriction). The format should be similar to the research papers we have been reading during the semester. It should contain the following information:
  • A description of the topic and why you were interested in it
  • A review of related literature; this should include at least 5 papers relevant to your topic. If you want help seeking such papers, let me know.
  • One or more specific questions that you wanted to investigate
  • The methodology you decided to follow (e.g. theory or experiments, what kind of experiments, etc.)
  • An analysis of your findings
  • Conclusion and possible future work directions
  The due date for the report is Monday, December 17, by 5pm.

• Project presentation.
  The presentations will take place in class during the lectures of November 27, November 29, and December 4. You should plan for a 15 minute talk, with 5 minutes for questions. You should assume that everyone knows the material covered during the class, so cover only the aspects that are particular to your topic. The presentations will be evaluated by everyone present (although I will make the final decision on the grade).

  Note that the presentations will take place much earlier than the project is due. Hence, you are not expected to have completed your research by that time. However, you should have a clear picture of what you are doing (which you should explain during your talk), and ideally you should have some preliminary results (where applicable).