An introduction to human-computing techniques. Theory and practice of crowdsourcing systems. Applications to citizen science and artificial intelligence.


Initiatives such as Amazon Mechanical Turk and OpenStreetMaps have proven that crowd-sourcing and human-computing techniques are valuable ways to both analyze and annotate large datasets that require human expertise as well as to solve problems that are difficult to treat with classical computer algorithms. Distributed computing platforms such as a SETI@home or Scientific games like Galaxy Zoo, Foldit and Phylo illustrate the potential of these techniques for studying, mining, and processing scientific data.
This course aims to provide an introduction to the emerging field of human computation and to discuss its impact on the scientific discovery processes. We will study existing platforms, review related theoretical frameworks, and study the impact of this technology in the field of artificial intelligence. Finally, we will discuss the perspectives offered by new mobile computing technologies.



Office hours:

Prof. Jérôme WaldispühlThursday 14:30-16:00ENGTR 3106

Course Webpage:

Course Material:
All the material needed for this class will be available on the public course web page. There is no required textbook. Although, we recommend the following textbooks to deepen the material presented in class:

Lecture slides will be made available in PDF form on the course web page.

Your final grade will be calculated as follows:


Release DateDue DateAnnounce
Nov 7Nov 16A description of the projects including task assignment, description and timeline is now available.
Oct 26Oct 27A memo to prepare the class project is now available.
Oct 18Oct 20Class of Oct 20th is cancelled. Replacement will be announced on a later date.
Oct 16The project assignments is now available.
Oct 4Oct 9The project selection form is now available
Sep 27Paper to review are available HERE
Sep 6Welcome to COMP766!


Lecture 1Sep 6Syllabus. Introduction to Human Computation[Slides]
Lecture 2Sep 8Designing Human-computation Algorithms[Slides]
Chapter 2 of [LVA2012]
Lecture 3Sep 13Aggregating outputs[Slides]
Chapter 3 of [LVA2012]
Lecture 4Sep 15Routing human computation tasks[Slides]
Chapter 4 of [LVA2012]
Lecture 5Sep 20Know your audience[Slides]
Chapter 5 of [LVA2012]
Lecture 6Sep 22The Art of Asking Questions[Slides]
Chapter 6 of [LVA2012]
Lecture 7Sep 27Presentation of projects[Slides]
Lecture 8Sep 29Case study: Phylo (part 1)[Slides]
Lecture 9Oct 4Case study: Phylo (part 2)
Lecture 10Oct 6Invited talk: Pascal Nataf, Affordance Studio
Lecture 11Oct 11Invited talk: Olivier Tremblay-Savard, University of Manitoba.[Slides]
Lecture 12Oct 13Student paper presentation
  • Rudolf Lam: Accounting for Taste: Ranking curators and content in social networks
  • Mansha Imtiyaz: Toward a Learning Science for Complex Crowdsourcing Tasks
Lecture 13Oct 18Student paper presentation
  • Patrick Sejour Chassagne: Supporting Collaborative Writing with Microtasks
  • Kininyiruchi Echomgbe: Curiosity Killed the Cat, but Makes Crowdwork Better
  • Faizy Ahsan: Citizens for Science and Science for Citizens: The View from Participatory Design
Lecture 14Oct 25Student paper presentation
  • Mathieu Nassif: Designing Engaging Games Using Bayesian Optimization
  • Olamilekan Fadahunsi: Atelier: Repurposing Expert Crowdsourcing Tasks as Micro-internships
  • Richard Olaniyan: Pay It Backward: Per-Task Payments on Crowdsourcing Platforms Reduce Productivity
  • Akash Singh: Crowdsourcing Annotations for Websites’ Privacy Policies: Can It Really Work?
Lecture 15Oct 27 Project Discussion
Lecture 16Nov 1Student platform presentation
  • Mathieu Nassif: Galazy Zoo
  • Rudolf Lam: Malaria Spot
Lecture 17Nov 3Student platform presentation
  • Patrick Sejour Chassagne: Eyewire
  • Richboy Echomgbe: Duolingo
Lecture 18Nov 8Student plateform presentation
  • Olamilekan Fadahunsi: Folding@Home
  • Richard Olaniyan: Foldit
Lecture 19Nov 10Student platform presentation
  • Akash Singh: Turkomatic
  • Mansha Imtiyaz: Tomnod
  • Faizy Ahsan: Crowdflower
Lecture 20Nov 15Invited talk: Edward Newell, McGill University[Slides]
Lecture 21Nov 17Swarm computing[Slides]
Lecture 20Nov 22Restructuring Expert Attention
Lecture 22Nov 24Patterns of Online Collaboration
Lecture 23Nov 29The challenges of open science
Lecture 24Dec 1Limits and the Potential of Collective Intelligence
Note: This schedule is subject to modification.


Background & Pre-requisites
Good understanding of basic algorithms and artificial intelligence concepts. Basic progamming in Python.
Undergraduate students: You can register to this class with the permission of the instructor.

Policy on collaborations
We greatly encourage you to discuss the assignment problems with each other. However, these discussions should not go so far that you are sharing code or giving away the answer. A rule of thumb is that your discussions should considered public in the sense that anything you share with a friend should be sharable with any student in the class. We ask you to indicate on your assignments the names of the persons with who you collaborated or discussed your assignments (including TA’s and instructors).

Policy on final grades
I will use the same rules and formula for calculating the final grade for everyone. We understand that your performances may be influenced by many factors, possibly out of your control. However, that is the only way we can be fair. The only exceptions will be medical exceptions. In that case, I will require a medical note, which has to be also reported to McGill, and to be informed as early as possible. Failure to comply to these rules, may results in the impossibility to invoke a medical exception.

Policy on written reports
Due date/time, location/mode for returning your solutions, and accepted formats will be announced in class and indicated on the course web page.
Failure to return your assignment in time will results in penalties or even absence of grading.
Importantly, solutions that do not follow the requested format will receive a penalty. By default, we only accept PDF or TEXT files. Images (if any) must be embedded in a PDF. Do not compress your files. All files must open on LINUX SOCS workstations.
The quality of the presentation of your solution is very important. Unreadable material, cryptic notations, or bad organization of the material will results in penalties, and potentialy even an absence of grading. If you scan your hand-written solutions, it is your responsability to ensure that you submit a high-quality image (i.e. excellent luminosity, contrast, focus and resolution). The clarity of your explanations will also be an integral part of your final grade.

Policy on programming code
Questions in assignments may require you to write a Python program. We will provide, as much as possible, input and output data to test your programs. However, it will be your duty to ensure that your files compile on LINUX SOCS workstations. We will not grade programs that do not compile on these machines.
Submission of class files (instead of Java source files) will be considered as an absence of submission. Do not compress your files.

Use of French in assignments and exams
In accordance with McGill University’s Charter of Students’ Rights, students in this course have the right to submit in English or in French any written work that is to be graded.

McGill policies
McGill University values academic integrity. Therefore, all students must understand the meaning and consequences of cheating, plagiarism and other academic offenses under the Code of Student Conduct and Disciplinary Procedures. See this link for more information.


If you have any additional question, you can contact the instructor:

Jérôme Waldispühl
3630 University Street, Room 3106, Montreal QC H3A 0C6
(Phone) +1 514 398 5018

Based on a template from