Term Project

• Project Part 1
• Project Part 2
• Project Part 3
• Project Part 4
• Project Part 5
• Project Part 6

Project Summary

Throughout the course, you will be involved in a group design project. Each project team should consist of two or three students, although exceptions may be made in consultation with the instructor. The purpose of the project is to gain experience in applying some of the concepts and techniques from the class to a real-world problem.

This semester, the suggested project theme is game-playing. You are free to propose another interactive application provided it satisifies the general requirements of interaction outlined below. For the reminder of this document, the term "game" will refer in general to any form of project.

The game you choose may be already implemented elsewhere, in which case, you are free to re-use software components with proper acknowledgement, it may be described but not available as a software implementation, or it may be something completely of your own invention. In any event, we will not be grading the game itself, but rather, your ability to design, develop, evaluate, and refine an effective interface that supports a compelling level of interaction, building on at least one (preferably more) of the following themes of interaction:

• physically distributed: multiple players in different rooms, sharing game context over network communicaiton
• bi-manual: using expressive gestures that leverage skills acquired from the physical world, taking advantage of new sensing affordances of the Wii motes or additional sensors you might propose to use both hands, and/or other limbs, in a manner that communicates more information to the computer than would a more conventional interface
• multimodal: exploiting the senses of vision, audition (hearing), and touch

To simplify your task (and hopefully, enrich the outcome), we will provide you with a physical I/O device, the Nintendo Wii remote that can be used to acquire a wide range of user inputs and generate both audio and force feedback output. The inputs include a three-axis accelerometer, which can sense linear motion and tilt, an optical sensor, which, in conjunction with a simple bank of IR LEDs, can determine pointing angle, and of course, a number of push-buttons. These devices will communicate wirelessly with Bluetooth-equipped PCs as described here.

Through the various stages of the project, in particular as you prepare to build your early prototypes, you will likely want to consult the list of project resources we have prepared. Examples include sample code for network communicaiton, pointers to prototyping environments, and gaming libraries.

Also, the Wii website has a number of video demonstrations of various application that may inspire your imagination.

As the term progresses, your system will evolve from a paper description, to a simple prototype, and finally to a functional physical system. There are seven components of the project that will be graded as described in the course outline. Every component (apart from your final presentation) is associated with a link in the team's project notebook, which will be downloaded as a snapshot at each due date. Your notebook entries for each project component must be organized in such a manner that a concise overview is presented to a reader who only has approximately five minutes to gain an understanding of your work. You may provide as much supplemental material as you wish, linked to each entry, but the main entry must make sense on its own. The detailed requirements for the notebook components will be specified here.

While your work will be assessed as a team effort, individual marks may be assigned based on the graders' assessment of a disproportionate contribution of effort or significantly varying quality of work by one or more members. As the evaluation of group work is a contentious process, we are requiring that you submit, in-class on each due date, a hardcopy summary of individual contributions for each component, signed by all members of your team. It is understood that various elements (e.g. web page setup, sensor processing, user testing) of the project will involve the combined efforts of multiple group members; nevertheless, the individual contributions of each member must be described in reasonable detail. This is being done both to reward exceptional efforts as appropriate and also to ensure that group members maintain clear communications with each other throughout the term.

Project Notebook

All reports and information on your project is to be collected into a Web-based notebook, maintained separately by each project team. You may wish to review some of the notebooks from previous years for examples.

Each group will designate one "webmaster" who will be responsible for maintaining the notebook on a publicly accessible web server. The location of the top-level URL for the notebook should be communicated to the instructor (by myCourses mail) no later than January 26. All of your project information must be stored under this directory. Its contents will be retrieved automatically by wget at 10am on the due date of each term project component. For important information bout the web notebook see these helpful guidelines prepared by previous TAs. Be forewarned: just because your web pages look right in your account does not mean they will appear correctly when transferred, and this may result in you losing marks.

Project Part 1

Defining the project

This assignment is intended to help you formalize your project plan and to develop evaluation criteria appropriate to a particular system. It will also help you gain an understanding of usability problems that can guide design decisions you will make for your new system. The latter will be accomplished through the summative evaluation, that is, evaluation of a system after it has been fully developed and is being used in the field.

Your task is to propose the overall structure of the game (or interactive application) that will constitute the basis for your work through the rest of the semseter. This includes a high-level definition of the game itself, a description of the interaction between players and the game environemnt, as well as players with each other, and the types of inputs and outputs required to make the game an effective experience. Your proposal should capitalize on the affordances of the hardware we expect to make available; for example, a game of pong, in which the players need only rotate a knob to slide their paddles back and forth across the screen, is of limited interest. We strongly encourage you to take advantage of bi-manual (two-handed) input and use multiple modalities (graphics, audio, haptics) of output.

The required components and grading scheme for this assignment are as follows:

• a concise description of the game environemnt and rules of play [out of 2]
• a discussion of requirements (whether software tools or hardware resources) you expect are necessary to carry out the development and a description of any software (e.g., open source game code) or hardware (beyond the Wii mote and Bluetooth dongles that you will be provided) that you have identified (include URL to product information and cost, as relevant) [out of 2]
• a brief user manual (keep in mind your intended audience) that summarizes how players interact with the game [out of 3]
• express two or more criteria that will help you to judge whether playing your proposed game is more engaging and/or enjoyable than similar classes of games available in either computer form or other environments [out of 3]

You may find it useful to consult last year's project notebooks to get a sense of expectations, both in terms of organization and depth of content.

Project Part 2

Low-fidelity prototype and evaluation plan.

The objective of this part of the project is for you to design and implement a low-fidelity mockup of your proposed system, based on team discussions that led to the completion of Part 1, instructor/TA feedback, and your experience in working through a previous prototyping effort. You will also prepare an evaluation plan that will be the basis for Part 3 of this project.

For this assignment, your team must:

• Determine a set of benchmark tasks or a usage scenario, as well as a set of usability goals that will be used for evaluation purposes. The graders will want to see that your chosen tasks demonstrate important functionality of the system and allow you to determine whether the key interactions are working successfully.
• Build a paper prototype or physical mock-up of your new design. Since the input device that will be used for the final system is already well defined, your prototype is likely to focus on the screen displays, sound, and vibration effects that would result from user interaction with the system. Keep in mind that the mock-up need not be overly detailed, in particular as you want to focus on the "big picture" at this stage. It will likely be useful (if not imperative) to employ "Wizard of Oz" techniques, with members of the design group animating the live operation of the envisioned system, possibly including sound effects, for this purpose.
• Design the test materials as relevant to evaluating your prototype. These may include an observer briefing, user introduction, pretest questionnaire, user or training documentation, a set of test tasks, data collection sheet, posttest questionnaire, and a test script. In addition to the material you saw in Howard Kiewe's guest lecture, some helpful examples can be found in this excellent overview on Usability Evaluations and (although oriented specifically to the evaluation of web sites), the W3C Web Accessibility Initiative Usability testing questions.
• Apply the usability testing procedure on at least two potential users (from an actual expected user population) and document the results. These will be helpful as you modify the design of your system throughout the term.

The required components and grading scheme for this assignment are as follows:

• benchmark tasks or usage scenario and usability goals [4 pts]
• illustrations or photographs of your paper prototype or physical mockup [2 pts]
• samples of test materials that you employed [2 pts]
• a brief summary of test results as well as any important lessons learned from the user testing [2 pts]

Project Part 3

Computer Prototype and Evaluation Plan

At this point, you are ready to give users a clearer sense of the look and feel of your application. This means that your prototypes will evolve from paper, cardboard, and styrofoam, to a second version that runs on a computer.

However, you are equally concerned with learning about any design problems at an early stage, before you have invested an inordinate effort in development. Thus, in addition to implementing your second prototype, you will now formulate a detailed usability test plan, expanding on your effort from Part 2. Rather than conduct the usability testing yourself, this will be the task of another team during Part 4 of the project.

For this assignment, your team must:

• Implement a computer-based prototype that allows users to evaluate the "look and feel" of your system with minimal or no use of Wizard of Oz techniques. While this prototype should be based on your description from earlier parts of the project, it is understood that the design will include modifications in light of TA feedback and initial user testing. Keep in mind that your prototype does not need to be functionally complete, nor is there any requirement that it use the Wii hardware at this stage. It should simply be sufficient to give a reasonable (albeit imprecise) impression of the intended interaction to candidate users.

  Your prototype must be executable on general-purpose workstations (either Windows, Linux, or Mac) available in the Trottier Engineering Building. Apart from the Wii controllers, sensor bars, bluetooth dongles (and possibly other items agreed to be provided by the instructor), it will be your responsibility to provide the test group with whatever additional resources are necessary to run your prototype.

• Provide a usability test plan for evaluating your new prototype. Note that the testing will be conducted with minimal or no assistance from you, so you must be very explicit as to exactly what evaluation should occur.

  Some of the options you may wish to consider:

  • A formal "laboratory" experiment in which some quantitative measurement (e.g., learning time before user is able to use the system as intended, number of mistakes users make before invoking a specific function, as instructed) can be made. Target numbers should be justified.
  • A cognitive walkthrough, which involves going through the operation of an interface, asking, at each point:
    • what actions the user needs to make
    • how the interface supports these actions
    • what assumptions are made about the user's knowledge and background related to the system

    The point of such an exercise is is to simulate a user's problem-solving process at each step in the the dialogue to see whether the desired/expected outcome occurs. In this case, you must describe in detail several common tasks that a user might want to perform with your system so that the evaluation team can "walk through" the appropriate operations. You should not have to provide instructions of the form: "now press this button and wait for the green light to flash"!

  • A heuristic evaluation of the interface. For ideas of what factors are worthy of consideration, you may wish to consult Jakob Nielsen's "top ten list" of Usability Heuristics. Note that the evaluation team will have the freedom to disregard your suggested heuristics if they find them to be inappropriate as a means of assessing your system.

The required components and grading scheme for this assignment are as follows:

• an updated user manual that summarizes how to install/run your software and how players interact with the game [2 pts]
• the prototype itself, which will be evaluated according to its functionality [2 pts] and how well it fulfills the objectives of the course project [2 pts]
• Usability Evaluation plan [4 pts] (2 for each of two best exercises)

Project Part 4

Formative Feedback

The objective is to give you experience in evaluating another group's design, by carrying out the exercises they have specified in Part 3 of their project notebook.

Your team must:

• Execute the evaluation exercises proposed in Part 3 of the notebook for the project you have been assigned to evaluate
• Document and analyze your evaluation results, commenting on whether the group has chosen reasonable evalution criteria for the intended tasks and whether these have been achieved; if you feel the exercises do not adequately serve to evaluate the strengths and weaknesses of the project, you may carry out additional exercises, provided that you justify their importance. Note that this does not mean you can ignore the exercises proposed by the development team (whose project you are evaluating).
• Determine what changes to the prototype system are suggested by your evaluation and describe how these would improve the design.

Team Assignments:

• Archerii evaluates Showdown
• Showdown evaluates 1st Edition
• 1st Edition evaluates WiiDutch
• WiiDutch evaluates Harry Potter Showdown
• Harry Potter Showdown evaluates Scratch Match
• Scratch Match Showdown evaluates Archerii

To assist your team in carrying out these exercises, the development team should designate one of its members to act as a liason to your group. This is the individual you should contact for any assistance, as required, related to the setup or operation of the prototype for evaluation.

The results and analysis of the evaluation exercises you conduct, along with discussion of the relevance of each exercise to the intended tasks, should be included in your Web notebook.

The grading scheme for this assignment is as follows:

• Completion of all evaluation plans [1 point]
• Adequate responses to questions, heuristics, or measurements [4 points]
• Independent critique of the system/prototype, with suggestions or recommendations (this should be independent of the evaluation plans provided by the other group) [3 points]
• Critique of the evaluation plan. [2 point]

Project Part 5

Alpha System

Now that you have completed an initial prototype, (hopefully) carried out your own evaluation of the interface and obtained the results of a formal evaluation conducted by another team, you are now ready to implement a first working version of your design.

Summarize the feedback provided by the evaluation team and note the major strengths and weaknesses of your interface identified by their report. If any of these observations surprised you, comment on how (and why) they differed from your earlier testing results.

Describe the changes that were made to your initial prototype in response to your own observations as well as those from the evaluation team. Explain the motivation for each change and its effect in terms of satisfying your design critieria or overcoming any limitations exposed by the evaluation.

Revise your user manual as appropriate. Indicate (ideally using a clearly defined colour scheme) changes that were made as a result of the feedback from (a) the TA (b) evaluation team (c) added functionality.

The required components and grading scheme for this assignment are as follows:

• description of interface evolution [4 pts]
• an updated user manual that summarizes how to install/run your software and how players interact with the game [2 pts]
• the prototype itself, which will be evaluated according to its functionality [2 pts] and how well it fulfills the objectives of the course project [2 pts]

Project Part 6

Project Presentation
Week of April 7

The in-class presentations will be an opportunity for you to showcase the results of your work and allow the rest of the class to play the game you have implemented. The presentation should begin with a brief (maximum five minutes) overview, explaining the rationale behind the game you chose to implement, with respect to the affordances of the input and output devices available to you. You should also include an analysis of the results your group obtained relative to the design goals you set in Part 1. Any necessary instructions regarding the game should be conveyed at this time as well. The remaining time will be allocated for game-play by other class members and (simultaneously) a question and answer period.

Given the different logistical needs of the various groups, we will attempt to schedule the presentations so that the class does not have to move back-and-forth between different locations, whether classroom (MC12), Trottier Building, or SRE lab. In order to facilitate this, if your group requires anything non-default (either equipment, location, or other) please indicate by posting your requirements on myCourses. We expect that groups will pool their Wii resources as appropriate in order to ensure, for example, a sufficient number of Wii remotes and bluetooth dongles. In the ideal, we will be able to compress all presentations into a single date.

Grading will be based on the following four factors:

• Technical presentation: [2 pts]
• Response to questions: [1 pt]
• Fulfillment of at least one of course project themes (physically distributed, bi-manual, multimodal): [2 pts]
• Effectiveness of user interface for the game itself: [5 pts]

Beta System
Due: April 10

For the beta system, you will likely be making a number of revisions and improvements to your system based on feedback you have received from the TAs as well as results of further user testing you have conducted. In addition, you should be implementating any important functionality in the game-play that was not included in your alpha system. The deliverables and grading criteria are identical to those from Part 5 (Alpha System) and must be posted in your web notebook.

Last updated on 30 March 2008
by Jeremy Cooperstock