Final Project
AR/VR implementation with an input and interaction focus + research paper · groups of 3–4
This page is a living document. More information will be added as we move forward. Check Canvas for the latest updates and official deadlines; Canvas is the final arbiter.
Project Requirements
Every project in CS 465 must:
- run in AR or VR on a headset;
- make input and interaction its central contribution — unimodal or multimodal, this is a multimodal interaction class;
- be human-centric and include an evaluation;
- be grounded in peer-reviewed material, either building on existing research or improving interaction.
You are not required to have a novel approach (though it is always encouraged). Evaluating existing approaches, or asking different questions about them, is welcome. You will need CITI training. If you want to publish later, you will need an approved IRB protocol (of interest to students working with other faculty).
Project categories
I will present project ideas in class. The categories are:
- A multimodal input technique in VR (e.g., gaze + pinch, speech + gesture).
- Interaction in passthrough MR or optical see-through AR.
- Evaluating an input method (e.g., a 3D Fitts' law study, a gesture elicitation study).
- Multimodal input with an LLM as part of the interaction (e.g., a speech or LLM agent acting in the XR scene).
You may also work on a problem with me, with your own faculty advisor, or with another faculty member, as long as it fits the requirements above. Talk to me early so we can scope it to the course deliverables.
Project Proposal
Submit a 2–3 page proposal with at least 3 references (peer-reviewed articles) about the project you want to work on. Submit a PDF that contains:
- Group members: list all members and describe what each person will be doing
- Introduction: one paragraph
- Motivation: one paragraph explaining why this is important
- State of the art (background): make the case for where we are currently and why you are working on this problem
- Description of the project: what you are planning to do
- Technology: the headset, engine, input devices, and any models you will use
- Deliverables: what you will deliver at the end of the semester (be specific)
- Contribution: what the contribution of this project would be
Submit the PDF in Canvas. You can use LaTeX or a word processor for the proposal, but the final paper and the checkpoints must be in LaTeX.
Deadlines
See Canvas for the official deadlines. These are the planned dates; this site is updated more slowly than Canvas.
| # | Deliverable | Planned date |
|---|---|---|
| 1 | Groups finalized | Thursday, January 28, 2027 |
| 2 | Project proposal | Friday, February 19, 2027 |
| 3 | Checkpoint 1 | TBA (see Canvas) |
| 4 | Checkpoint 2 | TBA (see Canvas) |
| 5 | Presentations (in class) | April 27 – May 6, 2027; format TBD |
| 6 | Final submission | TBA (see Canvas) |
| 7 | Borrowed equipment returned | Last week of classes |
Core Rubric
Here is the core rubric; an expanded rubric will be added later. Even if you won't run human-subject experiments, the CITI certificate is required, as this class touches on human-subject research. Without CITI training, you will receive a zero on your final project.
Final project: 350 points
- Proposal: 5 points
- Checkpoint 1: 15 points
- Checkpoint 2: 20 points
- Project (programming, experiment, and research paper): 310 points
Project breakdown (out of 310 points):
- Research paper (must use the template): 70 points
- Experiment design / system design: 25 points
- Evaluation: 25 points
- Prototype, running on the headset: 150 points
- Presentations: 20 points
- GitHub repository (or similar): 5 points
- Files (readme files in the required places, e.g., Canvas, GitHub): 5 points
- Videos (other than the presentation): 5 points
- Group member evaluation: 5 points
- Overall evaluation: 10 points
Research paper breakdown (70 points):
- Overall paper: 25 points
- State of the art: 20 points
- Methods and evaluations: 20 points
- Prototype description and reproducibility: 5 points
- Limitations, future work, and conclusion: 5 points
Grades are scaled to the number of group members. The professor reserves the right to deduct points for other problems not listed here. Failure to use LaTeX in the final paper costs 25 points, and points are also deducted at checkpoints.
Groups
Projects are done in groups of 3 or 4. Any other group size must be approved by me; hardware is the limiting factor. Students form their own groups, and I can assist if needed. Prototypes are weighted by the number of group members, and I will ask questions about the prototype and other items to assess that contributions are similar across members. No free riders.
| Group size | Peer-reviewed articles in the final report | Evaluations | Research report length |
|---|---|---|---|
| 3 | At least 24 | Three types: one controlled study, one of your choice, one selected by the professor | 12–14 pages + references |
| 4 | At least 32 | Three types: one controlled study, one of your choice, one selected by the professor | 12–14 pages + references |
For any group size, the maximum is 16 pages (but I don't need long reports). Appendices are welcome only when needed.
Checkpoints
To avoid leaving the project for the last few weeks, there are checkpoints. Each checkpoint includes:
- A video (3–5 minutes) discussing the current state and showing what has been accomplished, running on the headset
- A video discussing the code (3–5 minutes)
- All updated source code and documents pushed to GitHub (you may push every day if you like)
- A readme in GitHub with links to the videos and to the LaTeX files; additional documents may be included
- Each checkpoint has its own requirements, listed in its Canvas assignment
Videos are YouTube unlisted links in the readme.
About LaTeX
The research paper, starting with Checkpoint 1, must be written in LaTeX. I have never had anyone lose points for formatting when they used LaTeX. Use Overleaf, or install it locally: MacTeX on macOS, TeX Live on Windows and Linux. Any editor works (VS Code, Sublime, Vim, Emacs). Templates are provided on Canvas.
Final Project To-Do List
- Final report using the template provided. Length depends on the group size and the system contribution. More information will be given in class in the lecture about writing research papers in HCI.
- Deliver all files in the structure below.
- Deviation from the required work costs points. Not using the provided or an approved template costs points. Not delivering a professional-looking document costs points.
How to upload the final project
Upload a zip file. If the zip is too large, a working link to a downloadable file is required; test it with an anonymous link. If your link doesn't work, you lose points until I can get the file, and those points are not recovered.
- A top directory with the name of your group, containing:
- the final report PDF;
- a readme with links to the videos and the URL of your repository;
- a directory "Latex Source" with all the LaTeX source;
- a directory "PDFs-LiteratureSurvey" with the PDFs you found (exclude books and websites);
- a directory for the source code, or a text file with the GitHub link if it doesn't fit;
- the headset build (APK) or a link to it.
Videos
- Short video: no more than 5 minutes, ideally 3. A high-level overview of the project, running on the headset. Make it really cool.
- Presentation video: as long as you need to explain the project; no more than 30 minutes, ideally 12.
- Programming video: explain the code.
Provide links in the readme to view them and to download the actual video files. Verify with me if you are not sure; don't assume.
Programming Language and Engine
You can use any language or engine, but it must deploy to the headset. Most students use Unity; some use Unreal; a few have tried Godot. Support is provided for Unity only.
Instructor and TA Final Review
You may be required to present your final project to the instructor and/or the TA; a schedule will be provided. Individual members may be asked questions about the code to understand individual contributions. I reserve the right to change an individual's grade if the contribution was not similar to the rest of the group.
Equipment Available
- Your group can request a VR head-mounted display (Quest 2 or Quest 3, depending on availability). For online students, we do our best to provide one to each student.
- Logitech pen (one per group, limited quantities).
- In certain cases, additional devices: Leap Motion, Tilt Five AR glasses, and a few others.
- In extremely limited quantities, a laptop loan for online students (not guaranteed).
- Possibly a HoloLens 2.
Group Evaluations
Outside the final project grade, there are group evaluations (about three during the semester): an individual form where you tell us about the progress of your group.
Projects You Can Build From
The NUILAB has projects you can build from. We keep this source code private until we are ready to release it, so anything you do may be used (with acknowledgment) in our future work, and you may not distribute the software without my authorization.
- Kitchen AR Game — open source; HoloLens 2, notification interaction research; may be converted to Quest (Unity)
- VR Sketching (Unity)
- Wind Educational Project for VR (Unreal)
- Forest Bathing (Unreal)
- Assembly VR Task (Unity; old code, may need updating)
Older student projects: github.com/NUIG.