Course Schedule

Date Class Readings Homework
March 28 Introduction. General class intro: goals, format, assignment overview

Introduction to mixed and augmented reality. History of MR technology. VR & AR, Hard AR vs soft AR. Overview of HCI issues and class. Discussion around what is Mixed Reality? What is Augmented Reality? Where is it appropriate? AR in film and popular culture

Guest Speaker: Prof. Mark Billinghurst
Readings for next week:

HoloDesk: direct 3d interactions with a situated see-through display

Augmented reality: a class of displays on the reality-virtuality continuum

A review of head-mounted displays (HMD) technologies and applications for consumer electronics

Assignment 1 (due 4/4):

Find an existing augmented reality system (check out at least 5-10 before picking) Write 1-2 pages that:
  • Provide a link to the video, app, page, system, etc...
  • Describe the system & the key interaction techniques
  • Critique the video
    • Is it a real application or concept video?
    • Does it need to be augmented reality? does augmented reality improve the experience?
    • What works & doesn't work?
    • How would you improve the interaction?
April 4 Displays. Types of displays. Hand-held AR. Spatial AR. Optical and video see-through glasses. See-through displays. Discuss issues specific to each of these modalities.

Non-visual modalities

Input and Output hardware.

Readings for next week:
How to read a paper

A Review of 3D Gesture Interaction for Handheld Augmented Reality

Marker tracking and hmd calibration for a video-based augmented reality conferencing system

Parallel tracking and mapping for small AR workspaces

KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera

  • Need finding. What is the interesting opportunity? Explore the set of user, context, and task/opportunity that you want to address and be able to articulate that. We're interested in the why as much as the who/what.
  • Initial project ideas. We don't expect you to have fully baked solutions but you should be brainstorming generally how you might address the opportunity and exploring the possibilities. Have at least three potential directions.
  • Deliverables - 1-2 pages describing the above, create a slack channel, be able to discuss/present in class
April 11 Tracking and Registration. How do we understand the world and provide that information to the person and system. Introduction to existing tools like ARToolkit, Vuforia, Wikitude, and more advanced work like PTAM.

Model on a Marker. Implications for scene understanding in AR.

Project teams due

Readings for next week:
REVEL: tactile feedback technology for augmented reality

Twiddler typing: one-handed chording text entry for mobile phones

Haptic Ring Interface Enabling Air-Writing in Virtual Reality Environment

GyroWand: IMU-based Raycasting for Augmented Reality Head-Mounted Displays
Model on a marker

For the group assignment, you need to write up a a proposal for the project you're selecting and be prepared to discuss:
  • Project title
  • Group members and roles (e.g. Will has experience with Arduinos and will be making our magic wand, Sean built image processing systems and will be making a face recognizer)
  • Project idea and motivation (opportunity/problem, users, context, task)
  • Project plan of attack with weekly milestones (e.g., a rough plan of how you will accomplish the work)
  • Initial literature review (Has this been done before? What prior work is relevant? If your project is different, how is it different?)
  • If you're creating a new contribution, like many of the papers, what might you be contributing (e.g. a study, a new concept, old concept in a new application)
  • What hardware, software, or other resources you might need, and how you plan to get them? Do you need glasses (HoloLens, ODG, Meta), handheld displays, Tango or occipital depth cameras? Will you use Wikitude, Vuforia, or some other tracking system?
  • Success criteria (How will you know if you succeed? I will grade you partially on whether you met your own criteria.)

Note that along with a presentation and video of the project, one of your final deliverables will be writing up your work in the ACM SIGCHI Extended Abstract format. The official final for this class is June 7th from 7-10pm, when you'll present your projects to each other and guests.
April 18 Rendering & Authoring, Shadows, Relighting, Occlusions. How are virtual objects and affordances perceived as physical? Should they be?

Discuss draft project proposals

Guest Speaker: Dr. Kent Lyons
Readings for next week:
Perceptual Issues in Augmented Reality Revisited
Start on design sketches and exploration
April 25 Perceptual & Cognitive Issues. Visibility, object relationships, registration errors, layer interference, individual differences, depth perception cues.

Design implications and strategies

Final project proposals due

Guest Speaker: Dr. Oliver Aalami, M.D.
Readings for next week:
An introduction to 3-D user interface design

New Directions in 3D User Interfaces

Project Design
May 2 Interaction Techniques. 3DUI, Selection & manipulation, Tangible AR, Wayfinding, System Control,

Evaluation techniques for MR systems.

Guest Speaker: Dr. Ron Azuma
Readings for next week:
DART: a toolkit for rapid design exploration of augmented reality experiences.

Authoring Tools in Augmented Reality: An Analysis and Classification of Content Design Tools

Authoring 3d hypermedia for wearable augmented and virtual reality

Tiles: A mixed reality authoring interface

Immersive authoring of tangible augmented reality applications

Kharma: An open kml/html architecture for mobile augmented reality applications
Project Design & Prototype
May 9 Authoring. Creating AR systems, content in AR systems, out of system and in-situ authoring. AR-specific infrastructure. Unity.

Guest Speaker: Charlie Sutton
Readings for next week:
View management for virtual and augmented reality

An augmented reality x-ray system based on visual saliency

The Personal Interaction Panel - a Two-Handed Interface for Augmented Reality

Smart Vidente: advances in mobile augmented reality for interactive visualization of underground infrastructure

Image-driven view management for augmented reality browsers
Project Checkpoint
May 16 Visualization: Types of representation, situated visualization, situated learning, situated analytics. Spatial data. ZUIs in AR.

Project updates in class

Guest Speaker: Brian Mullins
Readings for next week:
Of vampire mirrors and privacy lamps: Privacy management in multi-user augmented environments

Security and privacy for augmented reality systems
Project Checkpoint II
May 23 Applications: Enterprise, Consumer, Medicine, Manufacturing, Education, Games, Art, Media, & Humanities

Ethical issues

Future of Mixed and Augmented Reality
No Readings Work on presentation and documentation
May 30 No Class (Memorial Day) No Readings Keep working on the project and presentation
June 7 Final Project Presentations No Readings
Having problems? Let us know!