Delivered at:: School of Software Engineering

Course type:: Elective course

When:: 4 year, 1-3 module

Instructor

Manakhov, Pavel

Full Syllabus

Abstract

The goal of this multidisciplinary course is to introduce students to the process of designing spatial user interfaces, more specifically 3D UIs of augmented and virtual reality apps. Iterative human-centered design process is the basis of all course’s assignments meaning that students, while working on their projects, will go through the following stages: (1) exploring the design space/refining a 3D UI, (2) creating/modifying an interactive prototype in Unity, (3) conducting user experience (UX) evaluation, (4) analyzing the evaluation data and returning to the 1st stage with new knowledge.

Learning Objectives

Introduce students to the process of designing user interfaces of AR/VR apps

Expected Learning Outcomes

Know the difference among augmented, mixed, virtual, and other types of realities
Be familiar with application areas of immersive tech
Be familiar with the current XR trends
Appreciate the need of prototyping UIs
Understand the classification of 3D UI prototyping methods
Acquire practical skills in prototyping 3D UIs
Appreciate the difference between interaction design and user interface design
Understand the concept of the user interface
Appreciate the difference between screen interfaces and 3D interfaces
Understand the concept of usability
Understand the context of use concept
Know the Goal-Directed Design process
Be prepared to create personas
Be prepared to write usage scenarios
Understand the concept of input/output modality
Understand the classification of various input/output devices used in 3D UIs
Understand how human's eye perceives depth
Be familiar with various types of positional tracking technologies
Understand the concept of AR Cloud
Understand the concept of implicit human-computer interaction
Understand the difference between interaction technique and interaction paradigm
Be familiar with current interaction paradigms for AR/VR
Know different forms of design knowledge such as design patterns and design rules
Know some 3D interaction design patterns
Be able to find appropriate design knowledge for a particular project
Understand the concept of usability goals
Know how to derive usability goals
Understand the concept of the design space
Understand the concept of design questions
Appreciate the need of exploring a number of design alternatives
Appreciate the difference between conceptual and detailed design
Know how to apply different creativity tools
Be prepared to conduct a brainstorming session
Know different forms of design representation
Acquire practical skills in designing a 3D UI for a given context of use
Acquire practical skills in presenting design concepts
Acquire practical skills in making design changes to eliminate found interaction problems
Understand the concept of research questions
Be familiar with various user research and usability evaluation methods
Be familiar with various ways of looking for study participants
Acquire practical skills in forming recruitment criteria
Acquire practical skills in recruiting human subjects
Understand Norman’s Stages-of-action model
Understand the concept of usability problems
Understand the difference among inefficiencies, feature requests, and software bugs
Acquire practical skills in planning usability testing
Appreciate the difference among moderation techniques for usability testing
Acquire practical skills in conducting usability tests
Acquire practical skills in making sense of data gathered through usability testing
Acquire practical skills in representing results of a usability study
Know the peculiarities of conducting usability tests for AR/VR
Know theories which explain the causes of motion sickness
Be familiar with web tools which are used to create XR apps
Be familiar with native tools which are used to create XR apps

Course Contents

Interaction Design for AR/VR: An Introduction
The concepts of augmented, mixed, virtual, and extended realities. Brief history and the current state of immersive tech. Use cases of immersive tech. XR trends. About this course.
Overview of 3D UI Prototyping Tools and Techniques
Why prototyping does matter. The classification of 3D UI prototyping methods. Bodystorming. Using 3D-painting VR apps to sketch 3D UIs. Using specialized no-code solutions to create interactive prototypes. Brief overview of code-based solutions.
The Essentials of Interaction Design
The concept of the user interface. The concept of the spatial interface (the 3D user interface). The concept of the interaction. The context of use concept. The concept of user experience. The concept of usability. Mark Hassenzahl’s Model. Goal-directed design process. Design-informing models.
Hardware Technologies for 3D UIs
The concept of modality in HCI. How we perceive depth. Vergence-accommodation conflict. Overview of computer-human modalities and related output devices. The classification of positional tracking technologies. Overview of human-computer modalities and related input devices. Implicit human-computer interaction.
Interaction Paradigms for XR
The concept of interaction techniques. The concept of interaction paradigms. Overview of modern interaction paradigms for AR/VR.
Overview of 3D UI Design Patterns
Various types of design knowledge. Overview of design patterns used in 3D UIs.
The Process of Design Creation
The concept of usability goals. Why an exploration of design alternatives is necessary. The concept of design questions. The process of synthesizing solutions and comparing them. The difference between conceptual and detailed design. Creativity tools. Types of design representations.
Introduction to User Research
The concept of research questions. The classification of user research methods. The classification of usability evaluation methods. Recruitment criteria. Ways of looking for participants.
Essential Usability Evaluation Concepts
How people interact with digital things. Norman’s Stages-of-action model. The concept of usability problems. Interaction problems of other kinds.
Usability Testing: Preparation
Planning usability tests. Data collection techniques.
Usability Testing: Moderation & Data Analysis
Overview of moderation techniques for usability testing. Moderating usability tests. Usability data analysis.
Conducting Usability Tests of XR Apps
Recruitment. Lab layout. Study design. Fitting and hygiene. Communication with participants. Data collection.
Overview of Dev Tools for Creating XR Apps
WebXR. Tools & frameworks for developing XR apps for the web. Unity XR. Overview of libraries for creating 3D UIs in Unity.
Fighting Motion Sickness in VR
Overview of the theories explaining the causes of motion sickness. Design best practices which help to avoid having motion sickness.

Assessment Elements

Homework 1 (HW1)
The assignment implies creating a simple prototype using Unity for Windows/macOS. A description of the prototype's UI and its behavior is given in a form of a list of requirements.
Homework 2 (HW2)
The assignment implies creating a Unity prototype for an XR device using provided by the professor middleware. A description of the prototype's UI and its behavior is given in a form of a list of requirements.
Team Project 3 (TP3)
This assignment implies proposing design concepts of a complete app by a given set of functions, user and/or interactions models, and best practices found in the analogous apps. The design concepts are prepared by every team and presented on the 7th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the proposed concepts with the team and gives tips on how to improve them.
Team Project 4 (TP4)
This assignment implies forming recruitment criteria which later will be used to look for participants of a usability study. A draft of the criteria is prepared by every team and presented on the 7th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminar the professor discusses the draft with the team and gives tips on how to improve it.
Team Project 5 (TP5)
This assignment implies developing a test plan to evaluate the team’s app. A draft of the plan is prepared by every team and presented on the 10th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the draft with the team and gives tips on how to improve it.
Team Project 6 (TP6)
This assignment implies creating a Unity prototype according to the design proposed earlier (TP3) and conducting usability tests in accordance with the plan prepared during the previous assignment, analyzing usability data gathered throughout the tests, and writing a report that describes all interaction problems found during the study. A draft of the report is prepared by every team and presented on the 12th or 13th seminars, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the draft with the team and gives tips on how to improve it.
Practice Activities (PA)
Final Exam (FE)
Oral examination, 2 questions.

Interim Assessment

Interim assessment (3 module)
0.3 * Final Exam (FE) + 0.1 * Homework 1 (HW1) + 0.1 * Homework 2 (HW2) + 0.08 * Practice Activities (PA) + 0.09 * Team Project 3 (TP3) + 0.04 * Team Project 4 (TP4) + 0.09 * Team Project 5 (TP5) + 0.2 * Team Project 6 (TP6)

Bibliography

Recommended Core Bibliography

Cooper, A., Reimann, R., Cronin, D., & Noessel, C. (2014). About Face : The Essentials of Interaction Design (Vol. Fourth edition Alan Cooper, Robert Reimann, David Cronin, Christopher Noessel). Hoboken: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=827342
Hartson, R., & Pyla, P. S. (2012). The UX Book : Process and Guidelines for Ensuring a Quality User Experience. Amsterdam: Elsevier Ltd. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=453819
LaViola, J. J. (2017). 3D User Interfaces : Theory and Practice: Vol. Second edition. Addison-Wesley Professional.
Schmalstieg, D., & Höllerer, T. (2016). Augmented Reality : Principles and Practice. Boston: Addison-Wesley Professional. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1601643

Recommended Additional Bibliography

Holtzblatt, K., & Beyer, H. (2015). Contextual Design : Evolved. Morgan & Claypool Publishers.
Kuniavsky, M., Goodman, E., & Moed, A. (2012). Observing the User Experience : A Practitioner’s Guide to User Research (Vol. 2nd ed). Burlington: Morgan Kaufmann. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=472263
Michael E. Wiklund P.E, Jonathan Kendler, & Allison Y. Strochlic. (2016). Usability Testing of Medical Devices: Vol. Second edition. CRC Press.
Norman, D. A. (2013). The Design of Everyday Things : Revised and Expanded Edition (Vol. Revised and expanded edition). New York: Basic Books. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=658202
Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing : How to Plan, Design, and Conduct Effective Tests (Vol. 2nd ed). Indianapolis, IN: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=233103

Bachelor’s Programme 'Software Engineering'

Contacts

Accredited by the Engineering Commission of ABET

Interaction Design for AR/VR