• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
Bachelor 2020/2021

Interaction Design for AR/VR

Category 'Best Course for Broadening Horizons and Diversity of Knowledge and Skills'
Type: Elective course (Software Engineering)
Area of studies: Software Engineering
When: 4 year, 1-3 module
Mode of studies: distance learning
Instructors: Pavel Manakhov
Language: English
ECTS credits: 8

Course Syllabus

Abstract

The goal of this multidisciplinary course is to introduce students to the process of designing spatial user interfaces, more specifically 3D UIs of augmented and virtual reality apps. Iterative human-centered design process is the basis of all course’s assignments meaning that students, while working on their projects, will go through the following stages: (1) exploring the design space/refining a 3D UI, (2) creating/modifying an interactive prototype in Unity, (3) conducting user experience (UX) evaluation, (4) analyzing the evaluation data and returning to the 1st stage with new knowledge.
Learning Objectives

Learning Objectives

  • Introduce students to the process of designing user interfaces of AR/VR apps
Expected Learning Outcomes

Expected Learning Outcomes

  • Know the difference among augmented, mixed, virtual, and other types of realities
  • Be familiar with application areas of immersive tech
  • Be familiar with the current XR trends
  • Appreciate the need of prototyping UIs
  • Understand the classification of 3D UI prototyping methods
  • Acquire practical skills in prototyping 3D UIs
  • Appreciate the difference between interaction design and user interface design
  • Understand the concept of the user interface
  • Appreciate the difference between screen interfaces and 3D interfaces
  • Understand the concept of usability
  • Understand the context of use concept
  • Know the Goal-Directed Design process
  • Be prepared to create personas
  • Be prepared to write usage scenarios
  • Understand the concept of input/output modality
  • Understand the classification of various input/output devices used in 3D UIs
  • Understand how human's eye perceives depth
  • Be familiar with various types of positional tracking technologies
  • Understand the concept of AR Cloud
  • Understand the concept of implicit human-computer interaction
  • Understand the difference between interaction technique and interaction paradigm
  • Be familiar with current interaction paradigms for AR/VR
  • Know different forms of design knowledge such as design patterns and design rules
  • Know some 3D interaction design patterns
  • Be able to find appropriate design knowledge for a particular project
  • Understand the concept of usability goals
  • Know how to derive usability goals
  • Understand the concept of the design space
  • Understand the concept of design questions
  • Appreciate the need of exploring a number of design alternatives
  • Appreciate the difference between conceptual and detailed design
  • Know how to apply different creativity tools
  • Be prepared to conduct a brainstorming session
  • Know different forms of design representation
  • Acquire practical skills in designing a 3D UI for a given context of use
  • Acquire practical skills in presenting design concepts
  • Acquire practical skills in making design changes to eliminate found interaction problems
  • Understand the concept of research questions
  • Be familiar with various user research and usability evaluation methods
  • Be familiar with various ways of looking for study participants
  • Acquire practical skills in forming recruitment criteria
  • Acquire practical skills in recruiting human subjects
  • Understand Norman’s Stages-of-action model
  • Understand the concept of usability problems
  • Understand the difference among inefficiencies, feature requests, and software bugs
  • Acquire practical skills in planning usability testing
  • Appreciate the difference among moderation techniques for usability testing
  • Acquire practical skills in conducting usability tests
  • Acquire practical skills in making sense of data gathered through usability testing
  • Acquire practical skills in representing results of a usability study
  • Know the peculiarities of conducting usability tests for AR/VR
  • Know theories which explain the causes of motion sickness
  • Be familiar with web tools which are used to create XR apps
  • Be familiar with native tools which are used to create XR apps
Course Contents

Course Contents

  • Interaction Design for AR/VR: An Introduction
    The concepts of augmented, mixed, virtual, and extended realities. Brief history and the current state of immersive tech. Use cases of immersive tech. XR trends. About this course.
  • Overview of 3D UI Prototyping Tools and Techniques
    Why prototyping does matter. The classification of 3D UI prototyping methods. Bodystorming. Using 3D-painting VR apps to sketch 3D UIs. Using specialized no-code solutions to create interactive prototypes. Brief overview of code-based solutions.
  • The Essentials of Interaction Design
    The concept of the user interface. The concept of the spatial interface (the 3D user interface). The concept of the interaction. The context of use concept. The concept of user experience. The concept of usability. Mark Hassenzahl’s Model. Goal-directed design process. Design-informing models.
  • Hardware Technologies for 3D UIs
    The concept of modality in HCI. How we perceive depth. Vergence-accommodation conflict. Overview of computer-human modalities and related output devices. The classification of positional tracking technologies. Overview of human-computer modalities and related input devices. Implicit human-computer interaction.
  • Interaction Paradigms for XR
    The concept of interaction techniques. The concept of interaction paradigms. Overview of modern interaction paradigms for AR/VR.
  • Overview of 3D UI Design Patterns
    Various types of design knowledge. Overview of design patterns used in 3D UIs.
  • The Process of Design Creation
    The concept of usability goals. Why an exploration of design alternatives is necessary. The concept of design questions. The process of synthesizing solutions and comparing them. The difference between conceptual and detailed design. Creativity tools. Types of design representations.
  • Introduction to User Research
    The concept of research questions. The classification of user research methods. The classification of usability evaluation methods. Recruitment criteria. Ways of looking for participants.
  • Essential Usability Evaluation Concepts
    How people interact with digital things. Norman’s Stages-of-action model. The concept of usability problems. Interaction problems of other kinds.
  • Usability Testing: Preparation
    Planning usability tests. Data collection techniques.
  • Usability Testing: Moderation & Data Analysis
    Overview of moderation techniques for usability testing. Moderating usability tests. Usability data analysis.
  • Conducting Usability Tests of XR Apps
    Recruitment. Lab layout. Study design. Fitting and hygiene. Communication with participants. Data collection.
  • Overview of Dev Tools for Creating XR Apps
    WebXR. Tools & frameworks for developing XR apps for the web. Unity XR. Overview of libraries for creating 3D UIs in Unity.
  • Fighting Motion Sickness in VR
    Overview of the theories explaining the causes of motion sickness. Design best practices which help to avoid having motion sickness.
Assessment Elements

Assessment Elements

  • non-blocking Homework 1 (HW1)
    The assignment implies creating a simple prototype using Unity for Windows/macOS. A description of the prototype's UI and its behavior is given in a form of a list of requirements.
  • non-blocking Homework 2 (HW2)
    The assignment implies creating a Unity prototype for an XR device using provided by the professor middleware. A description of the prototype's UI and its behavior is given in a form of a list of requirements.
  • non-blocking Team Project 3 (TP3)
    This assignment implies proposing design concepts of a complete app by a given set of functions, user and/or interactions models, and best practices found in the analogous apps. The design concepts are prepared by every team and presented on the 7th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the proposed concepts with the team and gives tips on how to improve them.
  • non-blocking Team Project 4 (TP4)
    This assignment implies forming recruitment criteria which later will be used to look for participants of a usability study. A draft of the criteria is prepared by every team and presented on the 7th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminar the professor discusses the draft with the team and gives tips on how to improve it.
  • non-blocking Team Project 5 (TP5)
    This assignment implies developing a test plan to evaluate the team’s app. A draft of the plan is prepared by every team and presented on the 10th seminar, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the draft with the team and gives tips on how to improve it.
  • non-blocking Team Project 6 (TP6)
    This assignment implies creating a Unity prototype according to the design proposed earlier (TP3) and conducting usability tests in accordance with the plan prepared during the previous assignment, analyzing usability data gathered throughout the tests, and writing a report that describes all interaction problems found during the study. A draft of the report is prepared by every team and presented on the 12th or 13th seminars, otherwise the team loses a portion of the grade for practice activities. During the seminars the professor discusses the draft with the team and gives tips on how to improve it.
  • non-blocking Practice Activities (PA)
  • non-blocking Final Exam (FE)
    Oral examination, 2 questions.
Interim Assessment

Interim Assessment

  • Interim assessment (3 module)
    0.3 * Final Exam (FE) + 0.1 * Homework 1 (HW1) + 0.1 * Homework 2 (HW2) + 0.08 * Practice Activities (PA) + 0.09 * Team Project 3 (TP3) + 0.04 * Team Project 4 (TP4) + 0.09 * Team Project 5 (TP5) + 0.2 * Team Project 6 (TP6)
Bibliography

Bibliography

Recommended Core Bibliography

  • Cooper, A., Reimann, R., Cronin, D., & Noessel, C. (2014). About Face : The Essentials of Interaction Design (Vol. Fourth edition Alan Cooper, Robert Reimann, David Cronin, Christopher Noessel). Hoboken: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=827342
  • Hartson, R., & Pyla, P. S. (2012). The UX Book : Process and Guidelines for Ensuring a Quality User Experience. Amsterdam: Elsevier Ltd. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=453819
  • LaViola, J. J. (2017). 3D User Interfaces : Theory and Practice: Vol. Second edition. Addison-Wesley Professional.
  • Schmalstieg, D., & Höllerer, T. (2016). Augmented Reality : Principles and Practice. Boston: Addison-Wesley Professional. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1601643

Recommended Additional Bibliography

  • Holtzblatt, K., & Beyer, H. (2015). Contextual Design : Evolved. Morgan & Claypool Publishers.
  • Kuniavsky, M., Goodman, E., & Moed, A. (2012). Observing the User Experience : A Practitioner’s Guide to User Research (Vol. 2nd ed). Burlington: Morgan Kaufmann. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=472263
  • Michael E. Wiklund P.E, Jonathan Kendler, & Allison Y. Strochlic. (2016). Usability Testing of Medical Devices: Vol. Second edition. CRC Press.
  • Norman, D. A. (2013). The Design of Everyday Things : Revised and Expanded Edition (Vol. Revised and expanded edition). New York: Basic Books. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=658202
  • Rubin, J., & Chisnell, D. (2008). Handbook of Usability Testing : How to Plan, Design, and Conduct Effective Tests (Vol. 2nd ed). Indianapolis, IN: Wiley. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=233103