MV4001 Human Factors of Virtual Environments

This course focuses on human factors issues in virtual environments (VEs). While the similarities of VEs to the real world can often make VE interfaces intuitive and easy to use, the differences between VEs and the real world can often be the cause of serious performance problems and physical inability to effectively use a system. The design of effective VE systems depends on an understanding of humans and their interaction with their environment. Only then, can a VE system hope to achieve reasonable performance levels and acceptability. This course will survey the VE literature on issues of human performance, perception, cognition, multimodal interfaces, locomotion, wayfinding, object selection and manipulation, visualization, simulator sickness, and performance differences between individuals. Prerequisites: None.

Lecture Hours

Lab Hours

Course Learning Outcomes

HUMAN FACTOR ISSUES OF VIRTUAL ENVIRONMENTS (VEs): INTRODUCTION:

Student will be able to define the field of HF and identify the areas that it is concerned with.
Student will be able to compare and contrast the areas of HF interest in the past with current approaches and disciplines that contribute to the body of knowledge in HF.
Student will be able to describe the role of HF in VEs and the areas of HF research for VEs
Student will be able to describe a rationale for studying human perceptual modalities.

PERCEPTUAL MODALITIES
Student will be able to define, compare and contrast sensing and perception.
Student will be able to describe the characteristics of sensory stimulation and how their study connects with design of VE systems.
Student will be able to describe the processes that support human sensing of visual, auditory, haptic, vestibular and olfactory stimuli, including the process of transduction.
Student will be able to describe the issues with human color perception, depth perception, contrast and compare different forms of depth cues.
Student will be able to describe the importance of addressing spatial disorientation and motion sickness.
Student will be able to describe the phenomena behind 3D sound, and elements that influence human perception of the sound.
Student will be able to describe the need to integrate haptic displays in VEs and how they relate to human performance.
Student will be able to apply the knowledge about haptic displays and determine what military tasks may benefit from that type of solutions.
Student will be able to compare and contrast passive and active haptics, and describe the processes of haptic rendering.
Student will be able to determine and argue if some task may benefit from 3DOF or 6DOF haptic device.
Student will be able to combine different sensory solutions in a coherent system that supports a practical human task.

VR INPUT & OUTPUT DEVICES:

Student will be able to describe the interaction, and make a difference between the interaction and interaction technique.
Student will be able to describe communication protocol and a flow of information between a user and a computer system.
Student will be able to describe the user tasks in a virtual environment.
Student will be able to compare and contrast two reference models for interaction in virtual environments: VR Model and Extended Desktop Model, and associate VR devices with each model.
Student will be able to describe the requirements for position trackers as they support the work of input devices.
Student will be able to describe the type of tracking technologies used in VEs.
Student will be able to apply the knowledge about user tracking and determine what military tasks may need particular forms of user tracking to support human operation in those systems.
Student will be able to describe the approaches that were used to realize large scale display surfaces.
Student will be able to couple input devices with and range of display solutions that satisfy particular user needs.
Student will be able to apply the knowledge about human sensory systems, the tasks they need to be able to do, and advise for a particular combination of input devices and display solutions (visual, auditory, haptic, olfactory)
Student will be able to describe, analyze and critique the technological advancements in systems that support mission planning and user interaction that is enabled in each system.

AUGMENTED REALITY (AR), MIXED REALITY (MR), EXTENDED REALITY (XR), AND 360 VIDEOS:

Student will be able to describe the concept of virtuality continuum.
Student will be able to describe the main ingredients of augmented reality definition used in scientific literature.
Student will be able to compare and contrast the definitions of augmented reality and mixed reality used in scientific literature and media (industry).
Student will be able to describe, analyze and critique the technological advancements in augmented reality headsets.
Student will be able to compare and contrast optical vs video HMD.
Student will be able to describe and discuss contemporary research topics in AR domain.
Student will be able to describe and analyze technical capabilities of 360 videos, including the resources needed to create them.
Student will be able to determine and select task that may benefit from using AR, MR, and 360 videos, and apply those technologies to support human performance.

INTERACTION TASK: NAVIGATION

Student will be able to describe the main elements of navigation task, and analyze the concepts of travel and wayfinding.
Student will be able to describe the concept of locomotion and critique the ways in which it can be realized in VE.
Student will be able to describe the differences between the perception of distances in VR and in real world.
Student will be able to describe the model of navigation and analyze the taxonomy of VE travel techniques.
Student will be able to compare and contrast different methods of steering.
Student will be able to describe and discuss performance metrics for travel techniques.
Student will be able to describe, analyze and critique a variety of navigation techniques, and discuss design guidelines for locomotion interfaces.
Student will be able to describe the differences between navigation in real and virtual environments.
Student will be able to describe the major principles of multiscale virtual environments and multiscale navigation techniques.
Student will be able to describe the elements of collaborative navigation.

INTERACTION TASK: OBJECT MANIPULATION

Student will be able to describe the main elements of object manipulation task, and analyze the taxonomy of object manipulation task.
Student will be able to discuss different elements that influences the success of object manipulation technique.
Student will be able to describe Fitts' Law.
Student will be able to describe and discuss egocentric and exocentric manipulation techniques, and analyze and compare different object interaction techniques.

REPRESENTING PEOPLE, MULTIUSER ENVIRONMENTS AND COLLABORATION IN VEs:

Student will be able to describe and discuss avatars and agents, and their role in VEs.
Student will be able to discuss different elements of spatial and behavioral representation of avatars and agents.
Student will be able to describe the issues related to crowd simulation.
Student will be able to discuss the elements of human syntax and semantics.
Student will be able to compare simulation of civilian and military characters.
Student will be able to analyze and compare physical simulations of humans (e.g., mannequins) and virtual humans.
Student will be able to discuss a range of issues associated with a realistic representation of virtual humans.
Student will be able to discuss and reflect on future research topics associated with virtual humans, and their role in military applications
Student will be able to describe and discuss the issues associated with validation of simulated human behavior.
Student will be able to describe and explain the phenomenon of uncanny valley.
Student will be able to describe the main reasons why we study multiuser environments and collaboration in VEs.
Student will be able to describe and discuss the components of multiuser virtual environments, and collaborative awareness-action-feedback loop.
Student will be able to analyze and compare different examples of multiuser virtual environments.
Student will be able to analyze and discuss the elements of Dismounted Soldier Training System.
Student will be able to analyze and discuss select examples of collaborative environments developed in support of NPS student thesis.
Student will be able to describe ongoing research topics in domain of multiuser virtual environments.

GAME BASED SYSTEMS & PERSISTENT ENVIRONMENTS

Student will be able to describe the values that game-based systems and persistent augmented and virtual environments (PAVE) bring to military domain.
Student will be able to analyze and discuss the main elements of game-based system and persistent augmented and virtual environments, and analyze and discuss select examples of those systems.
Student will be able to describe and discuss the concept of gamification.
Student will be able to describe and comment the ongoing research topics in domain of game-based systems.

IMMERSION, PRESENCE AND CO-PRESENCE

Student will be able to describe and discuss the elements of immersion, presence, and co-presence definitions.
Student will be able to describe and discuss qualitative measure of VEs
Student will be able to describe and discuss the concepts of mediated environment, virtual embodiment, self-location, agency, body ownership, and break in presence.
Student will be able to analyze and discuss the main factors that influence presence.
Student will be able to describe and discuss the ways in which we can measure presence and co-presence, and apply that knowledge to evaluate them in human tasks.

HUMAN PERFORMANCE EVALUATION

Student will be able to describe and discuss the steps of the process used to evaluate human performance, including the ethical considerations of human performance evaluation.
Student will be able to describe and discuss the validity and reliability in data collection, and threats to validity and reliability.
Student will be able to describe and discuss the types of data that can be collected when evaluating human performance, and time when they are collected.
Student will be able to describe the main reasons why we evaluate usability of human systems, and analyze a range of issues that need to be considered when doing usability evaluation.
Student will be able to describe empirical and non-empirical methods of doing usability evaluation, and discuss their advantages and disadvantages.
Student will be able to describe and discuss data collection methods and the types of measures collected.
Student will be able to design and execute a study that evaluates usability of VEs and its user interface.
Student will be able to describe and discuss the types of psychological data that can be collected when evaluating human performance
Student will be able to describe and discuss the sensors used to measure responses of the peripheral nervous system.
Student will be able to describe and discuss the technical solutions used for electroencephalography (EEG), noninvasive (dray and wet sensors).
Student will be able to describe and discuss a range of sensors used as wearables, and design a study that would collect that type of measurements.

Student will be able to analyze human task, design and execute a user study that would measure cyber sickness, examine usability of the system and evaluate human task performance (including knowledge and skill acquisition).

HEALTH AND SAFETY ISSUES

Student will be able to describe and discuss the goals of the research focused on health and safety in VEs, and the types of health issues in VEs.
Student will be able to describe the characteristics and symptoms of cybersickness, and the main theories that explain the causes of cybersickness.
Student will be able to compare motion sickness, cybersickness and simulator sickness.
Student will be able to describe and discuss the ways in which we can measure simulator sickness and cybersickness, including the ways in which we can reduce the symptoms of cybersickness.
Student will be able to describe and discuss the responsibilities of the researchers.
Student will be able to describe and discuss the issues related to hygiene in VR/AR HMDs.
Student will be able to describe and discuss the safety concerns in AR.

DIFFUSION AND LARGE SCALE ADOPTION OF TECHNOLOGY

Student will be able to describe and discuss the reasons why we care about large scale adoption of technical solutions - the problem and motivation for work in this domain.
Student will be able to describe and discuss the basic postulates of Diffusion of Innovation theory by Everett Rogers.
Student will be able to analyze the parameters that influence the adoption of innovation and apply them to the military domain.
Student will be able to discuss and analyze practical considerations and techniques for successful adoption of innovations (special emphasis is on effective use of virtual simulations and game-based systems in training).
Student will be able to discuss and analyze a set of approaches for the accelerated adoption rate.
Student will be able to design and propose additional approaches that can support accelerated adoption rate.

ETHICS IN VEs: PROFESSIONAL ETHICS FOR RESEARCHERS & ETHICS AND LAW IN VIRTUAL WORLDS

Student will be able to describe and discuss the elements of professional ethics in research domain and code of conduct for researchers.
Student will be able to describe and 'Obedience to authority' (Milgram) experiment, discuss the results of that research study and issues that it raised.
Student will be able to describe and discuss basic postulates of Belmont Report or 1979.
Student will be able to describe and discuss basic guidelines for approach to study with human subjects.
Student will be able to describe and discuss responsibilities of the Institutional Review Board (IRB).
Student will be able to describe and discuss the reasons for studying the law and ethics in Virtual Environments.
Student will be able to describe the main areas of law affecting virtual environments design and use.
Student will be able to describe and discuss current laws in virtual worlds.
Student will be able to describe and discuss privacy and data issues.
Student will be able to describe and discuss the issues that emerge with intelligent (autonomous) agents and their acting in virtual worlds.