XR Accessibility User Requirements

W3C Editor's Draft

This version:
https://w3c.github.io/apa/xaur/
Latest published version:
https://www.w3.org/TR/xaur/
Latest editor's draft:
https://w3c.github.io/apa/xaur/
Editors:
(W3C)
(W3C)
Participate:
GitHub w3c/apa
File a bug
Commit history
Pull requests

Abstract

This document lists user needs and requirements for people with disabilities when using virtual reality or immersive environments, augmented or mixed reality and other related technologies (XR). It first introduces a definition of XR as used throughout the document, then briefly outlines some uses of XR. It outlines the complexity of understanding XR, introduces some accessibility challenges such as the need for accessibility multimodal support for a range of input and output devices and the importance of customization. Based on this information, it outlines accessibility user needs for XR and their related requirements. This is followed by information about related work that may be helpful to understand the complex technical architecture and processes behind how XR environments are built and what may form the basis of a robust accessibility architecture.

This document is most explicitly not a collection of baseline requirements. It is also important to note that some of the requirements may be implemented at a system or platform level, and some may be authoring requirements.

Status of This Document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

To comment on this draft file an issue in the W3C APA GitHub repository. If this is not feasible, send email to public-apa@w3.org (archives). In-progress updates to the document may be viewed in the publicly visible editors' draft.

This document was published by the Accessible Platform Architectures Working Group as an Editor's Draft.

GitHub Issues are preferred for discussion of this specification.

Publication as an Editor's Draft does not imply endorsement by the W3C Membership.

This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the 1 August 2017 W3C Patent Policy. The group does not expect this document to become a W3C Recommendation. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 15 September 2020 W3C Process Document.

1. Introduction

XR is an acronym used to refer to the spectrum of hardware, applications, and techniques used for virtual reality or immersive environments, augmented or mixed reality and other related technologies. This document is developed as part of a discovery into accessibility related user needs and requirements for XR. This document does not represent a formal working group position, nor does it currently represent a set of technical requirements that a developer or designer need strictly follow. It aims to outline the diversity of some current accessibility related user needs in XR and what potential requirements to meet those needs may be.

1.1 What does the term 'XR' mean?

As with the WebXR API spec and as indicated in the related WebXR explainer, this document uses the acronym XR to refer to the spectrum of hardware, applications, and techniques used for virtual reality or immersive environments, augmented or mixed reality and other related technologies. Examples include, but are not limited to:

The important commonality between them being that they all offer some degree of spatial tracking with which to simulate a view of virtual content as well as navigation and interaction with the objects within these environments.

Terms like "XR Device", "XR Application", etc. are generally understood to apply to any of the above. Portions of this document that only apply to a subset of these devices will be indicated as appropriate.

1.2 Definitions of virtual reality and immersive environments

Virtual reality and immersive environment definitions vary but converge on the notion of immersive computer-mediated experiences. They involve interaction with objects, people and environments using a range of controls. These experiences are often multi-sensory and may be used for educational, therapeutic or entertainment purposes.

1.3 Definitions of augmented and mixed reality

Augmented and mixed reality definitions vary but converge on the notion of computer-mediated interactions involving overlays on the real world. These may be informational, or interactive depending on the application.

2. What is XR used for?

XR has a range of purposes from education, gaming, multimedia, immersive communication and many others. It is currently evolving at a very fast rate and is not yet mainstream. This will change as computing power increases, hardware improves as well as the quality of user experience, XR will be more commonly be used for the performance of everyday practical tasks, for therapeutic uses, education and for entertainment.

3. Understanding XR and Accessibility Challenges

Understanding XR itself presents various challenges that are very technical. They include technical issues with a range of hardware, software, authoring tools as well as a need to understand interaction design principles, accessibility semantics and more. These all represent 'basic' technical complexities that are in themselves substantial. To add to this, for many designers and authors they may neither know or have access to people with disabilities for user testing. Neither may they have a practical way of understanding user needs that they can build a solid set of requirements from. In short, they just may not understand what user needs they are trying to meet when making XR accessible.

Some of the issues in XR, for example in gaming, for people with disabilities including:

There are also a range of other disabilities that will need to be considered in making XR accessible. It is beyond the scope of this document to describe them all in detail. General categories or types of disabilities are:

A person may have one of these disabilities or a combination of several. Each of these 'types' will be presented as a user need that should be met and understanding these needs are crucial in rising to the range of interesting challenges XR designers and authors will have when supporting accessibility and multimodality in XR environments.

These may be:

3.1 Immersive Environment challenges

Some of the many challenges with immersive environments accessibility (and also gaming) include the use of extremely complex input devices, control schemes that require a high degree of precision, timing and simultaneous action; ability to distinguish subtle differences in busy visual and audio information, having to juggle multiple complex goals and objectives [web-adapt].

There are also currently very useful accessibility guidelines available that are specific to gaming [game-a11y].

3.2 XR and supporting multimodality

Modality relates to modes of sense perception such as sight, hearing, touch and so on. Accessibility can be thought of as supporting multi-modal requirements and the transformation of content or aspects of a user interface from one mode to another that will support various user needs.

Considering various modality requirements in the foundation of XR means these platforms will be better able to support accessibility related user needs. There will be many modality aspects for the developer and/or content author to consider. XR authors and content designers will also need access to tools that support the multimodal requirements listed below.

The following Inputs and Outputs can be considered modalities that should be supported in XR environments.

3.3 Various input modalities

The following are example of some of the diverse input methods used by people with disabilities. In many real world applications these input methods may be combined.

3.4 Various output modalities

The following are a list of outputs that can be available to a user to help them understand, interact with and 'sense' feedback from an XR application. Some of these are in common use on the Web and other exploratory (such as Olfactory and Gustatory.)

3.5 XR controller challenges

As mentioned there are a range of input devices that may be used. Supporting these controllers requires an understanding of what they are and how they work. There are a variety of alternative gaming controls that may be very useful in XR environments and applications. For example the Xbox Adaptive Controller.

While XR is the experience, the controller plays a critical part in overcoming some complexity as well as mediating issues that may relate to other challenges around usability and helping the user understand sensory substitution devices.

Controllers such as the Xbox Adaptive Controller and other switch type inputs allow the issuer to remapping keyboard inputs to control virtual environments. The powerful customizations may allow the user to "do that thing that is difficult" for them with ease. In conjunction with this controller, for example, users with limited mobility they can also simulate actions in the XR environment that they would not be able to physically perform. WalkinVRDriver is a good example of this where motion range, position and orientation can be set to the user's ability.

3.6 Customization of control inputs

Give the user the ability to modify their input preference or use a variety of input devices. The remapping of keys used to control movement or interaction in virtual environments is not currently required by WCAG. It is nevertheless noted in the literature as desirable.

3.7 Using multiple diverse inputs simultaneously

A user with a disability may have several input devices or different assistive technologies. A user may switch 'mode' of interaction or the tools used and should be able to do so without degrading into a poor user experience where they lose focus on a task and cannot return to it, or make unforced errors, accidental input and so on.

Complexity needs to be managed and co-ordinated between different kinds of assistive technology in immersive environments. There is a platform level requirement to support multiple assistive technology in a cohesive manner. This would allow combinations to be used in a co-ordinated way e.g where the users day-to-day AT, can be used with other AT that may be embedded in the environment already for example.

Note

The REQ 5b: Voice activation also indicates potential issues with pairing multiple devices via bluetooth.

3.8 Consistent tracking with multiple inputs

There may be tracking issues when switching input devices. A tracking issue is where the user may lose their focus or it can be modified in unpredictable or unwanted ways, this can cause loss of focus and potentially push the user to make unwanted inputs or choices.

Outputs sent to multiple devices will need to be synchronised.

3.9 Usability and affordances in XR

An XR application should have a high level of usability for someone with a disability who is using assistive technology. Some challenges in translating interaction models may be:

4. XR User Needs and Requirements

This document outlines various accessibility related user needs for XR. These user needs should drive accessibility requirements for XR and its related architecture. These come from people with disabilities who use assistive technologies and wish to see the features described available within XR enabled applications.

User needs and requirements are often dependent on context of use. The following outline some accessibility user needs and requirements that may be applicable in immersive environments, augmented reality and 360° applications.

These following are neither exhaustive, nor definitive but are presented in order to help orientate the reader towards understanding some broad user needs and how to meet them.

4.1 Immersive semantics and customization

Note

In an spatialized augmented reality environment a blind user may find a combination of text to speech and sonic symbols helpful. By using a combination of text to speech and sonic symbolism a blind user can do a self-guided tour of a given area using their smartphone. [spatialized-navigation]

4.2 Motion agnostic interactions

4.3 Immersive personalisation

Note

Personalization involves tailoring aspects of the user experience to meet the needs and preferences of the individual user. W3C are working on various modules for web content that aim to support personalization and are exploring areas such as: [personalization-semantics] [personalization-content] [personalization-requirements]

4.4 Interaction and target customization

4.5 Voice commands

4.6 Color changes

4.7 Magnification context and resetting

Note

There are customisation approaches such as the automatic generation of user interfaces as demonstrated in the SUPPLE project, which adapt to the different challenges the user may face, such as vision, motor control and other user preferences and abilities. A generated UI can make multiple adaptations for different user needs at the same time. This is achieved by generating a UI, or several - after testing a persons ability using an algorithm to learn their preferences. [supple-project]

4.8 Critical messaging and alerts

4.9 Gestural interfaces and interactions

4.10 Text description transformation

4.11 Safe harbour controls

4.12 Immersive time limits

4.13 Reset focus and orientation

4.14 Second screen

4.15 Interaction speed

4.16 Avoiding sickness triggers

4.17 Spatial audio tracks and alternatives

4.18 Captioning, Subtitling and Text: Support and customization

4.19 Mono audio option

A. Acknowledgements

A.1 Participants of the APA Working Group Active in the Development of This Document

A.2 Previously Active Participants, Commenters, and Other Contributors

A.3 Enabling Funders

This work is supported by the EC-funded WAI-Guide Project.

B. References

B.1 Informative references

[able-gamers]
Thought On Accessibility and VR. AJ Ryan. March, 2017. URL: https://ablegamers.org/thoughts-on-accessibility-and-vr/
[game-a11y]
Game Accessibility Guidelines. Barrie Ellis; Ian Hamilton; Gareth Ford-Williams; Lynsey Graham; Dimitris Grammenos; Ed Lee; Jake Manion; Thomas Westin. 2019. URL: http://gameaccessibilityguidelines.com
[inclusive-seattle]
W3C Workshop on Inclusive XR Seattle. W3C; Pluto VR. W3C. Nov 2019. URL: https://www.w3.org/2019/08/inclusive-xr-workshop/
[maidenbaum-amendi]
Non-visual virtual interaction: Can Sensory Substitution generically increase the accessibility of Graphical virtual reality to the blind?. Maidenbaum, S.; Amedi, A. In Virtual and Augmented Assistive Technology (VAAT), 2015 3rd IEEE VR International Workshop on (pp. 15-17). IEEE. 2015.
[mono-ios]
iPhone User Guide. Apple. 2020. URL: https://support.apple.com/en-gb/guide/iphone/iph3e2e2cdc/ios
[personalization-content]
Personalization Semantics Content Module 1.0. Lisa Seeman; Charles LaPierre; Michael Cooper; Roy Ran; Richard Schwerdtfeger. W3C. 2020. URL: https://www.w3.org/TR/personalization-semantics-content-1.0/
[personalization-requirements]
Requirements for Personalization Semantics. Lisa Seeman; Charles LaPierre; Michael Cooper; Roy Ran. W3C. 2020. URL: https://www.w3.org/TR/personalization-semantics-requirements-1.0/
[personalization-semantics]
Personalization Semantics Explainer 1.0. Lisa Seeman; Charles LaPierre; Michael Cooper; Roy Ran; Richard Schwerdtfeger. W3C. 2020. URL: https://www.w3.org/TR/personalization-semantics-1.0/
[spatialized-navigation]
What’s around Me? Spatialized Audio Augmented Reality for Blind Users with a Smartphone. Blum J.R.; Bouchard M.; Cooperstock J.R. . Springer. 2012. URL: https://link.springer.com/chapter/10.1007/978-3-642-30973-1_5
[supple-project]
SUPPLE: Automatically Generating Personalized User Interfaces. Krzysztof Gajos et al. Harvard. 2010. URL: http://www.eecs.harvard.edu/~kgajos/research/supple/
[web-adapt]
W3C Workshop on Web Games Position Paper: Adaptive Accessibility. Matthew Tylee Atkinson; Ian Hamilton; Joe Humbert; Kit Wessendorf. W3C. Dec 2018. URL: https://www.w3.org/2018/12/games-workshop/papers/web-games-adaptive-accessibility.html