Accessibility Conformance Testing Framework Requirements

Branch Snapshot,

This version:
https://w3c.github.io/wcag-act/act-fr-reqs.html
Previous Versions:
https://w3c.github.io/wcag-act/archive_act-fr-reqs/2016-12-09.html
Issue Tracking:
GitHub
Editor:
Wilco Fiers (Deque Systems)

Abstract

The Accessibility Conformance Testing (ACT) Framework is intended to become a W3C Recommendation. It is developed by the [WCAG Working Group](https://www.w3.org/WAI/GL/) through its [ACT Task Force](https://www.w3.org/WAI/GL/task-forces/conformance-testing/). This document outlines the requirements for the ACT Framework. An overview of ACT efforts including complimentary deliverables is described in [What is ACT](https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT).

1. Status of This Document

This is the first draft of the ACT Framework Requirements document. It has not yet been reviewed or approved by the WCAG Working Group. At this stage, input on the following aspects is requested:

2. Introduction

The Accessibility Conformance Testing (ACT) Framework will define how to write rules for accessibility conformance testing. It will standardize aspects that are common across different rules, to contribute to the development of rules that produce reliable results, are implementable by different testing and quality assurance tools, and that provide transparency and comparability for tool users. This should contribute to greater consistency of results across different tools, and avoid confusion on accessibility.

The development and implementation of ACT Rules is complementary effort to the development of this ACT Framework. ACT Rules are currently developed through the [Auto-WCAG Community Group](https://www.w3.org/community/auto-wcag/). Feedback from the development of these ACT Rules will refine the ACT Framework during its development. Over time it is expected that other organizations will develop or make their rules available in the format defined by the ACT Framework, to facilitate a scalable approach for ACT Rules development.

The work has been incubated in the Auto-WCAG Community Group for over 2 years, which has shown a growing interest in harmonized accessibility testing. Development and implementation of ACT Rules are outside the scope of the ACT Framework. This work is currently done within the [Auto-WCAG Community Group](https://www.w3.org/community/auto-wcag/). The ACT Taskforce will support the Auto-WCAG Community group in the development of the first set of ACT Rules.

More background on ACT efforts and deliverables is provided in [What is ACT](https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/ACT_Overview_-_What_is_ACT).

3. Requirements

3.1. Ensure Understandability

The ACT Framework ensures that ACT Rules are understandable by tool developers, accessibility auditors and testers. Users of an ACT Rule should be able to follow the steps of the rule and come to the same conclusion.

3.2. Ensure Justification

The ACT Framework ensures that sufficient documentation is provided, to justify that ACT Rule results are valid interpretation of the accessibility requirements.

3.3. Ensure Consistency

The ACT Framework ensures that ACT Rules can be implemented in a way that produces consistent results in different tools and in different runs. An ACT Rule must include a way to test the implementation of that rule.

3.4. Ensure Measurable Accuracy

The ACT Framework provides criteria and a benchmarking mechanism to validate and measure the accuracy of ACT Rules. Note: Accuracy is difference of actual test results to expected results.

3.5. Ensure Evolution

The ACT Framework provides a mechanism to manage updates to ACT Rules as technologies change. ACT Rules should be dated and / or versioned to enable users to figure out if new issues are caused by a rule change or a change of content. A possible approach to this could be semantic versioning.

3.6. Existing Rulesets can be Transformed to ACT Rules

The ACT Framework provides support for different organizations and vendors to migrate their test rules into the required format. This may include mappings to other formats, and tolerance for different test rules structures and parameters, where possible

3.7. Accessibility Support

The ACT Framework ensures that ACT Rules consider difference in assistive technologies. ACT Rules can be configured to use or ignore accessibility features that are not consistently implemented. Accessibility support data is input for a rule, and is not part of the rule’s design.

3.8. Rules Test for Failures

The ACT Framework results in negative feature tests, meaning that ACT Rules test for violations instead of compliance. ACT Rules should map to [WCAG 2.0 Failure Techniques](https://www.w3.org/TR/WCAG20-TECHS/failures.html) where possible to avoid duplication of work. In some cases absence of violations may be proof of compliance, if rules are available to test all possible violations for a requirement.

3.9. Provide a Common Output Format

The ACT Framework ensures that results from ACT Rules can be aggregated to gain higher level insights. This must be consistent across tools, so that the output of different tools can complement each other.

3.10. Rules Not Included

The ACT Framework is a W3C Recommendation on how to write rules. It does not include any ACT Rules, with the possible exception of rule examples.

3.11. No New Accessibility Requirements

The ACT Framework does not add or change accessibility requirements. It’s purpose is to standardize the way requirements are tested. The method of testing must never change the actual requirement.

3.12. Automation and Manual Rules

The ACT Framework provides mechanisms to develop rules that are:

3.13. Usable With Different Accessibility Requirements

ACT Rules will be developed as part of the project as evidence that the ACT Framework is capable of targeting different accessibility requirements for web content and digital publications.

3.14. Applicable to Web and Digital Publishing Technologies

The ACT Framework provides mechanisms for the development of rules to test current web and digital publishing technologies, including but not limited to: HTML, CSS, ARIA and EPUB.

4. Definitions

4.1. Rule

A rule is the description of the procedure used to evaluate the status (pass or fail) of one ore more conditions of a requirement in a given context.

The rule defines:

4.2. Test

A test is the application of a rule.

4.3. Requirement

A requirement is a statement of necessity in a given domain (e.g. accessbility). It can be broken down into a set of conditions.

Conformance

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

References

Normative References

[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119