Accessibility Conformance Testing Rules Format 1.0

Editor’s Draft,

This version:
Previous Versions:
Wilco Fiers (Deque Systems)
Maureen Kraft (IBM Corp.)


The Accessibility Conformance Testing (ACT) Rules Format 1.0 defines a format for writing accessibility test rules. These rules can be carried out fully-automatically, semi-automatically, and manually. This common format allows any party involved in accessibility testing to document and share their testing procedures in a robust and understandable manner. This enables transparency and harmonization of testing methods, including methods implemented by accessibility test tools.

Status of this document

1. Introduction

There are currently many test procedures and tools available which aid their users in testing web content for conformance to accessibility standards such as the Web Content Accessibility Guidelines (WCAG) [WCAG]. As the web develops in both size and complexity, these procedures and tools are essential for managing the accessibility of resources available on the web.

This format is intended to enable a consistent interpretation of how to test for accessibility requirements and promote consistent results of accessibility tests. It is intended to be used to describe both manual accessibility tests as well as automated tests performed by accessibility testing tools.

Describing how to test certain accessibility requirements will result in accessibility tests that are transparent, with test results that are reproducible. The Accessibility Conformance Testing (ACT) Rules Format defines the requirements for these test descriptions, known as Accessibility Conformance Testing Rules (ACT Rules).

2. Scope

The ACT Rules Format defined in this specification is focused on the description of rules that can be used in testing content created using web technologies, such as Hypertext Markup Language [HTML], Cascading Style Sheets [css-2018], Accessible Rich Internet Applications [WAI-ARIA], Scaleable Vector Graphics [SVG2] and more, including digital publishing. The ACT Rules Format is designed to be technology agnostic, meaning that it can conceivably be used to describe test rules for other technologies.

The ACT Rules Format can be used to describe ACT Rules dedicated to testing the accessibility requirements defined in Web Content Accessibility Guidelines [WCAG], which are specifically designed for web content. Other accessibility requirements applicable to web technologies can also be testable with ACT Rules. For example, ACT Rules could be developed to test the conformance of web-based user agents to the User Agent Accessibility Guidelines [UAAG20]. The ACT Rules Format might not always be suitable to describe tests for other types of accessibility requirements.

3. ACT Rule Types

In accessibility, there are often different technical solutions to make the same type of content accessible. Multiple solutions could be tested in a single rule; however, such a rule tends to be quite complex, making it difficult to understand and maintain. The ACT Rules Format solves this by providing two types of rules:

Composite rule: Header cells in HTML tables (WCAG 2 Success Criterion 1.3.1).

This rule uses results from the following atomic rules:

If any one of these rules passes, the outcome of the composite rule for the table cell is "passed".

The separation between atomic rules and composite rules can be seen as a division of responsibility. Atomic rules typically test if web content is correctly implemented in a particular solution. Composite rules can test if a combination of outcomes from atomic rules satisfy the accessibility requirement, in part or as a whole.

Not all atomic rules have to be part of a composite rule. Composite rules are used when the outcomes of multiple atomic rules need to be combined in order to determine (non-)conformance of a test subject to the Accessibility Requirements

4. ACT Rule Structure

An ACT Rule MUST consist of at least the following items:

ACT Rules MUST be written in a format that conforms to the Web Content Accessibility Guidelines [WCAG] or a comparable accessibility standard. ACT Rule test cases are allowed to contain inaccessible content. If any test case contains accessibility issues listed in WCAG 2.1 Section 5.2.5 Non-Interference, users MUST be warned of this in advance. Using an accessible format supports people with disabilities. It also makes internationalization of ACT Rules easier.

5. Rule Identifier

An ACT Rule MUST have a unique identifier that can be any unique text, such as plain text, URL or a database identifier.

Example: ACT Rules may use file names as identifiers. They include a technology directory, followed by a handle that includes an element name or attribute:

6. Rule Description

An ACT Rule MUST have a description that is in plain language, and provides a brief explanation of what the rule does.

Example: This rule checks that the HTML page has a title
Example: The rule checks that for the html element, there is no mismatch between the primary language in non-empty lang and xml:lang attributes, if both are used.

7. Accessibility Requirements

Accessibility requirements are just that: A requirement that a particular web page conforms to for it to be considered accessible. A common example of accessibility requirements are the WCAG 2.1 success criteria. Some organizations have additional requirements that come from different sources, such as laws, internal standards, etc. These too are considered accessibility requirements which can be tested using the ACT Rules Format.

7.1. Accessibility Requirements List

An ACT Rule SHOULD list an identifier and reference for each accessibility requirement that it maps to.

Example: The outcome of this rule affects conformance to the following WCAG 2.1 success criteria:

One reason why an atomic rule might not list accessibility requirements is because it is part of a composite rule, and only a failed outcome in the composite rule would mean not satisfying the accessibility requirement. Another reason might be that an ACT Rule is designed to evaluate a best practice that is not part of any accessibility standard.

7.2. Accessibility Requirements Mapping

If any accessibility requirements are listed, the ACT Rule MUST indicate how each outcome impacts the conformance to the listed accessibility requirements. For WCAG 2.1 Success Criteria, this means indicating whether or not the success criterion can be satisfied, or if further testing is needed.

Example: The outcome of this rule maps to the accessibility requirements in the following manner:

Note: In WCAG 2.1, success criteria do not evaluate to passed, failed or inapplicable. Rather they can be satisfied (or not). (See the WCAG 2.1 definition: satisfies a success criterion.) If a success criterion is not satisfied a web page can only conform if there is a conforming alternative version, as described in WCAG 2.1 Conformance Requirement 1.

8. Aspects Under Test (Atomic rules only)

An aspect is a distinct part of the test subject. For example, rendering a particular piece of content to an end user involves multiple different technologies, some or all of which can be of use in an [atomic rule]. Some rules need to operate directly on the Hypertext Transfer Protocol [http11] messages exchanged between a server and a client, while others need to operate on the Document Object Model [DOM] tree exposed by a web browser. Rules can operate on several aspects simultaneously, such as both the HTTP messages and the DOM tree.

Atomic rules MUST list the aspects used in the applicability and expectations.

Some aspects are already well defined in a formal specification within the context of web content, such as HTTP messages, DOM tree, and CSS styling [css-2018]. These do not warrant a detailed description further than a reference to the corresponding section in this specification (see Common Aspects under Test).

For other aspects that are not well defined, an ACT Rule MUST include either a detailed description of the aspect in question or a reference to a well defined description.

Example: Test aspects for a rule that checks if images have an accessible name:

Example: Test aspects for a rule that checks if a transcript is available for videos:

Example: Test aspects for a rule that checks for use of (language specific) generic link texts like "click here" and "more":

9. Atomic Rules List (Composite rules only)

A composite rule uses outcomes from atomic rules and applies a logic to them so that a single outcome can be determined for each test target. The identifiers of all the atomic rules used in the expectations MUST be listed in the composite rule. The atomic rules list describes the input for composite rules, similar to how aspects under test describe the input for atomic rules.

10. Applicability

The applicability describes what (parts of) the test subject are tested.

10.1. Applicability for Atomic Rules

The applicability section is a required part of an atomic rule. It MUST contain a precise description of the parts of the test subject to which the rule applies. For example, specific nodes in the DOM [DOM] tree, or tags that are incorrectly closed in an HTML [HTML] document. These are known as the test targets. The applicability MUST only use information provided through test aspects of the same rule. No other information can be used in the applicability.

Applicability MUST be described objectively, unambiguously and in plain language.

An objective description is one that can be resolved without uncertainty, in a given technology. Examples of objective properties in HTML are element names, their computed role, the spacing between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. Even concepts like headings and images can be misunderstood. For example, describing that the rule examines the tag name, the accessibility role, or the element’s purpose on the web page. The latter of which is almost impossible to define objectively. When used in applicability, potentially ambiguous concepts MUST be defined objectively. This definition can be part of a larger glossary shared between rules.

Example: The applicability of an atomic rule testing WCAG 2.1 Audio Control:

Any video or audio element(s) with the autoplay attribute, as well as any object element(s) that is used for automatically playing video or audio when the web page loads.

10.2. Applicability for Composite Rules

A composite rule defines how the outcomes from rules in its atomic rules list are used to determine a single outcome.

The applicability of a composite rule is defined as the union of all the applicability sections of its atomic rules. This can be inferred from the list of atomic rules. Rule authors MAY describe the applicability for composite rules. This can be useful if it is difficult to express the combined applicability in plain language. If the composite rule includes applicability, it MUST be the union of all the applicability sections of the atomic rules.

Note that atomic rules in a composite rule MAY have different applicability. Because of this, not every element applicable within the composite rule is tested by every atomic rule.

Example: A composite rule about img elements uses results from atomic rules that have the following applicability:

The applicability of the composite rule combines the applicability of both atomic rules. This becomes:

All img elements.

11. Expectations

The expectations describe what the requirements are for the test targets derived from the applicability.

11.1. Expectations for Atomic Rules

An atomic rule MUST contain one or more expectations. An expectation is an assertion that is true about each test target (see Applicability). Each expectation MUST be distinct, unambiguous, and be written in plain language. Unlike in applicability, a certain level of subjectivity is allowed in expectations. Meaning that the expectation has only one possible meaning, but that meaning MAY not fully quantifiable.

When all expectations are true for a test target, the test target passed the rule. If one or more expectations are false, the test target failed the rule. When atomic rules are used in composite rules, the outcome of the composite rules can be passed even if the outcome of one of its constituent atomic rule is failed, depending on the expectations of the composite rule.

Example: A rule for labels of HTML input elements might have the following expectations:

  1. The test target has an accessible name (as described in Accessible Name and Description: Computation and API Mappings 1.1). [accname-aam-1.1]
  2. The accessible name describes the purpose of the test target.

An atomic rule expectation MUST only use information available in the test aspects, from the applicability, and other expectations of the same rule. No other information can be used in the expectation. So for instance, an expectation could be "Expectation 1 is true and ...", but it can’t be "Rule XYZ passed and ...". This ensures the rule is encapsulated.

11.2. Expectations for Composite Rules

A composite rule MUST contain one or more expectations that describes the logic that is used to determine a single passed or failed outcome for each test target, based on the outcomes of atomic rules listed in its atomic rules list.

When all expectations are true for a test target, the test target passed the rule. If one or more expectations is false, the test target failed the rule. This works the same way for atomic rules. A composite rule expectation MUST NOT use information from test aspects.

A composite rule for WCAG 2.1 Audio Description or Media Alternative applies a logic across results from three atomic rules. The expectation of the composite rule is as follows:

For each test target, the outcome of one of the following rules is passed:

  • Video elements have an audio description (video-audio-desc)
  • Video elements have a media alternative (video-text-alt)

12. Assumptions

An ACT Rule MUST list any assumptions, limitations or any exceptions for the evaluation, the test environment, technologies being used or the subject being tested. For example, a rule that would partially test WCAG 2.0 Success Criterion 1.4.3 Contrast (Minimum) based on the inspection of CSS properties could state that it is only applicable to HTML text content stylable with CSS, and that the rule does not support images of text.

Sometimes there are multiple plausible ways that an accessibility requirement can be interpreted. For instance, it is not immediately obvious if emoji characters are "text" or "non-text content" under WCAG 2.1. Whatever the interpretation is, this MUST be documented in the rule.

While this item MUST be included in the ACT Rule, it MAY be empty when there are no known assumptions, limitations or exceptions.

13. Accessibility Support

Content can be designed to rely on the support for particular accessibility features by different assistive technologies and user agents. For example, content using a particular WAI-ARIA [WAI-ARIA] feature relies on that feature to be supported by assistive technologies and user agents. This content would not work for assistive technologies and user agents that do not support WAI-ARIA. WCAG [WCAG] provides a definition for accessibility supported use of a Web technology.

An ACT Rule MUST include known limitations on accessibility support. For example, an atomic rule that checks for a particular accessibility feature that is known not to be widely supported by assistive technologies and web browsers, or a composite rule that includes atomic rules with known accessibility support limitations.

While this item MUST be included in the ACT Rule, it MAY be empty when there are no known accessibility support issues.

Note: WCAG Evaluation Methodology (WCAG-EM) provides guidance on defining an accessibility support baseline for test scenarios, which can help users of ACT Rules to select the appropriate rules for their own circumstance.

14. Test Cases

Test cases are (snippets of) content that can be used to validate the implementation of ACT Rules. Each rule MUST have one or more test cases for passed, failed, and inapplicable outcome. A test case consists of two pieces of data, a snippet of each test aspect for a rule, and the expected result from evaluating that rule. Test cases serve two functions, firstly as example scenarios for readers to understand when a rule passes, when it fails, and when it is inapplicable. It also serves developers and users of accessibility testing tools and methodologies to validate that a rule is correctly implemented.

15. Change Log

It is important to keep track of changes to the ACT Rules so that users of the rules can understand if changes in test results are due to changes in the rules used when performing the tests, or from changes in the content itself. All changes to an ACT Rule that can change the outcome of a evaluation MUST be recorded in a change log. The change log can either be part of the rule document itself or be referenced from it.

Each new release of an ACT Rule MUST be identifiable with either a date or a version number. Additionally, a reference to the previous version of that rule MUST be available. For extensive changes, a new rule SHOULD be created and the old rule SHOULD be deprecated.

Example: An example of when a new rule ought to be created is when a rule that tests for the use of a blink element changes to instead look for any animated style changes. This potentially adds several new failures that were previously out of scope. Using that same rule as an example, adding an exception to allow blink elements positioned off screen can be done by updating the existing rule.

16. Issues List (optional)

An ACT Rule MAY include a list or a reference to a list of any known issues. The issues list would be used to record cases where the outcome of an ACT Rule was failed where it ought to have been passed or inapplicable, or vice versa. There are several reasons why this might occur, including:

The issues list serves two purposes. For users of ACT Rules, the issues list gives insight into why an inaccurate result occurred, as well as provide confidence in the result of that rule. For the designer of the rule, the issues list is also useful to plan future updates to the rule. In a new version of the rule, resolved issues would be moved to the change log.

17. Acknowledgements (optional)

An ACT Rule MAY contain acknowledgements. This can include, but is not limited to:

18. Background (optional)

An ACT Rule MAY contain information about the background for the development of the rule, or references to relevant reading. The relationship to the relevant reading should be specified. Examples of relevant background references for a rule for a WCAG [WCAG] success criterion could be Understanding documents, WCAG Techniques, or WAI-ARIA [WAI-ARIA], CSS [css-2018], or HTML [HTML] specifications.

19. Rule Accuracy

This section is non-normative.

While test cases can be used to determine if an ACT Rule was correctly implemented, they do not guarantee that implementations will never produce incorrect results. When writing ACT Rules, it is almost inevitable that edge cases will be overlooked. Technologies are always evolving, and content authors are constantly coming up with new and unexpected ways to use those.

There are two types of inaccuracies that can produce incorrect results. Inaccuracies in the implementation can be addressed with test cases, but inaccuracies in the ACT Rule itself can not. After all, rule inaccuracies come from the rule author being unaware of a particular edge case.

When a test result incorrectly indicates non-conformance to an accessibility requirement, this is known as a false positive. Opposite, when a rule incorrectly indicates conformance, this is a false negative. A percentage of false positives and false negatives can be calculated by comparing it to results from an accessibility audit:

The possibility of false positives and false negatives with ACT Rules mean they will likely require ongoing maintainence. Designing a process for maintaining ACT Rules is outside the scope of the ACT Rules Format, which is limited to the rules themselves. Neverthless, it is suggested that rule authors work out a process for maintaining their rules.

20. Harmonization

This section is non-normative.

While the ACT Rules Format is designed to stimulate harmonization, there are no direct requirement in the ACT Rules Format that prevent a rule author from writing rules inconsistent with already established ACT Rules. Neither are there requirements for ACT Rules to have a certain number of implementations, or to have a certain level of accuracy. This allows quality requirements to be different for different user groups, and allows them to develop over time.

Harmonization occurs when a group of rule implementors collectively accept the validity of an ACT Rule. For example, a community group of accessibility testing tool vendors could declare they’ve harmonized on a particular set of ACT Rules. Such a group can set acceptance criteria for rules, and have quality requirements that go beyond the ACT Rules Format. For example:

Example: The ACT EPUB group might have the following acceptance criteria:

An example of such a process is the WCAG ACT Review Process.

21. Definitions

Test Subject

A resource or collection of resources that can be evaluated by an ACT Rule.


  • An HTML page, including all embedded scripts, style and images
  • An EPUB document
  • A Vue component file

Note: Implementers using the [EARL10-Schema] can express the test subject with the subject property

Test Target

A distinct part of the test subject.


  • Nodes within an HTML page
  • A period of time within a video
  • A range of characters within a text document

Note: Implementers using the [EARL10-Schema] can express the test target with the pointer property


One of three types of conclusions that come from evaluating an ACT Rule on a test subject. An outcome can be:

  • Inapplicable: No part of the test subject matches the applicability
  • Passed: The test subject meets all expectations
  • Failed: The test subject does not meet all expectations

Note: While inapplicable is a valid outcome for ACT Rules, it might not be a valid result for all accessibility requirements. Notably the success criteria of WCAG 2.0 and WCAG 2.1 can only be evaluated to satisfied (passed and inapplicable) or not satisfied (failed).

Note: Implementers using the [EARL10-Schema] can express the outcome with the outcome property. In addition to passed failed and inapplicable, EARL 1.0 also defined an incomplete outcome. While this can not be the outcome of an ACT Rule when applied in its entirety, it often happens that rules are only partially evaluated. For example, when applicability was automated, but the expectations have to be evaluated manually. Such "interim" results can be expressed with the incomplete outcome.

Appendix 1: Expressing ACT Rule results with JSON-LD and EARL

This section is non-normative.

Example: Output data using EARL and JSON-LD. (See Evaluation and Report Language [EARL10-Schema] and Java Script Object Notation (JSON).)

  "@context": "https://raw.githubusercontent.com/w3c/wcag-act/master/earl-act.json",
  "@type": "Assertion",
  "subject": "https://example.org/",
  "test": "auto-wcag:rules/SC1-1-1-css-image.html",
  "result": {
    "outcome": "earl:failed",
    "pointer": "html > body > h1:first-child"


  "@context": "https://raw.githubusercontent.com/w3c/wcag-act/master/earl-act.json",
  "@type": "Assertion",
  "subject": "https://example.org/",
  "test": "auto-wcag:SC1-1-1-css-image.html",
  "result": {
    "outcome": "earl:failed",
    "source": [{
      "test": "auto-wcag:SC1-1-1-css-image.html",
      "result": {
        "outcome": "earl:failed",
        "pointer": "html > body > h1:first-child"
    }, {
      "test": "auto-wcag:SC1-1-1-css-image.html",
      "result": {
        "outcome": "earl:passed",
        "pointer": "html > body > h1:nth-child(2)"

Example: Aggregate outcomes to a WCAG success criterion

  "@context": "https://raw.githubusercontent.com/w3c/wcag-act/master/earl-act.json",
  "@type": "Assertion",
  "subject": "https://example.org/",
  "test": {
    "@id": "wcag20:#text-equiv-all",
    "title": "1.1.1 Non-text Content"
  "result": {
    "outcome": "Failed",
    "source": [{
      "test": "auto-wcag:SC1-1-1-css-image.html",
      "result": {
        "outcome": "Failed",
        "pointer": "html > body > h1:first-child"
    }, {
      "test": "auto-wcag:SC1-1-1-longdesc.html",
      "result": {
        "outcome": "Passed",
        "pointer": "html > body > img:nth-child(2)"

Example: Aggregate a list of results to a result for the website

  "@context": "https://raw.githubusercontent.com/w3c/wcag-act/master/earl-act.json",
  "@type": "Assertion",
  "subject": {
    "@type": ["WebSite", "TestSubject"],
    "@value": "https://example.org/"
  "test": "http://www.w3.org/WAI/WCAG2A-Conformance",
  "result": {
    "outcome": "Failed",
    "source": [{
      "test": "wcag20:text-equiv-all",
      "result": {
        "outcome": "Failed",
        "source": []
    }, {
      "test": "wcag20:media-equiv-av-only-alt",
      "result": {
        "outcome": "Passed",
        "source": []
    }, {
      "test": "wcag20:media-equiv-captions",
      "result" : {
        "outcome": "Inapplicable",
        "source": []

Appendix 2: Acknowledgments

This section is non-normative.

Participants of the AG WG active in the development of this document

Shadi Abou-Zahra, Trevor Bostic, Romain Deltour, Kathy Eng, Wilco Fiers, Alistair Garrison, Kasper Isager, Maureen Kraft, Mary Jo Mueller, Jey Nandakumar, Charu Pandhi, Stein Erik Skotkjerra, Anne Thyme Nørregaard, Kathleen Wahlbin

Enabling funders

This publication has been developed with support from the WAI-Tools Project, co-funded by the European Commission (EC) under the Horizon 2020 Program (Grant Agreement 780057). The content of this publication does not necessarily reflect the views or policies of the European Commission (EC) or any of the European Union (EU) Member States.


Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.


Terms defined by this specification


Normative References

S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119

Informative References

Joanmarie Diggs; Bryan Garaventa; Michael Cooper. Accessible Name and Description Computation 1.1. 18 October 2018. PR. URL: https://www.w3.org/TR/accname-1.1/
Tab Atkins Jr.; Elika Etemad; Florian Rivoal. CSS Snapshot 2018. 15 November 2018. NOTE. URL: https://www.w3.org/TR/css-2018/
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
Shadi Abou-Zahra. Evaluation and Report Language (EARL) 1.0 Schema. 2 February 2017. NOTE. URL: https://www.w3.org/TR/EARL10-Schema/
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
R. Fielding, Ed.; J. Reschke, Ed.. Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing. June 2014. Proposed Standard. URL: https://tools.ietf.org/html/rfc7230
Amelia Bellamy-Royds; et al. Scalable Vector Graphics (SVG) 2. 4 October 2018. CR. URL: https://www.w3.org/TR/SVG2/
James Allan; et al. User Agent Accessibility Guidelines (UAAG) 2.0. 15 December 2015. NOTE. URL: https://www.w3.org/TR/UAAG20/
James Craig; Michael Cooper; et al. Accessible Rich Internet Applications (WAI-ARIA) 1.0. 20 March 2014. REC. URL: https://www.w3.org/TR/wai-aria/
Wendy Chisholm; Gregg Vanderheiden; Ian Jacobs. Web Content Accessibility Guidelines 1.0. 5 May 1999. REC. URL: https://www.w3.org/TR/WAI-WEBCONTENT/