W3C

JSON-LD Test Suite

This manifest loads additional manifests for specific behavior tests for JSON-LD 1.1 API

This is an HTML version of a test manifest. The JSON-LD version of this manifest may be found at manifest.jsonld. The manifest vocabulary is described in the JSON-LD Test Vocabulary (JSON-LD, Turtle) and is based on the RDF Test Vocabulary.

The JSON-LD Test Suite is a set of tests that can be used to verify JSON-LD Processor conformance to the set of specifications that constitute JSON-LD. The goal of the suite is to provide an easy and comprehensive JSON-LD testing solution for developers creating JSON-LD Processors.

The JSON-LD Framing Specification maintains its own test suite.

General instructions for running the JSON-LD Test suites

compact tests have input, expected and context documents.

The expected results can be compared using JSON-LD object comparison with the processor output. Additionally, if the ordered option is not set, result should be expanded and compared with the expanded expected document also using JSON-LD object comparison.

For NegativeEvaluationTests, the result is a string associated with the expected error code.

expand tests have input and expected documents.

The expected results can be compared using JSON-LD object comparison with the processor output.

Expansion tests may have a expandContext option, which is treated as an IRI relative to the manifest.

For NegativeEvaluationTests, the result is a string associated with the expected error code.

html tests have input and expected documents and an optional context document.

The expected results can be compared using JSON-LD object comparison with the processor output after potentially remapping blank node identifiers (see below). Additionally, if the result is compacted and the ordered option is not set, result should be expanded and compared with the expanded expected document also using JSON-LD object comparison.

For NegativeEvaluationTests, the result is a string associated with the expected error code.

flatten tests have input and expected documents and an optional context document.

The expected results can be compared using JSON-LD object comparison with the processor output after potentially remapping blank node identifiers (see below). Additionally, if the result is compacted and the ordered option is not set, result should be expanded and compared with the expanded expected document also using JSON-LD object comparison.

For NegativeEvaluationTests, the result is a string associated with the expected error code.

remote-doc tests have input and expected documents.

The expected results can be compared using JSON-LD object comparison with the processor output.

For NegativeEvaluationTests, the result is a string associated with the expected error code.

Options may be present to describe the intended HTTP behavior:

fromRdf tests have input and expected documents.

The expected results can be compared using JSON-LD object comparison with the processor output.

toRdf tests have input and expected documents.

Some tests require the use of JSON Canonicalization Scheme to properly generate RDF Literals from JSON literal values. This algorithm is non-normative, but is assumed to be used to properly compare results. These tests are marked using the useJCS option.

The expected results can be compared using RDF Dataset Isomorphism.

A PositiveSyntaxTest looks specifically for syntax-related issues. A PositiveSyntaxTest succeeds when no error is found when processing.

ToRdf tests may have a expandContext option, which is treated as an IRI relative to the manifest.

Unless processingMode is set explicitly in a test entry, processingMode is compatible with both json-ld-1.0 and json-ld-1.1.

Test results that include a context input presume that the context is provided locally, and not from the referenced location, thus the results will include the content of the context file, rather than a reference.

Developers are encouraged to make a local copy of the test suite (available on GitHub) and simulate the behavior of fetching test files remotely and setting HTTP headers as described in a particular test entry.

JSON-LD Object comparison

If algorithms are invoked with the ordered flag set to true, simple JSON Object comparison may be used, as the order of all arrays will be preserved (except for fromRdf, unless the input quads are also ordered). If ordered is false, then the following algorithm will ensure arrays other than values of @list are compared without regard to order.

JSON-LD Object comparison compares JSON objects, arrays, and values recursively for equality.

Note that some tests require re-expansion and comparison, as list values may exist as values of properties that have @container: @list and the comparison algorithm will not consider ordering significant.

Running tests

The top-level manifest references the specific test manifests, which in turn reference each test associated with a particular type of behavior.

Implementations create their own infrastructure for running the test suite. In particular, the following should be considered:

Contributing Tests

If you would like to contribute a new test or a fix to an existing test, please follow these steps:

  1. Notify the JSON-LD mailing list, public-json-ld-wg@w3.org, that you will be creating a new test or fix and the purpose of the change.
  2. Clone the git repository: git://github.com/w3c/json-ld-api.git
  3. Make your changes and submit them via github, or via a 'git format-patch' to the JSON-LD Working Group mailing list.

Distribution

Distributed under the W3C Test Suite License. To contribute to a W3C Test Suite, see the policies and contribution forms.

Disclaimer

UNDER THE EXCLUSIVE LICENSE, THIS DOCUMENT AND ALL DOCUMENTS, TESTS AND SOFTWARE THAT LINK THIS STATEMENT ARE PROVIDED "AS IS," AND COPYRIGHT HOLDERS MAKE NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, OR TITLE; THAT THE CONTENTS OF THE DOCUMENT ARE SUITABLE FOR ANY PURPOSE; NOR THAT THE IMPLEMENTATION OF SUCH CONTENTS WILL NOT INFRINGE ANY THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS. COPYRIGHT HOLDERS WILL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF ANY USE OF THE DOCUMENT OR THE PERFORMANCE OR IMPLEMENTATION OF THE CONTENTS THEREOF.

Test sequence: