Test Specifications

For each normative assertion, state how it is to be tested. This should include the tools used, and links to any test cases. An assertion may require more than one test for particular configurations or combinations of options. Ideally, for each test, if possible a counterexample that fails the test should also be provided.

td-vocabulary
Test Method: Parent

This assertion will be considered satisfied if and only if all assertions that it is a parent of are satisfied. These all have ID's of the form td-vocab-*.

td-unique-identifiers
Test Method: JSON Schema
  • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
td-jsonld-keywords:
Test Method: JSON Schema
  • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
  • EXAMPLE EXISTS only for @context
  • COUNTER EXAMPLE CANNOT EXIST

Perhaps should be checked as a JSON-LD file; it is not clear that is the intent. Assertion seems ambiguous. Other assertions constrain the location and values associated with these keywords. In particular td-context restricts it, so this could be considered a parent of that assertion.

td-string-type:
Test Method: JSON Schema
  • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
  • ASSERTION NOT CLEAR: Does this mean that strings we have in TD such as "type":"number" must be "type" and not (type) or (type is number) etc.?
  • EXAMPLE DOES NOT EXIST
  • COUNTER EXAMPLE DOES NOT EXIST

Can be checked with a JSON Schema. However, the actual assertion wording is ambiguous. Should probably be "Vocabulary terms that identify values that use simple types...". On the other hand, this rule may be included in more general rules that:

  1. A JSON TD MUST be serialized as a JSON file, and
  2. Values associated with vocabulary terms MUST have the type given in the table.
    td-integer-type:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR: does this mean that strings we have in TD such as "type":123 must be 123 and not (1 2 3)?
    • EXAMPLE DOES NOT EXIST
    • COUNTER EXAMPLE DOES NOT EXIST

    See comments under td-string-type.
    td-number-type:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR: does this mean that strings we have in TD such as "type":123 must be 123 and not (1 2 3)?
    • EXAMPLE DOES NOT EXIST
    • COUNTEREXAMPLE DOES NOT EXIST

    See comments under td-string-type.
    td-context:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR: Can you have an @context key without this context file?
    • IF YES THEN TESTED WRONGLY

    Assertion should be revised to say that if the @context is used, it MUST have the given value (or the primary, without a prefix). Assuming this is what is meant, it can be checked with a JSON Schema.

    td-context-jsonld:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    td-additional-contexts:
    Test Method: JSON Schema
    • TEST EXISTS in AssertionTester
    • EXAMPLE EXISTS BUT NOT EXPLICITLY DONE FOR THIS ASSERTION
    td-context-toplevel:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • EXAMPLE EXISTS BUT NOT EXPLICITELY DONE FOR THIS ASSERTION
    • COUNTER EXAMPLE DOESN'T EXIST
    td-objects:
    Test Method: JSON Schema
    • NO TEST EXISTS
    • EXAMPLE EXISTS BUT NOT EXPLICITLY DONE FOR THIS ASSERTION
    • COUNTER EXAMPLE DOESN'T EXIST
    td-arrays:
    Test Method: JSON Schema
    • NO TEST EXISTS
    • EXAMPLE EXISTS BUT NOT EXPLICITLY DONE FOR THIS ASSERTION
    • COUNTER EXAMPLE DOESN'T EXIST
    td-properties:
    Test Method: JSON Schema
    • NO TEST EXISTS
    • TDS THAT DON'T RESPECT THIS ASSERTION ARE VALIDATED
    td-property-names:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR
    td-property-objects:
    Test Method: JSON Schema
    • TEST EXISTS in AssertionTester
    • ASSERTION NOT CLEAR: Sounds wrong since items can be in an array in JSON Schema. Currently test does NOT include items, and restriction on items MAY be wrong in assertion.
    td-property-arrays:
    Test Method: JSON Schema
    • TEST EXISTS IN AssertionTester
    • TEST IS INCOMPLETE: does not check for some of the elements (required and enum) specified in the assertion. The master JSON Schema also does not check for these.
    td-property-semantic:
    Test Method: JSON Schema?
    • NO TEST EXISTS
    • CANNOT BE TESTED
    • EXAMPLE EXISTS
    td-property-defaults:
    Test Method: Unknown
    • NO TEST EXISTS
    td-actions:
    Test Method: JSON Schema; Parent?
    • NO TEST EXISTS
    td-action-names:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR
    td-action-objects:
    Test Method: JSON Schema
    • TEST EXISTS in AssertionTester
    • EXAMPLE DOES NOT EXIST
    td-action-arrays:
    Test Method: JSON Schema
    • TEST EXISTS IN AssertionTester
    • Assertion does not include elements mentioned in td-property-arrays and td-event-arrays: required and enum. Is this correct or an oversight?
    • EXAMPLE EXISTS
    • COUNTER EXAMPLE DOES NOT EXIST
    td-action-semantic:
    Test Method: JSON Schema?
    • NO TEST EXISTS
    • CANNOT BE TESTED
    td-events:
    Test Method: JSON Schema; Parent?
    • NO TEST EXISTS
    • HOW TO TEST COUNTER EXAMPLE?
    • WHAT IS A COUNTER EXAMPLE?
    td-event-names:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR
    td-event-objects:
    Test Method: JSON Schema
    • TEST EXISTS in AssertionTester
    • EXAMPLE DOES NOT EXIST
    td-event-arrays:
    Test Method: JSON Schema
    • TEST EXISTS IN AssertionTester
    • TEST IS INCOMPLETE: does not check for some of the elements (required and enum) specified in the assertion. The master JSON Schema also does not check for these.
    • EXAMPLE DOES NOT EXIST EXPLICITLY FOR THIS
    td-event-response-arrays:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • EXAMPLE DOES NOT EXIST EXPLICITLY FOR THIS
    td-event-semantic:
    Test Method: JSON Schema?
    • NO TEST EXISTS
    • CANNOT BE TESTED
    td-forms:
    Test Method: JSON Schema
    • HOW TO TEST COUNTEREXAMPLE?
    • WHAT IS A COUNTEREXAMPLE?
    td-form-protocolbindings:
    Test Method: JSON Schema?
    • NO TEST EXISTS
    • IF JSON Schema not possible, CONSIDER TESTING WITH SHEX
    td-form-contenttype:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    td-form-links:
    Test Method: Unknown
    • HOW TO TEST COUNTEREXAMPLE?
    • WHAT IS A COUNTEREXAMPLE?
    td-security:
    Test Method: JSON Schema; Parent?
    • NO TEST EXISTS
    • HOW TO TEST COUNTEREXAMPLE?
    • WHAT IS A COUNTEREXAMPLE?
    td-security-mandatory:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • EXAMPLE EXISTS
    • COUNTEREXAMPLE DOESN'T EXIST
    td-security-binding:
    Test Method: Manual
    • For HTTP: Test with Postman and/or inspection of headers with curl.
    • For CoAP: Unknown.
    • PENETRATION TEST
    td-version:
    Test Method: JSON Schema
    • TEST EXISTS in AssertionTester
    • EXAMPLE EXISTS
    • Test exists, but may be incorrect or out of date!
    td-media-type:
    Test Method: JSON Schema
    • NO TEST EXISTS; NEEDS SCHEMA IN AssertionTester
    • ASSERTION NOT CLEAR
    td-writable-observable-default:
    Test Method: Unknown
    • NO TEST EXISTS
    td-content-type-default:
    Test Method: Unknown
    • NO TEST EXISTS
    td-jsonld-preprocessing-context:
    Test Method: Unknown
    • NO TEST EXISTS
    • HOW TO DISGUISH WHETHER IT SHOULD BE A JSON-LD WHICH LACKS THE @CONTEXT?
    td-jsonld-preprocessing-prefix:
    Test Method: Unknown
    • NO TEST EXISTS
    td-jsonld-preprocessing-defaults:
    Test Method: Unknown
    • NO TEST EXISTS
    td-forms-response:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-security-overrides:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-security-no-extras:
    Test Method: Manual
    • For HTTP: Test with Postman and/or inspection of headers with curl.
    • For CoAP: Unknown.
    • PENETRATION TEST
    td-readOnly-observable-default:
    Test Method: Unknown
    • NO TEST EXISTS
    td-vocab-security:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-version:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-actions:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-events:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-base:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-links:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-name:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-lastModified:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-created:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-description:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-properties:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-support:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-securityDefinitions:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-uriVariables:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-title:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-scopes:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-forms:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-observable:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-idempotent:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-safe:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-output:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-input:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-subscription:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-data:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-cancellation:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-href:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-op:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-subprotocol:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-contentType:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-response:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-type:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-anchor:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-rel:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-instance:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-readOnly:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-const:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-unit:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-enum:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-writeOnly:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-oneOf:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-minItems:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-items:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-maxItems:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-maximum:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-minimum:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-required:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-scheme:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-proxy:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-in:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-qop:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-format:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-alg:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-authorization:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-identity:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-token:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-flow:
    Test Method: JSON Schema
    • NO TEST EXISTS
    td-vocab-refresh:
    Test Method: JSON Schema
    • NO TEST EXISTS
    client-data-schema:
    Test Method: Unknown
    • NO TEST EXISTS
    client-uri-template:
    Test Method: Unknown
    • NO TEST EXISTS