Proposal Status: Rejected

This document summarizes the discussions on simplifying the JSON-LD 1.0-based WoT Thing Description and proposes a simplified TD serialization based on JSON-LD 1.1. The proposal lists the features of the TD core model and gives use cases per feature. The proposed "Simplified TD" serialization is given in the form of extensive examples. Specification text will be added when merging this proposal into the WoT Thing Description Editors' Draft. For convenience, examples of the JSON-LD 1.0 serialization and the alternative JSON Web Thing Description (a.k.a. JSON-TD) are given for comparison. This document concludes with a list of differences in the proposals.

Introduction

The WoT Thing Description based on JSON-LD 1.0 has been criticized for several limitations including complexity and the JSON-LD 1.0 array structures and keywords even when no semantic annotations are used. This lead to an alternative draft, the JSON Web Thing Description, with a pure JSON serialization, which however, would fragment W3C WoT due to incompatible models behind the serializations. To remedy the situation, this document proposes a simplified TD serialization based on JSON-LD 1.1, thereby combining both requirements: machine-understandability through a formal Linked Data model from the original WoT Thing Description and improved usability from the JSON Web Thing Description. Being able to conduct these simplifications first required a full understanding of the requirements, which have been analyzed by the Working Group since its chartering. This technical deep-dive completed around TPAC 2017 (Burlingame), where the Working Group announced to change the goal toward simplification from there onward.

Use Cases per TD Feature

Many discussions are rooted on the question whether a certain feature should be included or not. This section lists the features currently included in the TD core model and backs them by use cases.

Interaction Model

To represent a contract between applications with programming APIs and IoT devices with network APIs, the communication with Things must be modeled with a formal interaction model. This describes not only what data a Thing provides or requires, but also how to access and manipulate the corresponding internal state of a Thing.

Based on the Web Thing Model and results of the COMPOSE project the WoT Interest Group explored a threefold of Interactions, which was adopted by the WoT Working Group after 2 years of stability:

Properties

Properties expose internal state of a Thing that can be directly accessed (get) and optionally manipulated (set). Things can also choose to make Properties observable by pushing the new state (not an event) after a change; this must follow eventual consistency (also see CAP Theorem).

  • Read-only Properties: state cannot be changed from the outside
    • sensors
    • results computed from internal state
  • Writable Properties: state needs to be changed directly from the outside
    • configuration state
    • setpoints
    • more concise model when intended state can be applied to actual state immediately (no Action needed)
  • Observable Properties: optimize communication through server push
    • keep state in sync with relaxed consistency (eventual consistency)
    • directly maps to CoAP Observe
  • TODO: Recursive Properties: address sub-properties of a complex Property (to be done for Simplified TD)
    • nested Properties that are also exposed by the Thing (e.g., LWM2M Devices/IPSO Objects)

Actions

Actions offer functions of the Thing. These functions may manipulate the interal state of a Thing in a way that is not possible through setting Properties. Examples are changing internal state that is not exposed as Property, changing multiple Properties, changing Properties over time or with a process that shall not be disclosed. Actions may also be pure functions, that is, they do not use any internal state at all, e.g., for processing input data and returning the result directly.

  • Action arguments: the function needs input parameters
    • write-multiple commands
    • functions with parameters
  • Action results: the function returns a result
    • read-multiple commands (e.g., CoAP FETCH)
    • sensors that are not based on sampling / fit for Properties (e.g., power-intensive sensors)
    • function reports (e.g., calibration Action)
    • processing functions (e.g., data conversion)
  • TODO: Monitorable Actions: process can be monitored, changed, canceled once invoked (not in TD core model yet)
    • physical processes (i.e., intended state can not be applied immediately to actual state)
    • cloud services (e.g., client needs to be sure Thing successfully completes Action)
    • user interfaces (e.g., progress bars)

Events

The Event Interaction Pattern describes event sources that asynchronously push messages. Here not state, but state transitions (events) are communicated (e.g., "clicked"). Events may be triggered by internal state changes that are not exposed as Properties. Events usually follow strong consistency, where messages need to be queued to ensure eventual delivery of all occured events.

  • Events opposed to observable Properties: why two mechanisms
    • Events triggered by state that is not exposed through Properties
    • Separation of consistency requirements
    • Duality of REST and PubSub systems
  • TODO: Actionable Events: Event not only has data but also Interactions
    • alerts that need to be confirmed and later marked as resolved
    • events that contain dynamic state (Properties)

Data Model

Interactions exchange data using representation formats (e.g., JSON), which are identified by Media Types (e.g., application/json. Usually, applications use generic serialization formts without application-specific definitions (cf. JSON). Thus, TDs also needs to include metadata about the data model used, so-called schemas.

Representation Formats

JSON reflects the state of the art of data interchange formats. It is not focused on how data is represented by a specific system (e.g., in 16 bits), but only on the fundamentals of the information.

  • Dominant data format on the Web
  • Implementation-independent representation of data

CBOR is a concise binary representation that is related to JSON. It can express all JSON structures, but also has additional features such as a binary type to improve over base64-encoded blobs.

  • Compact
  • Low parsing complexity

Schema

A popular validation schema for JSON data is JSON Schema. It defines a vocabulary and a syntax to specify a schema for data validation at the syntactic level.

The Web of Things aims at semantic interoperability for Things. For this,

Directly using the JSON Schema syntax within the TD breaks paradigms. Two different implementations would be needed to conduct validation. At least JSON Schema could be reused. However, semantic queries and resoning could not work on data structures, that is, make sense of the data in relation to the Thing and its context.

  • Re-use of JSON Schema vocabulary
  • Enable validation through JSON-LD

Security Metadata

The TD has a top-level security field to carry security metadata such as the required authorization mechanism, authorization server, required root certificates, etc.

Communication Metadata

The TD Interactions have a form field to carry protocol-specific metadata such as method used and header fields required. The concept of machine understandable forms matches the requirement of RAML, Swagger, OpenAPI for RESTful Web APIs.

The W3C Web of Things was charted to counter the fragmentation in the IoT. Creating yet another standard does not help here, as existing standards with strong domain-specific knowledge have shown to hold out. Thus, W3C WoT was charted to describe and complement existing IoT platforms and standards. For this, a TD must be able to describe how a client can communicate successfully with a server.

Request Metadata

Up to date, not even so-called RESTful Web services managed to follow a single guidline when designing their API. Requests always have to be tailored for each service (cf. usage of Swagger/OpenAPI).

To reflect this situation in the TD, where a situation similar to RESTful APIs is expected for Things, a TD is able to include so called form descriptions, which go beyond Web linking: The not only describe the relation to a target resource and attributes of the target resource, but also descriptions how to interact with that resource, that is, how a client has to formulate requests (e.g., header fields or payloads).

  • Compensate for small variations in methods
  • Compensate for small variations in header fields

Extension Point

So far, there is no single IoT protocol to rule them all. The only convergence visible is a trend toward Internet protocols defined by the IETF and other bodies such as OASIS. Yet still every IoT platform and standard tends to introduce small variations, thereby creating protocol dialects.

To allow for convergence of currently deployed IoT protocols and to be future proof for new protocols that might emerge as de facto standards, the TD has protocol bindings as explicit extension point.

  • Use HTTP long-poll for Events
  • Use HTTP Server-Sent Events for Events
  • Use WebSocket sub-protocols that do not have a URI scheme
  • Use CoAP dialects such as OCF or oneM2M
  • Use protocols of large IoT platforms such as Amazon IoT or Azure IoT

Binding Templates

The WoT Binding Templates document serves as catalog to pick the right communication metadata for well-known IoT protocols. The templates provide vocabulary to describe the required protocol stack configuration (e.g., specific header fields) and security mechanisms in the TD.

Binding Templates are mainly expected for dialects of the Web protocols HTTP/1.1, HTTP/2, and CoAP, standardized sub-protocols for WebSockets, and widely deployed IoT protocols such as AMQP or MQTT profiles (e.g., Amazon IoT).

  • Provide a common vocabulary for communication metadata

Semantic Annotations

The Web of Things is centered on things. During runtime, data and metadata are primarily exchanged among machines for machines. Furthermore, WoT content is produced by a large number of machines. Thus, machine assistance is also needed for consuming the data and bulding viable business cases.

Annotation Framwork

Semantic annotations are based on JSON-LD, which gives meaning to JSON field names and values by defining the terms used in the @context and linking them to Link Data. TDs can be annotated as follows:

  • Thing types in the top-level @type array
  • Thing capabilities
  • Thing metadata in additional top-level fields that follow the rules of JSON-LD
  • Interaction types in the @type array within Interaction elements
  • Interaction metadata in additional sub-fields that follow the rules of JSON-LD
  • Data types in the @type array within schema

Extension Point

The W3C has no knowledge in the concrete application domains of Web of Things. Thus, the W3C is not standardizing any domain-specific vocabulary.

Still, domain-specific vocabularies are of utmost importance for semantic interoperability. Thus, the TD has an explicit extension point that comes for free when using JSON-LD and Linked Data.

Web Linking

The TD now has a top-level link field to carry links to other Web resources.

Serialization Requirements

Human-readability

The Web of Things is supposed to make IoT technology more usable, which includes usability for developers. This is fundamental to the success of Web Technology in general.

This requirement lead to the choice of a JSON-based format for the TD.

Machine-understandability

The Web of Things is centered on things. During runtime, data and metadata are primarily exchanged among machines for machines. Furthermore, WoT content is produced by a large number of machines. Thus, machine assistance is also needed for consuming the data and bulding viable business cases.

Thus, TDs must not only be machine-readible, but machine-understandable. Moreover, the TD metadata can help to make optimized runtime data (i.e., no metadata included in the live data exchanged) also machine-understandable.

This requirement lead to the choice of JSON-LD, an existing W3C standard for machine-understandable JSON documents based on Linked Data vocabularies.

Introspection

Applications consuming TDs should be able to easily introspect the capabilites, possible interactions, and other metadata of the corresponding Things.

In particular the WoT Scripting API benefits from an alternative structure of the TD, such as separated entry points for Properties, Actions, and Events.

Default Values

Implementations for the PlugFest showed that basically every developer is counting on defaults, that is, certain elements can be omitted as their meaning is known implicitly or even irrelevant. Examples are omitting the security metadata for test Things that use no-security, omitting the writable flag for read-only Properties, or omitting inputSchema/outputSchema for Actions that do not take arguments and/or return nothing.

Treating the JSON-LD TD strictly as Linked Data document, a collection of triples, would not allow for such defaults, and hence convenience. Linked Data underly the open-world assumption. That means, when a TD is omitting statements (e.g., writable flag), then the consumer cannot make any assumptions about its actual value.

Based on the knowledge that it is a TD, the consumer could, however, apply a set rules, which infer the defaults. This leads to the preprocessing introduced in the next section.

Robustness Principle

Be conservative in what you do, be liberal in what you accept from others. Applying Postel's law fosters interoperability and should also be used in the Web of Things.

This requirement also leads to a preprocessing step that can fill in defaults and correct minor issues.

Simplified TD

The Simpified TD proposal aims at solving the requirements given in the previous section.

Preprocessing

The main change and enabler for a simplified TD is introducing a preprocessing step.

Due to the open-world assumption, some preprocessing of the TD is required. The rules to fill in defaults could be applied in the following way:

Normalizing the TD in a preprocessing step also helps for robustness. Thus, this is a favorable solution.

JSON-LD 1.1 Frameing can be used to define and apply the preprocessing steps.

NOTE: Even with JSON-LD 1.1 available, there are still open issues. For instance, the object key becomes the @id of the Property node defined in the object value: To derive sensible triples, the object key would need to be declared either as "blank node" by prefixing with _:, or the container mechanism must be able generate relative @ids to avoid conflicts when an object key appears twice at different levels.

Media Type

When TDs are not a valid JSON-LD document due to necessary preprocessing or simply an omitted @context, they need their own media type registered with IANA.

We propose application/td+json.

New Terms

link

The Working Draft for the Prague 2018 PlugFest already adopted a top-level link field, so that a TD can express relations to other Things or other Web resources.

Note that because the TD is JSON-LD, such relations were always possible and are actually always expressed for all top-level fields (it is Linked Data). Having the term link makes it explicit that the elements included there are actual Web Links to fetchable Web resources, opposed to Linked Data URIs ("IRIs") that are often only an identifier without an actual Web resource.

description

The term description is very useful and shall become part of the TD core vocabulary for both Thing and Interactions.

name

The term name would not be the identifier for Interactions anymore. Following JSON-LD 1.1, the object key would become @id, which would serve as handle to select Interactions, e.g., for the Scripting API.

With that, name becomes either obsolete or rather could be used as optional, human-readible label. As a result, for Things, name becomes a mandatory label. For Interactions, the new, optional term label is used.

id

On the top level, id will become important for security and identity management. It identifies the instance of Thing and also serves as the @base for all other @id definitions (e.g., of Interactions). Only this fully enables the object notation of Interactions (opposed to the previous array notation). The top-level id must be mandatory and internally maps to the @id of the Thing. Therefore, a Thing id must be a URI (which includes URNs).

For Interactions, @id would be the handle for scripts and it would enable references to and between Interactions, e.g., for cross-Property constraints or for expressing relations between an action and one or more Properties. Note that this would result automatically from the object structure listing Interactions because of JSON-LD 1.1 (Node Identifier Indexing).

Model Changes

To enable alignment with other technologies such as JSON Schema, the Simplified TD needs a few small changes to the TD core model. It has to be evaluated if these changes are actually breaking changes (i.e., a JSON-LD 1.0 TD would create different triples than a Simplified TD).

Properties

Using the term properties also for the object field names of complex Properties (i.e., "type": "object") aligns the TD with JSON Schema syntax. This results in recursive Properties, where

  • Properties also have the DataSchema RDF-type
  • Only properties that have a forms field are Interaction Properties.
  • Sub-properties of complex Properties might be directly addressable and accessible, when they have the forms field.

Simplified TD by Example

When no @context is given, the Simplified TD context is assumed based on the media type (file location needs update for adoption). The current TD context also needs an additional definition of @base set to the Thing's @id, so that the object representation of Interactions produces correct @ids for the Interactions. 

{
  "@context": {
    /* enable JSON-LD 1.1 processing */
    "@version": 1.1,
    /* valid @id generation relative to Thing instance */
    "@base": "@id",
    /* object notation for Interactions */
    "properties": {
      "@container": "@id"
    },
    "actions": {
      "@container": "@id"
    },
    "events": {
      "@container": "@id"
    },
    /* optional if schema definitions are wanted */
    "definitions": {
      "@container": "@id"
    },
    /* TD term definitions as in https://w3c.github.io/wot/w3c-wot-td-context.jsonld */
    ...
  }
}

When @context is given, the Simplified TD context is applied last to ensure that the terms of the TD core vocabulary are valid and not shadowed by other contexts.

Minimal Simplified TD

The first example reflects the minimal option. It contains no semantic annotations, nor human-readible decorators such as name or description. 

{
  "name": "Lamp",
  "id": "urn:dev:wot:com:example:servient:lamp",
  "base": "https://servient.example.com/",
  "properties": {
    "on": {
      "type": "boolean",
      "forms": [{
        "href": "/things/lamp/properties/on",
        "mediaType": "application/json"
      }]
    },
    "status": {
      "readOnly": true,
      "type": "object",
      "properties": {
        "battery": {
          "type": "number",
          "minimum": 0.0,
          "maximum": 100.0,
          "forms": [{
            "href": "/things/lamp/properties/status/batt",
            "mediaType": "application/json"
          }]
        },
        "rssi": {
          "type": "number",
          "minimum": 0.0,
          "maximum": 1.0
        },
        "level": {
          "type": "integer",
          "minimum": 0,
          "maximum": 100
        }
      },
      "forms": [{
        "href": "/things/lamp/properties/status",
        "mediaType": "application/json"
      }]
    },
    "brightness": {
      "type": "integer",
      "minimum": 0,
      "maximum": 100
      "forms": [{
        "href": "/things/lamp/properties/brightness",
        "mediaType": "application/json"
      }]
    }
  },
  "actions": {
    "fade": {
      "input": {
        "type": "object",
        "properties": {
          "from": {
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          },
          "to": {
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          },
          "duration": { "type": "number" }
        }
      },
      "forms": [{
        "href": "/things/lamp/actions/fade",
        /* encType would be for the request body opposed to mediaType, which is for target */
        "encType": "application/json",
        "mediaType": "application/json"
      }]
    }
  },
  "events": {
    "overheated": {
      "type": "object",
      "properties": {
        "temperature": { "type": "number" }
      },
      "forms": [{
        "href": "/things/lamp/events/overheated",
        "http:subProtocol": "http:EventSource",
        "mediaType": "application/json"
      }]
    }
  },
  "links": [{
    "href": "https://servient.example.com/things/motion-detector",
    "rel": "controlledBy",
    "mediaType": "application/td+json"
  }]
}

Annotated Simplified TD (with semantics)

{
  "@context": { "iot": "https://iot.schema.org/" },
  "name": "Lamp",
  "description": "Corner torchiere",
  "@type": ["Thing", "iot:Light"],
  "id": "urn:dev:wot:com:example:servient:lamp",
  "base": "https://servient.example.com/",
  "properties": {
    "on": {
      "label": "On/Off",
      "description": "Power switch",
      "@type": "iot:SwitchStatus",
      "type": "boolean",
      "forms": [{
        "href": "/things/lamp/properties/on",
        "mediaType": "application/json"
      }]
    },
    "status": {
      "readOnly": true,
      "@type": "iot:DeviceStatus",
      "type": "object",
      "properties": {
        "battery": {
          "@type": "iot:BatteryLevel",
          "type": "number",
          "minimum": 0.0,
          "maximum": 100.0,
          "forms": [{
            "href": "/things/lamp/properties/status/batt",
            "mediaType": "application/json"
          }]
        },
        "rssi": {
          "@type": "iot:SignalStrength",
          "type": "number",
          "minimum": 0.0,
          "maximum": 1.0
        },
        "level": {
          "@type": "iot:CurrentLevel",
          "type": "integer",
          "minimum": 0,
          "maximum": 100
	}
      },
      "forms": [{
        "href": "/things/lamp/properties/status",
        "mediaType": "application/json"
      }]
    },
    "brightness": {
      "@type": "iot:CurrentLevel",
      "type": "integer",
      "minimum": 0,
      "maximum": 100,
      "forms": [{
        "href": "/things/lamp/properties/status",
        "mediaType": "application/json"
      }]
    }
  },
  "actions": {
    "fade": {
      "input": {
        "type": "object",
        "properties": {
          "from": {
            "@type": "iot:CurrentLevel",
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          },
          "to": {
            "@type": "iot:TargetLevel",
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          },
          "duration": {
	    "@type": "iot:TransitionTimeData",
	    "type": "number"
	  }
        }
      },
      "forms": [{
        "href": "/things/lamp/actions/fade",
        /* encType would be for the request body
           opposed to mediaType, which is for target */
        "encType": "application/json",
        "mediaType": "application/json",
        /* redundant to default here, just to indicate
           imagine a PATCH Action or a CoAP FETCH Action */
        "http:methodName": "POST"
      }]
    }
  },
  "events": {
    "overheated": {
      "type": "object",
      "properties": {
        "temperature": { "type": "number" }
      },
      "forms": [{
        "href": "/things/lamp/events/overheated",
        "http:subProtocol": "http:EventSource",
        "mediaType": "application/json"
      }]
    }
  },
  "links": [{
    "href": "https://servient.example.com/things/motion-detector",
    "rel": "controlledBy",
    "mediaType": "application/td+json"
  }]
}

JSON-LD 1.0 TD

This section summerizes the status of the JSON-LD 1.0-based Thing Description (until Prague 2018 WD).

JSON-LD 1.0 TD by Example

Minimal JSON-LD

This example reflects the minimal option. It contains no semantic annotations, nor human-readible decorators such as name or description. 

  {
    "@context": "https://w3c.github.io/wot/w3c-wot-td-context.jsonld",
    "name": "Lamp",
    "@id": "urn:dev:wot:com:example:servient:lamp",
    "base": "https://servient.example.com/",
    "interaction": [{
      "@type": "Property",
      "name": "on",
      "writable": true,
      "schema": { "type": "boolean" },
      "form": [{ "href": "/things/lamp/properties/on" }]
    }, {
      "@type": "Property",
      "name": "status",
      "writable": false,
      "schema": {
        "type": "object",
	"field": [{
	  "name": "battery",
	  "type": "number",
          "minimum": 0.0,
          "maximum": 100.0
	}, {
	  "name": "rssi",
	  "type": "number",
          "minimum": 0.0,
          "maximum": 1.0
	}, {
	  "name": "level",
	  "type": "integer",
          "minimum": 0,
          "maximum": 100
	}]
      },
      "form": [{ "href": "/things/lamp/properties/on" }]
    }, {
      "@type": "Property",
      "name": "brightness",
      "writable": true,
      "schema": {
        "type": "number",
        "minimum": 0,
        "maximum": 100
      },
      "form": [{ "href": "/things/lamp/properties/brightness" }]
    }, {
      "@type": "Action",
      "name": "fade",
      "inputSchema": {
        "type": "object",
        "field": [{
          "name": "from",
          "schema": {
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          }
        }, {
          "name": "to",
          "schema": {
            "type": "integer",
            "minimum": 0,
            "maximum": 100
          }
        }, {
          "name": "duration",
          "schema": { "type": "number" }
        }]
      },
      "form": [{ "href": "/things/lamp/actions/fade" }]
    }, {
      "@type": "Event",
      "name": "overheated",
      "schema": {
        "type": "object",
        "field": [{
          "name": "temperature",
          "schema": { "type": "number" }
        }]
      },
      "form": [{ "href": "/things/lamp/events/overheated" }]
    }],
    "link": [{
      "href": "https://servient.example.com/things/motion-detector",
      "rel": "controlledBy",
      "mediaType": "application/ld+json"
    }]
  }

Annotated JSON-LD 1.0 TD

JSON Web Thing Description (JSON-TD)

The following JSON example was build from snippets taken from here.

JSON-TD by Example

{
  "name": "My Lamp",
  "type": "thing",
  "description": "A web connected lamp",
  "properties": {
    "on": {
      "type": "boolean",
      "description": "Whether the lamp is turned on",
      "href": "/things/lamp/properties/on"
    },
    "brightness": {
      "type": "number",
      "description": "The level of light from 0-100",
      "minimum": 0,
      "maximum": 100,
      "href": "/things/lamp/properties/brightness"
    }
  },
  "actions": {
    "fade": {
      "name": "Fade",
      "description": "Fade from one brightness to another",
      "data": {
        "from": {
          "type": "number",
          "minimum": 0,
          "maximum": 100
        },
        "to": {
          "type": "number",
          "minimum": 0,
          "maximum": 100
        },
        "duration": {
          "type": "number",
          "unit": "milliseconds"
        }
      },
      "href": "/things/lamp/actions/fade"
    }
  },
  "events": {
    "overheated": {
      "name": "Overheated",
      "description": "The device exceeded its maximum safe operating temperature",
      "data": {
        "temperature": {
          "type": "number",
          "unit": "celcius"
        }
      }
    }
  },
  "links": {
    "properties": "/thing/lamp/properties",
    "actions": "/things/lamp/actions",
    "events": "/things/lamp/events"
  }
}

Comparing "Progressive Simplified TD" to "JSON-TD"