W3C

Reporting API

Editor’s Draft,

This version:
https://w3c.github.io/reporting/
Version History:
https://github.com/w3c/reporting/commits/master/index.src.html
Editors:
(Google Inc.)
(Google Inc.)
(Google Inc.)
(Google Inc.)
Participate:
File an issue (open issues)

Abstract

This document defines a generic reporting framework which allows web developers to associate a set of named reporting endpoints with an origin. Various platform features (like Content Security Policy, Network Error Reporting, and others) will use these endpoints to deliver feature-specific reports in a consistent manner.

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

This document was published by the Web Performance Working Group as an Editors Draft. This document is intended to become a W3C Recommendation.

Feedback and comments on this specification are welcome, please send them to public-web-perf@w3.org (subscribe, archives) with [reporting] at the start of your email’s subject.

Publication as an Editors Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 February 2018 W3C Process Document.

1. Introduction

[INTRODUCTION GOES HERE]

1.1. Guarantees

This specification aims to provide a best-effort report delivery system that executes out-of-band with website activity. The user agent will be able to do a better job prioritizing and scheduling delivery of reports, as it has an overview of cross-origin activity that individual websites do not, and can deliver reports based on error conditions that would prevent a website from loading in the first place.

The delivery is not, however, guaranteed in a strict sense. We spell out a reasonable set of retry rules in the algorithms below, but it’s quite possible for a report to be dropped on the floor if things go badly.

Reporting can generate a good deal of traffic, so we allow developers to set up groups of endpoints, using a failover and load-balancing mechanism inspired by the DNS SRV record. The user agent will do its best to deliver a particular report to at most one endpoint in a group. Endpoints can be assigned weights to distribute load, with each endpoint receiving a specified fraction of reporting traffic. Endpoints can be assigned priorities, allowing developers to set up fallback collectors that are only tried when uploads to primary collectors fail.

1.2. Examples

MegaCorp Inc. wants to collect Content Security Policy and Key Pinning violation reports. It can do so by delivering the following header to define a set of reporting endpoints named "endpoint-1":
Report-To: { "group": "endpoint-1",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/reports", "priority": 1 },
               { "url": "https://backup.com/reports", "priority": 2 }
             ] }

And the following headers, which direct CSP and HPKP reports to that group:

Content-Security-Policy: ...; report-to=endpoint-1
Public-Key-Pins: ...; report-to=endpoint-1
After processing reports for a little while, MegaCorp Inc. decides to split the processing of these two types of reports out into two distinct endpoints in order to make the processing scripts simpler. It can do so by delivering the following header to define two reporting endpoints:
Report-To: { "group": "csp-endpoint",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/csp-reports" }
             ] },
           { "group": "hpkp-endpoint",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/hpkp-reports" }
             ] }

And the following headers, which direct CSP and HPKP reports to those named endpoint:

Content-Security-Policy: ...; report-to=csp-endpoint
Public-Key-Pins: ...; report-to=hpkp-endpoint

2. Concepts

2.1. Clients

A client represents a particular origin’s relationship to a set of endpoints.

Each client has an origin, which is an origin.

Each client has an endpoint-groups list, which is a list of endpoint groups, each of which MUST have a distinct name. (The algorithm in §3.2 Process reporting endpoints for response to request guarantees this by keeping the first entry in a Report-To header with a particular name.)

2.2. Endpoint groups

An endpoint group is a set of endpoints that will be used together for backup and failover purposes.

Each endpoint group has a name, which is an ASCII string.

Each endpoint group has an endpoints list, which is a list of endpoints.

Each endpoint group has a subdomains flag, which is either "include" or "exclude".

Each endpoint group has a ttl representing the number of seconds the group remains valid for an origin.

Each endpoint group has a creation which is the timestamp at which the group was added to an origin.

An endpoint group is expired if its creation plus its ttl represents a time in the past.

2.3. Endpoints

An endpoint is location to which reports for a particular origin may be sent.

Each endpoint has a url, which is a URL.

Each endpoint has a priority, which is a non-negative integer.

Each endpoint has a weight, which is a non-negative integer.

Each endpoint has a failures, which is a non-negative integer representing the number of consecutive times this endpoint has failed to respond to a request.

Each endpoint has a retry_after, which is either null, or a timestamp after which delivery should be retried.

An endpoint is pending if its retry_after is not null, and represents a time in the future.

2.4. Report Type

A report type is a non-empty string that specifies the set of data that is contained in the body of a report.

When a report type is defined (in this spec or others), it can be specified to be visible to ReportingObservers, meaning that reports of that type can be observed by a reporting observer. By default, report types are not visible to ReportingObservers.

Note: See §6.1 Deprecation as an example report type definition.

2.5. Reports

A report is a collection of arbitrary data which the user agent is expected to deliver to a specified endpoint.

Each report has a body, which is either null or an object which can be serialized into a JSON text. The fields contained in a report’s body are determined by the report’s type.

Each report has a url, which is typically the address of the Document or Worker from which the report was generated.

Note: We strip the username, password, and fragment from this serialized URL. See §10.1 Capability URLs.

Each report has a user agent, which is the value of the User-Agent header of the request from which the report was generated.

Note: The user agent of a report represents the User-Agent sent by the browser for the page which generated the report. This is potentially distinct from the User-Agent sent in the HTTP headers when uploading the report to a collector — for instance, where the browser has chosen to use a non-default User-Agent string such as the "request desktop site" feature.

Each report has a group, which is a string representing the name of the origin’s endpoint group that the report will be sent to.

Each report has a type, which is a report type.

Each report has a timestamp, which records the time at which the report was generated, in milliseconds since the unix epoch.

Each report has an attempts counter, which is a non-negative integer representing the number of times the user agent attempted to deliver the report.

2.6. Storage

A conformant user agent MUST provide a reporting cache, which is a storage mechanism that maintains a set of endpoint groups that websites have instructed the user agent to associate with their origins, and a set of reports which are queued for delivery.

This storage mechanism is opaque, vendor-specific, and not exposed to the web, but it MUST provide the following methods which will be used in the algorithms this document defines:

  1. Insert, update, and remove clients.

  2. Enqueue and dequeue reports for delivery.

  3. Retrieve a list of client objects for an origin.

  4. Retrieve a list of queued report objects.

  5. Clear the cache.

2.7. Failover and load balancing

The endpoints in an endpoint group that all have the same priority form a failover class. Failover classes allow the developer to provide backup collectors (those with higher priority values) that will only receive reports if all of the primary collectors (those with lower priority values) fail.

Developers can assign each endpoint in a failover class a weight, which determines how report traffic is balanced across the failover class.

The algorithm that implements these rules is described in §4.3 Choose an endpoint from a group.

Note: The priority and weight fields have the same semantics as the corresponding fields in a DNS SRV record.

Note: Failover and load balancing is a feature that would be generally useful outside of Reporting. Reporting delegates to the [FETCH] API to actually upload reports once an endpoint has been selected. If, in the future, the Fetch API adds native support for failover and load balancing of requests, a future version of the Reporting API will be updated to use it instead of this bespoke mechanism.

3. Endpoint Delivery

A server MAY define a set of reporting endpoints for an origin it controls via the Report-To HTTP response header field. This mechanism is defined in §3.1 The Report-To HTTP Response Header Field, and its processing in §3.2 Process reporting endpoints for response to request.

The Report-To HTTP response header field instructs the user agent to store reporting endpoints for an origin. The header is represented by the following ABNF grammar [RFC5234]:

Report-To = json-field-value
            ; See Section 2 of [[HTTP-JFV]], and Section 2 of [[RFC8259]]

The header’s value is interpreted as a JSON-formatted array of objects without the outer [ and ], as described in Section 4 of [HTTP-JFV].

Each object in the array defines an endpoint group to which reports may be delivered, and will be parsed as defined in §3.2 Process reporting endpoints for response to request.

The following subsections define the initial set of known members in each JSON object the header’s value defines. Future versions of this document may define additional such members, and user agents MUST ignore unknown members when parsing the header.

3.1.1. The group member

The OPTIONAL group member is a string that associates a name with the endpoint group. The member’s value MUST be a string; any other type will result in a parse error. If no member named "group" is present in the object, the endpoint group will be given the name "default".

3.1.2. The include_subdomains member

The OPTIONAL include_subdomains member is a boolean that enables this endpoint group for all subdomains of the current origin’s host. If no member named "include_subdomains" is present in the object, or its value is not "true", the endpoint group will not be enabled for subdomains.

3.1.3. The max_age member

The REQUIRED max_age member defines the endpoint group’s lifetime, as a non-negative integer number of seconds. The member’s value MUST be a non-negative number; any other type will result in a parse error.

A value of "0" will cause the endpoint group to be removed from the user agent’s reporting cache.

3.1.4. The endpoints member

The REQUIRED endpoints member defines the list of endpoints that belong to this endpoint group. The member’s value MUST be an array of JSON objects.

The following subsections define the initial set of known members in each JSON object in the array. Future versions of this document may define additional such members, and user agents MUST ignore unknown members when parsing the elements of the array.

3.1.5. The endpoints.url member

The REQUIRED url member is a string that defines the location of the endpoint. The member’s value MUST be a string; any other type will result in a parse error.

Moreover, the URL that the member’s value represents MUST be potentially trustworthy [SECURE-CONTEXTS]. Non-secure endpoints will be ignored.

3.1.6. The endpoints.priority member

The OPTIONAL priority member is a number that defines which failover class the endpoint belongs to. The member’s value, if present, MUST be a non-negative integer; any other type will result in a parse error.

3.1.7. The endpoints.weight member

The OPTIONAL weight member is a number that defines load balancing for the failover class that the endpoint belongs to. The member’s value, if present, MUST be a non-negative integer; any other type will result in a parse error.

3.2. Process reporting endpoints for response to request

Given a response (response) and a request (request), this algorithm extracts a list of endpoints and endpoint groups for the request’s origin, and updates the reporting cache accordingly.

Note: This algorithm is called from around step 13 of main fetch [FETCH], and only updates the reporting cache if the response has been delivered securely.

Fetch monkey patching. Talk to Anne.

  1. Abort these steps if any of the following conditions are true:

    1. response’s HTTPS state is not "modern", and the origin of response’s url is not potentially trustworthy.

    2. response’s header list does not contain a header whose name is "Report-To".

  2. Let origin be the origin of response’s url.

  3. Let header be the value of the header in response’s header list whose name is "Report-To".

  4. Let list be the result of executing the algorithm defined in Section 4 of [HTTP-JFV] on header. If that algorithm results in an error, abort these steps.

  5. Let groups be an empty list.

  6. For each item in list:

    1. If item has no member named "max_age", or that member’s value is not a number, skip to the next item.

    2. If item has no member named "endpoints", or that member’s value is not an array, skip to the next item.

    3. Let name be item’s "group" member’s value if present, and "default" otherwise.

    4. If there is already an endpoint group in groups whose name is name, skip to the next item.

    5. Let endpoints be an empty list.

    6. For each endpoint item in the value of item’s "endpoints" member:

      1. If endpoint item has no member named "url", or that member’s value is not a string, skip to the next endpoint item.

      2. If endpoint item has a member named "priority", whose value is not a non-negative integer, skip to the next endpoint item.

      3. If endpoint item has a member named "weight", whose value is not a non-negative integer, skip to the next endpoint item.

      4. Let endpoint be a new endpoint whose properties are set as follows:

        url

        The result of executing the URL parser on endpoint item’s "url" member’s value.

        priority

        The value of the endpoint item’s "priority" member, if present; 1 otherwise.

        weight

        The value of the endpoint item’s "weight" member, if present; 1 otherwise.

        failures

        0

        retry_after

        null

      5. Add endpoint to endpoints.

    7. Let group be a new endpoint group whose properties are set as follows:

      name

      name

      subdomains

      "include" if item has a member named "include_subdomains" whose value is true, "exclude" otherwise.

      ttl

      item’s "max_age" member’s value.

      creation

      The current timestamp

      endpoints

      endpoints

    8. Add group to groups.

  7. Let client be a new client whose properties are set as follows:

    origin

    origin

    endpoint-groups

    groups

  8. If there is already an entry in the reporting cache for origin, replace it with client. Otherwise, insert client into the reporting cache for origin.

4. Report Delivery

Over time, various features will queue up a list of reports in the user agent’s reporting cache. The user agent will periodically grab the list of currently pending reports, and deliver them to the associated endpoints. This document does not define a schedule for the user agent to follow, and assumes that the user agent will have enough contextual information to deliver reports in a timely manner, balanced against impacting a user’s experience.

That said, a user agent SHOULD make a effort to deliver reports as soon as possible after queuing, as a report’s data might be significantly more useful in the period directly after its generation than it would be a day or a week later.

4.1. Media Type

The media type used when POSTing reports to a specified endpoint is application/reports+json.

4.2. Queue data as type for endpoint group on settings

Given a serializable object (data), a string (type), another string (endpoint group), an environment settings object (settings), and an optional URL (url), the following algorithm will create a report, and add it to reporting cache’s queue for future delivery.

  1. Let report be a new report object with its values initialized as follows:

    body

    data

    user agent

    The current value of navigator.userAgent

    group

    endpoint group

    type

    type

    timestamp

    The current timestamp.

    attempts

    0

  2. If url was not provided by the caller, let url be settings’s creation URL.

  3. Set url’s username to the empty string, and its password to null.

  4. Set report’s url to the result of executing the URL serializer on url with the exclude fragment flag set.

  5. Append report to the reporting cache.

  6. Let environment be settings’s realm execution context’s realm’s ECMAScript global environment.

  7. Execute §5.2 Notify reporting observers on environment with report with environment and report.

Note: reporting observers can only observe reports from the same environment settings object.

Note: We strip the username, password, and fragment from the serialized URL in the report. See §10.1 Capability URLs.

Note: The user agent MAY reject reports for any reason. This API does not guarantee delivery of arbitrary amounts of data, for instance.

Note: Non user agent clients (with no JavaScript engine) should not interact with reporting observers, and thus should return in step 6.

4.3. Choose an endpoint from a group

Note: This algorithm is the same as the target selection algorithm used for DNS SRV records.

Given an endpoint group (group), this algorithm chooses an arbitrary eligible endpoint from the group, if there is one, taking into account the priority and weight of the endpoints.

  1. If group is expired, return null.

    Note: In this case, the user agent MAY remove group from its client, or it may wait and collect garbage en masse at some point in the future as described in §7.2 Garbage Collection.

  2. Let endpoints be a copy of group’s endpoints list.

  3. Remove every endpoint from endpoints that is pending.

  4. If endpoints is empty, return null.

  5. Let priority be the minimum priority value of each endpoint in endpoints.

  6. Remove every endpoint from endpoints whose priority value is not equal to priority.

  7. If endpoints is empty, return null.

  8. Let total weight be the sum of the weight value of each endpoint in endpoints.

  9. Let weight be a random number ≥ 0 and < total weight.

  10. For each endpoint in endpoints:

    1. If weight is less than endpoint’s weight, return endpoint.

    2. Subtract endpoint’s weight from weight.

  11. It should not be possible to fall through to here, since the random number chosen earlier will be less than total weight.

4.4. Send reports

A user agent sends reports by executing the following steps:

  1. Let reports be a copy of the list of queued report objects in reporting cache.

  2. Let endpoint map be an empty map of endpoint objects to lists of report objects.

  3. For each report in reports:

    1. Let origin be the origin of report’s url.

    2. Let client be the entry in the reporting cache for origin.

    3. If there exists an endpoint group (group) in client’s endpoint-groups list whose name is report’s group:

      1. Let endpoint be the result of executing §4.3 Choose an endpoint from a group on group.

      2. If endpoint is a not null:

        1. Append report to endpoint map’s list of reports for endpoint.

        2. Skip to the next report.

    4. For each parent origin that is a superdomain match for origin [RFC6797]:

      1. Let client be the entry in the reporting cache for parent origin.

      2. If there exists an endpoint group (group) in client’s endpoint-groups list whose name is report’s group and whose subdomains flag is "include":

        1. Let endpoint be the result of executing §4.3 Choose an endpoint from a group on group.

        2. If endpoint is an endpoint:

          1. Append report to endpoint map’s list of reports for endpoint.

          2. Skip to the next report.

    5. If we reach this step, the report did not match any endpoint and the user agent MAY remove report from the reporting cache directly. Depending on load, the user agent MAY instead wait for §7.2 Garbage Collection at some point in the future.

  4. For each (endpoint, reports) pair in endpoint map:

    1. Let origin map be an empty map of origins to lists of report objects.

    2. For each report in reports:

      1. Let origin be the origin of report’s url.

      2. Append report to origin map’s list of reports for origin.

    3. For each (origin, per-origin reports) pair in origin map, execute the following steps asynchronously:

      1. Let result be the result of executing §4.5 Attempt to deliver reports to endpoint on endpoint, origin, and per-origin reports.

      2. If result is "Success":

        1. Set endpoint’s failures to 0, and its retry_after to null.

        2. Remove each report in reports from the reporting cache.

        Otherwise, if result is "Remove Endpoint":

        1. Remove endpoint from the reporting cache.

          Note: reports remain in the reporting cache for potential delivery to other endpoints.

        Otherwise (if result is "Failure"):

        1. Increment endpoint’s failures.

        2. Set endpoint’s retry_after to a point in the future which the user agent chooses.

          Note: We don’t specify a particular algorithm here, but user agents are encouraged to employ some sort of exponential backoff algorithm which increases the retry period with the number of failures, with the addition of some random jitter to ensure that temporary failures don’t lead to a crush of reports all being retried on the same schedule.

          Add in a reasonable reference describing a good algorithm. Wikipedia, if nothing else.

Note: User agents MAY decide to attempt delivery for only a subset of the collected reports or endpoints (because, for example, sending all the reports at once would consume an unreasonable amount of bandwidth, etc). As reports are only removed from the cache when they’re successfully delivered, skipped reports will simply be delivered later.

4.5. Attempt to deliver reports to endpoint

Given an endpoint (endpoint), an origin (origin), and a list of reports (reports), this algorithm will construct a request, and attempt to deliver it to endpoint. It returns "Success" if that delivery succeeds, "Remove Endpoint" if the endpoint explicitly removes itself as a reporting endpoint by sending a 410 response, and "Failure" otherwise.

  1. Let collection be a new ECMAScript Array object [ECMA-262].

  2. For each report in reports:

    1. Let data be a new ECMAScript Object with the following properties [ECMA-262]:

      age

      The number of milliseconds between report’s timestamp and the current time.

      type

      report’s type

      url

      report’s url

      user_agent

      report’s user agent

      body

      report’s body

      Note: Client clocks are unreliable and subject to skew. We therefore deliver an age attribute rather than an absolute timestamp. See also §11.2 Clock Skew

    2. Increment report’s attempts.

    3. Append data to collection.

  3. Let request be a new request with the following properties [FETCH]:

    url

    endpoint’s url

    origin

    origin

    header list

    A new header list containing a header named "Content-Type" whose value is "application/reports+json"

    client

    null

    window

    "no-window"

    skip-service-worker flag

    Set.

    initiator

    ""

    destination

    "report"

    destination

    ""

    mode

    "cors"

    unsafe-request flag

    set

    credentials

    "include"

    body

    The string resulting from executing the JSON.stringify() algorithm on collection [ECMA-262]

  4. Queue a task to fetch request.

  5. Wait for a response (response).

  6. If response’s status is an OK status (200-299), return "Success".

  7. If response’s status is 410 Gone [RFC7231], return "Remove Endpoint".

  8. Return "Failure".

5. Reporting Observers

A reporting observer observes some types of reports from JavaScript, and is represented in JavaScript by the ReportingObserver object.

Each ECMAScript global environment has a registered reporting observer list, which is an ordered set of reporting observers.

Any reporting observer that is in a registered reporting observer list is considered registered.

Each ECMAScript global environment has a report buffer, which is a list of reports that have been generated in that ECMAScript global environment. This list is initially empty, and the reports are stored in the same order in which they are generated.

Note: The purpose of the report buffer is to allow reporting observers to observe reports that were generated earlier than that observer could be created (via the buffered option). For example, some reports might be generated during an earlier stage of page loading than when an observer could first be created, or before a JavaScript library is loaded that wishes to observe these reports.

Note: Reporting observers are only relevant for user agents with JavaScript engines.

5.1. Interface ReportingObserver

interface ReportBody {
};

interface Report {
  readonly attribute DOMString type;
  readonly attribute DOMString url;
  readonly attribute ReportBody? body;
};

[Constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options)]
interface ReportingObserver {
  void observe();
  void disconnect();
  ReportList takeRecords();
};

callback ReportingObserverCallback = void (sequence<Report> reports, ReportingObserver observer);

dictionary ReportingObserverOptions {
  sequence<DOMString> types;
  boolean buffered = false;
};

typedef sequence<Report> ReportList;

A Report is the application exposed representation of a report. type returns type, url returns url, and body returns body.

Each ReportingObserver object has these associated concepts:

  • A callback function set on creation.

  • A ReportingObserverOptions dictionary called options.

  • A list of Report objects called the report queue, which is initially empty.

A ReportList represents a sequence of Reports, providing developers with all the convenience methods found on JavaScript arrays.

The ReportingObserver(callback, options) constructor, when invoked, must run these steps:

  1. Create a new ReportingObserver object observer.

  2. Set observer’s callback to callback.

  3. Set observer’s options to options.

  4. Return observer.

The observe() method, when invoked, must run these steps:

  1. Let environment be the ECMAScript global environment associated with the context object.

  2. Append the context object to the registered reporting observer list of its associated ECMAScript global environment.

  3. If the context object’s buffered option is false, return.

  4. Set context object’s buffered option to false.

  5. For each report in the report buffer associated with environment, execute §5.3 Add report to observer with report and the context object.

The disconnect() method, when invoked, must run these steps:

  1. If the context object is not registered, return.

  2. Remove the context object from the registered reporting observer list of its associated ECMAScript global environment.

The takeRecords() method, when invoked, must run these steps:

  1. Let reports be a copy of the context object’s report queue.

  2. Empty the context object’s report queue.

  3. Return reports.

5.2. Notify reporting observers on environment with report

This algorithm makes report’s contents available to any registered reporting observers on the provided ECMAScript global environment environment.

  1. For each ReportingObserver observer registered with environment, execute §5.3 Add report to observer on report and observer.

  2. Append report to the report buffer associated with environment.

  3. If the report buffer now contains more than 100 reports, remove the item at the beginning of the report buffer.

5.3. Add report to observer

Given a report report and a ReportingObserver observer, this algorithm adds report to observer’s report queue, so long as report’s type is observable by observer.

  1. If report’s type is not visible to ReportingObservers, return.

  2. If observer’s options has a non-empty types member which does not contain report’s type, return.

  3. Create a new Report r with type initialized to report’s type, url initialized to report’s url, and body initialized to report’s body.

how to polymorphically initialize body?

  1. Append r to observer’s report queue.

  2. If the size of observer’s report queue is 1, Queue a task to §5.4 Invoke reporting observers with notify list with a copy of the registered reporting observer list of the ECMAScript global environment associated with observer.

5.4. Invoke reporting observers with notify list

This algorithm invokes observer callback functions for reports of previously observed behavior.

  1. For each ReportingObserver observer in notify list:

    1. If observer’s report queue is empty, then continue.

    2. Let reports be a copy of observer’s report queue

    3. Empty observer’s report queue

    4. Invoke observer’s callback with a list of arguments consisting of reports and observer, and observer as the callback this value. If this throws an exception, report the exception.

6. Report Types

6.1. Deprecation

Deprecation reports indicate that a browser API or feature has been used which is expected to stop working in a future update to the browser.

Deprecation reports have the report type "deprecation".

Deprecation reports are visible to ReportingObservers.

interface DeprecationReportBody : ReportBody {
  readonly attribute DOMString id;
  readonly attribute Date? anticipatedRemoval;
  readonly attribute DOMString message;
  readonly attribute DOMString? sourceFile;
  readonly attribute unsigned long? lineNumber;
  readonly attribute unsigned long? columnNumber;
};

A deprecation report’s body, represented in JavaScript by DeprecationReportBody, contains the following fields:

  • id: an implementation-defined string identifying the feature or API that will be removed. This string can be used for grouping and counting related reports.

  • anticipatedRemoval: A JavaScript Date object (rendered as an ISO 8601 string) indicating roughly when the browser version without the specified API will be generally available (excluding "beta" or other pre-release channels). This value should be used to sort or prioritize warnings. If unknown, this field should be null, and the deprecation should be considered low priority (removal may not actually occur).

  • message: A human-readable string with details typically matching what would be displayed on the developer console. The message is not guaranteed to be unique for a given id (eg. it may contain additional context on how the API was used).

  • sourceFile: If known, the file which first used the indicated API, or null otherwise.

  • lineNumber: If known, the line number in sourceFile where the indicated API was first used, or null otherwise.

  • columnNumber: If known, the column number in sourceFile where the indicated API was first used, or null otherwise.

{
  "type": "deprecation",
  "age": 27,
  "url": "https://example.com/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "id": "websql",
    "anticipatedRemoval": "1/1/2020",
    "message": "WebSQL is deprecated and will be removed in Chrome 97 around January 2020",
    "sourceFile": "https://example.com/index.js",
    "lineNumber": 1234,
    "columnNumber": 42
  }
}

6.2. Intervention

Intervention reports indicate that a user agent has decided not to honor a request made by the application (e.g. for security, performance or user annoyance reasons). Note that the report properties are the same as those for deprecation reports, except for the absence of anticipatedRemoval.

Intervention reports have the report type "intervention".

Intervention reports are visible to ReportingObservers.

interface InterventionReportBody : ReportBody {
  readonly attribute DOMString id;
  readonly attribute DOMString message;
  readonly attribute DOMString? sourceFile;
  readonly attribute unsigned long? lineNumber;
  readonly attribute unsigned long? columnNumber;
};

An intervention report’s body, represented in JavaScript by InterventionReportBody, contains the following fields:

  • id: an implementation-defined string identifying the specific intervention that occurred. This string can be used for grouping and counting related reports.

  • message: A human-readable string with details typically matching what would be displayed on the developer console. The message is not guaranteed to be unique for a given id (e.g. it may contain additional context on what led to the intervention).

  • sourceFile: If known, the file which first used the indicated API, or null otherwise.

  • lineNumber: If known, the line number in sourceFile of the offending behavior (which prompted the intervention), or null otherwise.

  • columnNumber: If known, the column number in sourceFile of the offending behavior (which prompted the intervention), or null otherwise.

{
  "type": "intervention",
  "age": 27,
  "url": "https://example.com/",
  "body": {
    "id": "audio-no-gesture",
    "message": "A request to play audio was blocked because it was not triggered by user activation (such as a click).",
    "sourceFile": "https://example.com/index.js",
    "lineNumber": 1234,
    "columnNumber": 42
  }
}

6.3. Crash

Crash reports indicate that the user was unable to continue using the page because the browser (or one of its processes necessary for the page) crashed. For security reasons, no details of the crash are communicated except for a unique identifier (which can be interpreted by the browser vendor), and optionally the reason for the crash (such as "oom").

Crash reports have the report type "crash".

Crash reports are visible to ReportingObservers.

interface CrashReportBody : ReportBody {
  readonly attribute DOMString crashId;
  readonly attribute DOMString? reason;
};

An crash report’s body, represented in JavaScript by CrashReportBody, contains the following fields:

  • crashId: An implementation-defined unique identifier (100 character maximum string). This identifier will not be meaningful to developers directly, but it can potentially be supplied to the browser vendor for more details.

  • reason: A more specific classification of the type of crash that occured. Currently, the only valid reason is "oom" (omitted otherwise), indicating an out-of-memory crash.

{
  "type": "crash",
  "age": 42,
  "url": "https://example.com/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "crashId": "30437694edfeae5b",
    "reason": "oom"
  }
}

7. Implementation Considerations

7.1. Delivery

The user agent SHOULD attempt to deliver reports as soon as possible to provide feedback to developers as quickly as possible. However, when this desire is balanced against the impact on the user, the user wins. With that in mind, the user agent MAY delay delivery of reports based on its knowledge of the user’s activities and context.

For instance, the user agent SHOULD prioritize the transmission of reporting data lower than other network traffic. The user’s explicit activities on a website should preempt reporting traffic.

The user agent MAY choose to withhold report delivery entirely until the user is on a fast, cheap network in order to prevent unnecessary data cost.

The user agent MAY choose to prioritize reports from particular origins over others (perhaps those that the user visits most often?)

7.2. Garbage Collection

Periodically, the user agent SHOULD walk through the cached reports and endpoints, and discard those that are no longer relevant. These include:

  • endpoint groups which are expired

  • endpoint groups which have not been used in some arbitrary period of time (perhaps a ~week?)

  • endpoints whose failures exceed some user-agent-defined threshold (~5 seems reasonable)

  • reports whose attempts exceed some user-agent-defined threshold (~5 seems reasonable)

  • reports which have not been delivered in some arbitrary period of time (perhaps ~2 days?)

For any reports that are discarded, these reports should also be removed from the report buffer of any reporting observer.

8. Sample Reports

POST / HTTP/1.1
Host: example.com
...
Content-Type: application/reports+json

[{
  "type": "csp",
  "age": 10,
  "url": "https://example.com/vulnerable-page/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "blocked": "https://evil.com/evil.js",
    "directive": "script-src",
    "policy": "script-src 'self'; object-src 'none'",
    "status": 200,
    "referrer": "https://evil.com/"
  }
}, {
  "type": "hpkp",
  "age": 32,
  "url": "https://www.example.com/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "date-time": "2014-04-06T13:00:50Z",
    "hostname": "www.example.com",
    "port": 443,
    "effective-expiration-date": "2014-05-01T12:40:50Z"
    "include-subdomains": false,
    "served-certificate-chain": [
      "-----BEGIN CERTIFICATE-----\n
      MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT\n
      ...
      HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto\n
      WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6\n
      yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx\n
      -----END CERTIFICATE-----",
      ...
    ]
  }
}, {
  "type": "nel",
  "age": 29,
  "url": "https://example.com/thing.js",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "referrer": "https://www.example.com/",
    "server-ip": "234.233.232.231",
    "protocol": "",
    "status-code": 0,
    "elapsed-time": 143,
    "age": 0,
    "type": "http.dns.name_not_resolved"
  }
}]

9. Automation

For the purposes of user-agent automation and application testing, this document defines a number of extension commands for the [WebDriver] specification.

9.1. Generate Test Report

The Generate Test Report extension command simulates the generation of a report for the purposes of testing. This report will be observed by any registered reporting observers.

The extension command is defined as follows:

dictionary GenerateTestReportParameters {
  required DOMString message;
  DOMString group;
};
HTTP Method Prefix Name
POST /session/{session id}/reporting generate_test_report

The remote end steps are:

  1. If parameters is not a JSON Object, return a WebDriver error with WebDriver error code invalid argument.

  2. Let message be the result of trying to get parameters’s message property.

  3. If message is null, return a WebDriver error with WebDriver error code invalid argument.

  4. Let group be the result of trying to get parameters’s group property.

  5. If group is null, set group to "default".

  6. Let body be a new object that can be serialized into a JSON text, containing a single string field, body_message.

  7. Set body_message to message.

  8. Let settings be the environment settings object of the current browsing context’s active document.

  9. Execute §4.2 Queue data as type for endpoint group on settings with body, "test", group, and settings.

  10. Return success with data null.

10. Security Considerations

10.1. Capability URLs

Some URLs are valuable in and of themselves. To mitigate the possibility that such URLs will be leaked via this reporting mechanism, we strip out credential information and fragment data from the URL we store as a report’s originator. It is still possible, however, for a feature to unintentionally leak such data via a report’s body. Implementers SHOULD ensure that URLs contained in a report’s body are similarly stripped.

11. Privacy Considerations

11.1. Network Leakage

Because this reporting mechanism is out-of-band, and doesn’t rely on a page being open, it’s entirely possible for a report generated while a user is on one network to be sent while the user is on another network, even if they don’t explicitly open the page from which the report was sent.

Consider mitigations. For example, we could drop reports if we change from one network to another. <https://github.com/w3c/BackgroundSync/issues/107>

11.2. Clock Skew

Each report is delivered along with an age property, rather than the timestamp at which it was generated. We do this because each user’s local clock will be skewed from the clock on the server by an arbitrary amount. The difference between the time the report was generated and the time it was sent will be stable, regardless of clock skew, and we can avoid the fingerprinting risk of exposing the clock skew via this API.

11.3. Cross-origin correlation

If multiple origins all use the same reporting endpoint, that endpoint may learn that a particular user has interacted with a certain set of websites, as it will receive origin-tagged reports from each. This doesn’t seem worse than the status quo ability to track the same information from cooperative origins, and doesn’t grant any new tracking ability above and beyond what’s possible with <img> today.

11.4. Subdomains

This specification allows any resource on a host to declare a set of reporting endpoints for that host and each of its subdomains. This doesn’t have privacy implications in and of itself (beyond those noted in §11.5 Clearing the reporting cache), as the reporting endpoints themselves don’t take any real action, as features will need to opt-into using these reporting endpoints explicitly. Those features certainly will have privacy implications, and should carefully consider whether they should be enabled across origin boundaries.

11.5. Clearing the reporting cache

A user agent’s reporting cache contains data about a user’s activity on the web, and user agents ought to handle this data carefully. In particular, if a user agent gives users the ability to clear their site data, browsing history, browsing cache, or similar, the user agent MUST also clear the reporting cache. Note that this includes both the pending reports themselves, as well as the endpoints to which they would be sent. Both MUST be cleared.

11.6. Disabling Reporting

Reporting is, to some extent, a question of commons. In the aggregate, it seems useful for everyone for reports to be delivered. There is direct benefit to developers, as they can fix bugs, which means there’s indirect benefit to users, as the sites they enjoy will be more stable and enjoyable. As a concrete example, Content Security Policy grants something like herd immunity to cross-site scripting attacks by alerting developers about potential holes in their sites' defenses. Fixing those bugs helps every user, even those whose user agents don’t support Content Security Policy.

The calculus, of course, depends on the nature of data that’s being delivered, and the relative maliciousness of the reporting endpoints, but that’s the value proposition in broad strokes.

That said, it can’t be the case that this general benefit be allowed to take priority over the ability of a user to individually opt-out of such a system. Sending reports costs bandwidth, and potentially could reveal some small amount of additional information above and beyond what a website can obtain in-band ([NETWORK-ERROR-LOGGING], for instance). User agents MUST allow users to disable reporting with some reasonable amount of granularity in order to maintain the priority of constituencies espoused in [HTML-DESIGN-PRINCIPLES].

12. IANA Considerations

12.1. The Report-To Header

The permanent message header field registry should be updated with the following registration: [RFC3864]

Header field name

Report-To

Applicable protocol

http

Status

standard

Author/Change controller

W3C

Specification document

This specification (see §3.1 The Report-To HTTP Response Header Field)

12.2. The application/reports+json Media Type

Type name

application

Subtype name

reports+json

Required parameters

N/A

Optional parameters

N/A

Encoding considerations

Encoding considerations are identical to those specified for the "application/json" media type. See [RFC8259].

Security considerations

See §10 Security Considerations.

Interoperability considerations

This document specifies the format of conforming messages and the interpretation thereof.

Published specification

§4.1 Media Type

Applications that use this media type
Fragment identifier considerations
Additional information

N/A

Person and email address to contact for further information

This document’s editors.

Intended usage:

COMMON

Restrictions on usage:

N/A

Author

This document’s editors.

Change controller

W3C

Provisional registration?

Yes.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[ECMA-262]
ECMAScript Language Specification. URL: https://tc39.github.io/ecma262/
[FETCH]
Anne van Kesteren. Fetch Standard. Living Standard. URL: https://fetch.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[HTTP-JFV]
J. Reschke. A JSON Encoding for HTTP Header Field Values. 24 October 2017. Active Internet-Draft. URL: https://tools.ietf.org/html/draft-reschke-http-jfv
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[RFC2782]
A. Gulbrandsen; P. Vixie; L. Esibov. A DNS RR for specifying the location of services (DNS SRV). February 2000. Proposed Standard. URL: https://tools.ietf.org/html/rfc2782
[RFC3864]
G. Klyne; M. Nottingham; J. Mogul. Registration Procedures for Message Header Fields. September 2004. Best Current Practice. URL: https://tools.ietf.org/html/rfc3864
[RFC5234]
D. Crocker, Ed.; P. Overell. Augmented BNF for Syntax Specifications: ABNF. January 2008. Internet Standard. URL: https://tools.ietf.org/html/rfc5234
[RFC6797]
J. Hodges; C. Jackson; A. Barth. HTTP Strict Transport Security (HSTS). November 2012. Proposed Standard. URL: https://tools.ietf.org/html/rfc6797
[RFC7231]
R. Fielding, Ed.; J. Reschke, Ed.. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. June 2014. Proposed Standard. URL: https://tools.ietf.org/html/rfc7231
[RFC8259]
T. Bray, Ed.. The JavaScript Object Notation (JSON) Data Interchange Format. December 2017. Internet Standard. URL: https://tools.ietf.org/html/rfc8259
[SECURE-CONTEXTS]
Mike West; Yan Zhu. Secure Contexts. URL: https://w3c.github.io/webappsec-secure-contexts/
[URL]
Anne van Kesteren. URL Standard. Living Standard. URL: https://url.spec.whatwg.org/
[WebDriver]
Simon Stewart; David Burns. WebDriver. 5 June 2018. REC. URL: https://www.w3.org/TR/webdriver1/
[WebIDL]
Cameron McCormack; Boris Zbarsky; Tobie Langel. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

Informative References

[CSP3]
Mike West. Content Security Policy Level 3. 13 September 2016. WD. URL: https://www.w3.org/TR/CSP3/
[HTML-DESIGN-PRINCIPLES]
Anne van Kesteren; Maciej Stachowiak. HTML Design Principles. 26 November 2007. WD. URL: https://www.w3.org/TR/html-design-principles/
[NETWORK-ERROR-LOGGING]
Ilya Grigorik; et al. Network Error Logging. 20 July 2016. NOTE. URL: https://www.w3.org/TR/network-error-logging/
[RFC7469]
C. Evans; C. Palmer; R. Sleevi. Public Key Pinning Extension for HTTP. April 2015. Proposed Standard. URL: https://tools.ietf.org/html/rfc7469

IDL Index

interface ReportBody {
};

interface Report {
  readonly attribute DOMString type;
  readonly attribute DOMString url;
  readonly attribute ReportBody? body;
};

[Constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options)]
interface ReportingObserver {
  void observe();
  void disconnect();
  ReportList takeRecords();
};

callback ReportingObserverCallback = void (sequence<Report> reports, ReportingObserver observer);

dictionary ReportingObserverOptions {
  sequence<DOMString> types;
  boolean buffered = false;
};

typedef sequence<Report> ReportList;

interface DeprecationReportBody : ReportBody {
  readonly attribute DOMString id;
  readonly attribute Date? anticipatedRemoval;
  readonly attribute DOMString message;
  readonly attribute DOMString? sourceFile;
  readonly attribute unsigned long? lineNumber;
  readonly attribute unsigned long? columnNumber;
};

interface InterventionReportBody : ReportBody {
  readonly attribute DOMString id;
  readonly attribute DOMString message;
  readonly attribute DOMString? sourceFile;
  readonly attribute unsigned long? lineNumber;
  readonly attribute unsigned long? columnNumber;
};

interface CrashReportBody : ReportBody {
  readonly attribute DOMString crashId;
  readonly attribute DOMString? reason;
};

dictionary GenerateTestReportParameters {
  required DOMString message;
  DOMString group;
};

Issues Index

Fetch monkey patching. Talk to Anne.
Add in a reasonable reference describing a good algorithm. Wikipedia, if nothing else.
how to polymorphically initialize body?
Consider mitigations. For example, we could drop reports if we change from one network to another. <https://github.com/w3c/BackgroundSync/issues/107>