Fetch Metadata Request Headers

Editor’s Draft,

This version:
https://github.com/w3c/webappsec-fetch-metadata/
Latest published version:
https://www.w3.org/TR/fetch-metadata/
Previous Versions:
Version History:
https://github.com/w3c/webappsec-fetch-metadata/commits/master/index.bs
Feedback:
public-webappsec@w3.org with subject line “[fetch-metadata] … message topic …” (archives)
Issue Tracking:
GitHub
Inline In Spec
Editor:
(Google Inc.)
Explainer:
https://github.com/w3c/webappsec-fetch-metadata
Participate:
File an issue (open issues)
Tests:
web-platform-tests fetch/sec-metadata/

Abstract

This document defines a set of Fetch metadata request headers that aim to provide servers with enough information to make a priori decisions about whether or not to service a request based on the way it was made, and the context in which it will be used.

Status of this document

This is a public copy of the editors’ draft. It is provided for discussion only and may change at any moment. Its publication here does not imply endorsement of its contents by W3C. Don’t cite this document other than as work in progress.

Changes to this document may be tracked at https://github.com/w3c/webappsec.

The (archived) public mailing list public-webappsec@w3.org (see instructions) is preferred for discussion of this specification. When sending e-mail, please put the text “fetch-metadata” in the subject, preferably like this: “[fetch-metadata] …summary of comment…

This document was produced by the Web Application Security Working Group.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 March 2019 W3C Process Document.

1. Introduction

Interesting web applications generally end up with a large number of web-exposed endpoints that might reveal sensitive data about a user, or take action on a user’s behalf. Since users' browsers can be easily convinced to make requests to those endpoints, and to include the users' ambient credentials (cookies, privileged position on an intranet, etc), applications need to be very careful about the way those endpoints work in order to avoid abuse.

Being careful turns out to be hard in some cases ("simple" CSRF), and practically impossible in others (cross-site search, timing attacks, etc). The latter category includes timing attacks based on the server-side processing necessary to generate certain responses, and length measurements (both via web-facing timing attacks and passive network attackers).

It would be helpful if servers could make more intelligent decisions about whether or not to respond to a given request based on the way that it’s made in order to mitigate the latter category. For example, it seems pretty unlikely that a "Transfer all my money" endpoint on a bank’s server would expect to be referenced from an img tag, and likewise unlikely that evil.com is going to be making any legitimate requests whatsoever. Ideally, the server could reject these requests a priori rather than delivering them to the application backend.

Here, we describe a mechanims by which user agents can enable this kind of decision-making by adding additional context to outgoing requests. By delivering metadata to a server in a set of fetch metadata headers, we enable applications to quickly reject requests based on testing a set of preconditions. That work can even be lifted up above the application layer (to reverse proxies, CDNs, etc) if desired.

1.1. Examples

A request generated by a picture element would result in a request containing the following HTTP request headers:

Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: cross-site

A top-level navigation from https://example.com to https://example.com/ caused by a user’s click on an in-page link would result in a request containing the following HTTP request header:

Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: same-origin
Sec-Fetch-User: ?1

2. Fetch Metadata Headers

The following sections define several fetch metadata headers, each of which exposes an interesting request attribute to a server.

2.1. The Sec-Fetch-Dest HTTP Request Header

The Sec-Fetch-Dest HTTP request header exposes a request's destination to a server. It is a Structured Header whose value MUST be a token. [I-D.ietf-httpbis-header-structure] Its ABNF is:

Sec-Fetch-Dest = sh-token

Valid Sec-Fetch-Dest values include "audio", "audioworklet", "document", "embed", "empty", "font", "image", "manifest", "object", "paintworklet", "report", "script", "serviceworker", "sharedworker", "style", "track", "video", "worker", "xslt", and "nested-document".

In order to support forward-compatibility with as-yet-unknown request types, servers SHOULD ignore this header if it contains an invalid value.

// fetch()'s destination is the empty string:
Sec-Fetch-Dest: empty

// <img>'s destination is "image"
Sec-Fetch-Dest: image

// new Worker()'s destination is "worker"
Sec-Fetch-Dest: worker

// Top-level navigations' destinations are "document"
Sec-Fetch-Dest: document

// <iframe> navigations' destinations are "document"
Sec-Fetch-Dest: document
To set the Sec-Fetch-Dest header for a request r:
  1. Assert: r’s url is a potentially trustworthy URL.

  2. Let header be a Structured Header whose value is a token.

  3. If r’s destination is the empty string, set header’s value to the string "empty". Otherwise, set header’s value to r’s destination.

    Note: We map Fetch’s empty string destination onto an explicit "empty" token in order to simplify processing.

  4. Set a structured header `Sec-Fetch-Dest`/header in r’s header list.

2.2. The Sec-Fetch-Mode HTTP Request Header

The Sec-Fetch-Mode HTTP request header exposes a request's mode to a server. It is a Structured Header whose value is a token. [I-D.ietf-httpbis-header-structure] Its ABNF is:

Sec-Fetch-Mode = sh-token

Valid Sec-Fetch-Mode values include "cors", "navigate", "nested-navigate", "no-cors", "same-origin", and "websocket". In order to support forward-compatibility with as-yet-unknown request types, servers SHOULD ignore this header if it contains an invalid value.

To set the Sec-Fetch-Mode header for a request r:
  1. Assert: r’s url is a potentially trustworthy URL.

  2. Let header be a Structured Header whose value is a token.

  3. Set header’s value to r’s mode.

  4. If header’s value is "navigate", and r’s reserved client is either null or an environment whose target browsing context is a nested browsing context, set header’s to "nested-navigate".

    NOTE: We’re doing this work because Fetch does not currently define nested-navigate. See § 3 Integration with Fetch and HTML.

  5. Set a structured header `Sec-Fetch-Mode`/header in r’s header list.

2.3. The Sec-Fetch-Site HTTP Request Header

The Sec-Fetch-Site HTTP request header exposes the relationship between a request initiator’s origin and its target’s origin. It is a Structured Header whose value is a token. [I-D.ietf-httpbis-header-structure] Its ABNF is:

Sec-Fetch-Site = sh-token

Valid Sec-Fetch-Site values include "cross-site", "same-origin", "same-site", and "none". In order to support forward-compatibility with as-yet-unknown request types, servers SHOULD ignore this header if it contains an invalid value.

To set the Sec-Fetch-Site header for a request r:
  1. Assert: r’s url is a potentially trustworthy URL.

  2. Let header be a Structured Header whose value is a token.

  3. Set header’s value to same-origin.

  4. If r is a navigation request that was explicitly caused by a user’s interaction with the user agent (by typing an address into the user agent directly, for example, or by clicking a bookmark, etc.), then set header’s value to none.

    Note: See § 4.3 Directly User-Initiated Requests for more detail on this somewhat poorly-defined step.

  5. If header’s value is not none, then for each url in r’s url list:

    1. If url is same origin with r’s origin, continue.

    2. Set header’s value to cross-site and break if any of the following are true:

    3. Set header’s value to same-site.

  6. Set a structured header `Sec-Fetch-Site`/header in r’s header list.

2.4. The Sec-Fetch-User HTTP Request Header

The Sec-Fetch-User HTTP request header exposes whether or not a navigation request was triggered by user activation. It is a Structured Header whose value is a boolean. [I-D.ietf-httpbis-header-structure] Its ABNF is:

Sec-Fetch-User = sh-boolean

Note: The header is delivered only for navigation requests, and only when its value is true. It might be reasonable to expand the headers' scope in the future to include subresource requests generally if we can spell out some use cases that would be improved by exposing that information (and if we can agree on ways to define that status for all the subresource request types we’d be interested in), but for the moment, navigation requests have clear use cases, and seem straightforward to define interoperably.

To set the Sec-Fetch-User header for a request r:
  1. Assert: r’s url is a potentially trustworthy URL.

  2. If r is not a navigation request, or if r’s user activation flag is false, return.

  3. Let header be a Structured Header whose value is a token.

  4. Set header’s value to the value of r’s user activation flag.

    This flag is defined here, in § 3 Integration with Fetch and HTML. Ideally, we can move it to Fetch rather than monkey-patching. <https://github.com/whatwg/fetch/issues/885>

  5. Set a structured header `Sec-Fetch-User`/header in r’s header list.

3. Integration with Fetch and HTML

To support Sec-Fetch-User, request needs to be taught about requests which were triggered by user activation:

Monkeypatching [FETCH]:

A request has a boolean user activation flag. Unless stated otherwise, it is false.

Note: This is only used for navigation requests, and reflects whether a given navigation was triggered by user activation.

This flag could be populated from HTML’s process a navigate fetch algorithm, perhaps by inserting the following step after the current algorithm’s step 2:

Monkeypatching [HTML]:

  1. If this algorithm was triggered by user activation, set request’s user activation flag to true.

We’ll also want to resolve whatwg/fetch#755 to add a "nested-navigate" mode to support Sec-Fetch-Mode.

To set the Fetch metadata headers for a request, given request r:
  1. If r’s url is not an potentially trustworthy URL, return.

  2. Set the Sec-Fetch-Dest header for r.

  3. Set the Sec-Fetch-Mode header for r.

  4. Set the Sec-Fetch-Site header for r.

  5. Set the Sec-Fetch-User header for r.

Fetch will call into the algorithm above from within its main fetch algorithm. Please consult that specification for integration details [FETCH].

Monkey patching! <https://github.com/whatwg/fetch/issues/885>

4. Security and Privacy Considerations

4.1. Redirects

The user agent will send a Sec-Fetch-Site header along with each request in a redirect chain. The header’s value will shift in the presence of cross-origin or cross-site redirection in order to mitigate confusion.

The algorithm to set the Sec-Fetch-Site header walks the request's entire url list, and will send cross-site if any URL in the list is cross-site to the request’s current url, same-site only if all URLs in the list are same-site with the request’s current url, and same-origin only if all URLs in the list are same-origin with the request’s current url.

For example, if https://example.com/ requests https://example.com/redirect, the initial request’s Sec-Fetch-Site value would be same-origin. If that response redirected to https://subdomain.example.com/redirect, that request’s Sec-Fetch-Site value would be same-site (as https://subdomain.example.com/ and https://example.com/ have the same registrable domain). If that response redirected to https://example.net/redirect, that request’s Sec-Fetch-Site value would be cross-site (as https://example.net/ is not same-site with https://example.com/ and https://subdomain.example.com/). If that response redirects all the way back to https://example.com/, the final request’s Sec-Fetch-Site value would still be cross-site (as the redirect chain includes https://example.net/, which is still not same-site with the other URLs.

4.2. The Sec- Prefix

Each of the headers defined in this document is prefixed with Sec-, which makes them all forbidden header names, and therefore unmodifiable from JavaScript. This will prevent malicious websites from convincing user agents to send forged metadata along with requests, which should give sites a bit more confidence in their ability to respond reasonably to the advertised information.

4.3. Directly User-Initiated Requests

When setting the Sec-Fetch-Site header, user agents are asked to distinguish between navigation requests that are "explicitly caused by a user’s interaction". This somewhat poorly defined phrase is pulled from HTML, which suggests that "A user agent may provide various ways for the user to explicitly cause a browsing context to navigate, in addition to those defined in this specification."

The goal is to distinguish between "webby" navigations that are controlled by a given (potentially malicious!) website (e.g. links, the window.location setter, form submissions, etc.), and those that are not (e.g. user interaction with a user agent’s address bar, bookmarks, etc). The former will be delivered with a Sec-Fetch-Site header whose value is same-origin, same-site, or cross-site, as appropriate. The latter will be distinguished with a value of none, as no specific site is actually responsible for the request, and it makes sense to allow servers to treat them as trusted, as they somehow represent a user’s direct intent.

Each user agent is likely to have a distinct set of interactions which might fall into one or the other category, and it will be hard to share an automated test suite for these cases. Still, it would be ideal to align on behavior for those which are likely to be common. Sme examples follow:

5. Deployment Considerations

5.1. Vary

If a given endpoint’s response depends upon the values the client delivers in a Fetch metadata header, developers should be careful to include an appropriate Vary header [RFC7231], in order to ensure that caches handle the response appropriately. For example, Vary: Accept-Encoding, Sec-Fetch-Site.

5.2. Header Bloat

An earlier version of this document defined a single Sec-Metadata header, whose contents were a dictionary. Subsequent discussion (as well as Mark Nottingham’s excellent [mnot-designing-headers]) shifted the design away from a single dictionary to a series of simple headers, each of which contains only a single token. This design should perform significantly better under HTTP’s current HPACK compression algorithms.

Further discussion on the topic can be found on the review thread in w3ctag/design-reviews#280.

6. IANA Considerations

The permanent message header field registry should be updated with the following registrations for Fetch metadata headers: [RFC3864]

6.1. Sec-Fetch-Dest Registration

Header field name

Sec-Fetch-Dest

Applicable protocol

http

Status

standard

Author/Change controller

Me

Specification document

This specification (See § 2.1 The Sec-Fetch-Dest HTTP Request Header)

6.2. Sec-Fetch-Mode Registration

Header field name

Sec-Fetch-Mode

Applicable protocol

http

Status

standard

Author/Change controller

Me

Specification document

This specification (See § 2.2 The Sec-Fetch-Mode HTTP Request Header)

6.3. Sec-Fetch-Site Registration

Header field name

Sec-Fetch-Site

Applicable protocol

http

Status

standard

Author/Change controller

Me

Specification document

This specification (See § 2.3 The Sec-Fetch-Site HTTP Request Header)

6.4. Sec-Fetch-User Registration

Header field name

Sec-Fetch-User

Applicable protocol

http

Status

standard

Author/Change controller

Me

Specification document

This specification (See § 2.4 The Sec-Fetch-User HTTP Request Header)

7. Acknowledgements

Thanks to Anne van Kesteren, Artur Janc, Dan Veditz, Łukasz Anforowicz, Mark Nottingham, and Roberto Clapis, who all provided substantial support in the design of this mechanism.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[FETCH]
Anne van Kesteren. Fetch Standard. Living Standard. URL: https://fetch.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[I-D.ietf-httpbis-header-structure]
Mark Nottingham; Poul-Henning Kamp. Structured Headers for HTTP. ID. URL: https://tools.ietf.org/html/draft-ietf-httpbis-header-structure
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[RFC3864]
G. Klyne; M. Nottingham; J. Mogul. Registration Procedures for Message Header Fields. September 2004. Best Current Practice. URL: https://tools.ietf.org/html/rfc3864
[SECURE-CONTEXTS]
Mike West. Secure Contexts. 15 September 2016. CR. URL: https://www.w3.org/TR/secure-contexts/
[URL]
Anne van Kesteren. URL Standard. Living Standard. URL: https://url.spec.whatwg.org/

Informative References

[MNOT-DESIGNING-HEADERS]
Mark Nottingham. Designing Headers for HTTP Compression. URL: https://www.mnot.net/blog/2018/11/27/header_compression
[RFC7231]
R. Fielding, Ed.; J. Reschke, Ed.. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. June 2014. Proposed Standard. URL: https://httpwg.org/specs/rfc7231.html

Issues Index

This flag is defined here, in § 3 Integration with Fetch and HTML. Ideally, we can move it to Fetch rather than monkey-patching. <https://github.com/whatwg/fetch/issues/885>
Monkey patching! <https://github.com/whatwg/fetch/issues/885>