Upgrade Insecure Requests

W3C First Public Working Draft,

This version:
http://www.w3.org/TR/2015/WD-upgrade-insecure-requests-20150226/
Latest version:
http://www.w3.org/TR/upgrade-insecure-requests/
Editor's Draft:
https://w3c.github.io/webappsec/specs/upgrade/
Feedback:
public-webappsec@w3.org with subject line “[upgrade-insecure-requests] … message topic …” (archives)
Issue Tracking:
Inline In Spec
Editor:
(Google Inc.)

Abstract

This document defines a mechanism which allows authors to instruct a user agent to upgrade a priori insecure resource requests to secure transport before Fetching them.

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document was published by the Web Application Security Working Group as a Working Draft. This document is intended to become a W3C Recommendation.

The (archived) public mailing list public-webappsec@w3.org (see instructions) is preferred for discussion of this specification. When sending e-mail, please put the text “upgrade-insecure-requests” in the subject, preferably like this: “[upgrade-insecure-requests] …summary of comment…

This document is a First Public Working Draft.

Publication as a First Public Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by the Web Application Security Working Group.

This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 August 2014 W3C Process Document.

Table of Contents

1. Introduction

This section is not normative.

Increasingly, we encourage authors to transition their sites and applications away from insecure transport, and onto encrypted and authenticated connections [WEB-HTTPS]. While this migration has significant advantages for both authors and users, it isn’t without negative side-effects.

Most notably, mixed content checking [MIX] has the potential to cause real headache for administrators tasked with moving substantial amounts of legacy content onto HTTPS. In particular, going through old content and rewriting subresource URLs manually is a huge undertaking. Moreover, it’s often the case that truly legacy content is difficult or impossible to update. Consider the BBC’s archived websites [BBC-ARCHIVE], or the New York Times' hard-coded URLs [NYT-HTTPS].

We should remove this burden from site authors by allowing them to assert to a user agent that they intend a site to load only secure resources, and that insecure URLs ought to be treated as though they had been replaced with equivalent secure URLs.

This document defines a new Content Security Policy directive, upgrade-insecure-requests, through which authors can make this assertion. Note: Delivering the policy as a header allows an administrator to easily opt a set of pages into the upgrade mechanism without touching their source code individually. The legacy content examples above would not be feasible with an approach that inlined the policy into HTML, for example.

1.1. Goals

The overarching goal is to reduce the burden of migrating websites from a priori insecure origins by reducing the negative side effects of mixed content blocking [MIX].

If we assume that authors do the server-side legwork (obtaining a certificate, configuring the server, setting up redirects), and that authors also ensure that content is accessible at the same host and path on a secure scheme, then the following statements ought to hold after implementing this feature:

  1. Authors should be able to ensure that all content requested by a given page loads successfully, and securely. Mixed content blocking should not break pages as a result of migrating to a secure origin.

    Note: This requirement is not met by Mixed Content’s strict mode, which makes something like the opposite assertion.

  2. As a result of #1, the user agent should not degrade any security indicators related to requesting mixed content, as no insecure content should be requested.
  3. Authors should be able to ensure that all internal links correctly send users to the site’s secure address, and not to its pre-migration insecure address.
  4. Authors should be able to achieve all these goals without editing a site’s content. This is particularly important for archived content and legacy systems for which maintenance is difficult enough, never mind upgrades.
  5. Authors should be able to pursue a gradual transition from insecure to secure, serving secure resources to clients that support upgrades, while retaining insecure resources for clients that don’t.

Note: The mechanism defined here does not intend to supplant Strict Transport Security [RFC6797]. See §7.2 Relation to HSTS for details.

1.2. Examples

Megacorp, Inc. wishes to migrate http://example.com/ to https://example.com. They set up the server-side requirements to make their own resources available over HTTPS, and work with partners in order to make third-party widgets available as well.

They quickly realize, however, that the majority of their long-tail content is locked up in a database tied to an old content management system that they’ve been avoiding for years ("If it ain’t broke..."). They’ll need to spend quite a bit of time either migrating the content to their new system, or modifying it in-place in the old system. Either way, it ends up being a substantial amount of work, and though they put top programmers on the task, it clearly won’t be done any time soon.

As a stopgap measure, Megacorp injects the following header into every response that goes out through their servers:

Content-Security-Policy: upgrade-insecure-requests

This automatically upgrades all insecure resource requests for their pages to secure variants, allowing a user agent to treat the following HTML code:

<img src="http://example.com/image.png">

as though it had been delivered as:

<img src="https://example.com/image.png">

The URL will be rewritten before the request is made, meaning that no insecure requests will hit the network. Users will be safer, and Megacorp’s administrators will be happier.

2. Key Concepts and Terminology

upgrade

A Request is said to be upgraded if it is rewritten to contain a URL with a scheme of https or wss.

The Augmented Backus-Naur Form (ABNF) notation used in §3.1 Upgrade Policy Delivery is specified in RFC5234. [ABNF]

3. Upgrading Insecure Resource Requests

In order to allow authors to mitigate the negative side-effects of migration away from a priori insecure origins, authors may instruct the user agent to transparently upgrade resource requests to potentially secure variants of the original request’s URL.

To support this instruction, environment settings objects and browsing contexts have an upgrade insecure resource requests flag which is set to No Upgrade unless otherwise specified. This flag is checked in §4.1 Upgrade request to a potentially secure URL, if appropriate in order to determine whether or not resource requests should be upgraded during Fetching.

3.1. Upgrade Policy Delivery

A server MAY instruct a user agent to upgrade insecure resource requests for a particular protected resource by sending a Content-Security-Policy header [CSP] that contains a upgrade-insecure-requests directive, defined via the following ABNF grammar:

directive-name  = "upgrade-insecure-requests"
directive-value = ""

When enforcing the upgrade-insecure-requests directive, set the protected resource’s incumbent settings object’s upgrade insecure resource requests flag to Enforced Upgrade.

When monitoring the upgrade-insecure-requests directive, set the protected resource’s incumbent settings object’s upgrade insecure resource requests flag to Monitored Upgrade.

In a thread on public-webappsec@, Peter Eckersley suggested modifying this to allow whitelisting specific hosts, rather than upgrading everything: "That way, if you have N third parties on a site, and (say) two of them provide images only, and don’t support HTTPS at all, you can use the upgrade mechanism for scripts on the other N - 2 origins."

For example, sending the following header with a document will ensure that subresource requests are upgraded, and that reports are sent out for each upgrade:
Content-Security-Policy: upgrade-insecure-requests;
                         report-uri /upgrade-endpoint

Sending the following header will not upgrade requests, but will send reports:

Content-Security-Policy-Report-Only: upgrade-insecure-requests;
                                     report-uri /upgrade-endpoint

3.2. Feature Detecting Clients Capable of Upgrading

If a site requires the upgrade mechanism described in this document in order to provide users with a reasonable experience over secure transit, then authors need to determine whether or not it is safe to redirect a client to the secure version of a site. Rather than relying on user-agent sniffing to make this decision, user agents MUST advertise their capabilities when making insecure navigational requests by sending a Prefer HTTP request header expressing the desire for a secure representation [RFC7240] as described in §3.2.1 The return=secure-representation Preference .

3.2.1. The return=secure-representation Preference

The "return=secure-representation" preference indicates that the client prefers that the server redirect from an insecure resource representation to a secure one, and that it can successfully handle the upgrade-insecure-requests directive in order to make that redirection as seamless as possible.

This new preference changes the return ABNF defined in RFC7240 Section 4.2 to the following:

return = "return" BWS "=" BWS ( "secure-representation" / "representation" / "minimal" )

When a server encounters this preference in an HTTP request’s headers, it SHOULD redirect the user to a secure version of the resource being requested.

User agent implementation details are described in step #2 of the the §4.1 Upgrade request to a potentially secure URL, if appropriate algorithm. Note in particular that to mitigate the risk that this header will become a vestigial part of the platform, user agents SHOULD omit the preference when making requests to potentially secure origins.

A client that supports this document’s upgrade mechanism requests http://example.com/ as follows:
GET / HTTP/1.1
Host: example.com
Connection: keep-alive
...
Prefer: return=secure-representation

The server parses the preference, notices that the user’s client can deal well with upgrade requests, and therefore responds to the request by redirecting the user to a secure version of the resource she’s requesting:

HTTP/1.1 302 Moved Temporarily
Location: https://example.com/

3.3. Policy Inheritance

If a Document's incumbent settings object’s upgrade insecure resource requests flag is set to true, the user agent MUST ensure that all nested browsing contexts inherit the setting in the following ways:

  1. When a nested browsing context context is created, set its upgrade insecure resource requests flag to true if context’s embedding document’s upgrade insecure resource requests flag is set to true.
  2. When creating a new Document object, set its incumbent settings object’s upgrade insecure resource requests flag to true if its browsing context’s upgrade insecure resource requests flag is true.

4. Processing Algorithms

4.1. Upgrade request to a potentially secure URL, if appropriate

Given a Request request, this algorithm will rewrite its url if the client from which the request originates has opted-in to upgrades. It will also inject a return=secure-representation preference for insecure navigational requests in order to improve a server’s ability to feature-detect a client’s upgrade capabilities.

We will not upgrade cross-origin navigational requests, with the exception of form submissions. Form submissions will be upgraded to mitigate the risk of data leakage via plaintext submissions.

This should be called from Fetch, probably after the existing step #3.

  1. If request’s url is potentially secure: return without modifying request.
  2. If request’s context-frame-type is top-level, nested, or auxiliary, then append a header named Prefer with a value of return=secure-representation to request’s header-list.
  3. If request’s context-frame-type is top-level or auxiliary, then:
    1. If request’s context is form, skip the remaining substeps, and continue upgrading request.
    2. If request’s url's host is the same as request’s client's origin’s host, skip the remaining substeps, and continue upgrading request.

      Note: We allow only same-origin upgrades in order to ensure that navigations between pages of a single site that has opted-into the upgrade behavior remain on HTTPS, regardless of the hard-coded values in <a> tags. Performing upgrades for navigations to third-party resources brings a significantly higher potential for breakage, so we’re avoiding it for the moment.

      This isn’t correct for nested documents. We need to pass the policy-setting document’s host down to ensure that we’re only attempting "first-party" upgrades.

    3. Return without further modifying request.
  4. Let upgrade state be the result of executing §4.2 Should insecure Requests be upgraded for client? upon request’s client.
  5. If upgrade state is No Upgrade, return without modifying request.
  6. Execute §4.3 Report an upgrade for request on request.
  7. If upgrade state is Monitored Upgrade, return without modifying request.
  8. If request’s url's scheme is http, set request’s url's scheme to https, and return.
  9. If request’s urls port is 80, set request’s urls port to 443.

    Note: This will only change the URL’s port if the port is explicitly set to 80. If the port is not set, or if it is set to some non-standard value, the user agent will not modify that value. This implementation makes the same tradeoffs as HSTS (see [RFC6797], and specifically step #5 of Section 8.3, and item #6 in Appendix A).

Note: Due to Fetch’s recursive nature, this algorithm will upgrade insecurely-redirected requests as well as insecure initial requests.

4.2. Should insecure Requests be upgraded for client?

Given an Request's client client (an environment settings object), this algorithm returns Enforced Upgrade if a priori insecure requests associated with that client should be upgraded, Monitored Upgrade if requests should be reported (via §4.3 Report an upgrade for request ) but not actually upgraded, and No Upgrade otherwise. In short, this will check the client and return the appropriate upgrade insecure resource requests flag set on it or its browsing context.

  1. If client has a responsible document, return the value of its upgrade insecure resource requests flag.

    Note: This catches Documents or Workers whose flag is set directly by the upgrade-insecure-requests directive, or which have inherited the flag from an embedding document.

  2. If client has a responsible browsing context, return the value of its upgrade insecure resource requests flag.

    Note: This catches requests triggered from detached clients. Not sure this is necessary, really, given the inheritance structure defined in §3.3 Policy Inheritance.

  3. Return No Upgrade.

4.3. Report an upgrade for request

When a Request is upgraded, a CSP violation is triggered with the intent of informing authors that resources on their sites will fail to load in user agents that do not support this upgrade mechanism.

Note: This violation report will be triggered for the Document or Worker that triggers the request. This might or might not be the same protected resource that set the upgrade-insecure-requests directive, due to §3.3 Policy Inheritance. See §6.2 CSP Violation Reports for detail.

  1. Report a violation for request’s client's responsible document, using upgrade-insecure-requests as the violated-directive and effective-directive, and request’s url as the blocked-uri.

5. Modifications to WebSockets

WebSockets do not use the Fetching algorithm, so we need to handle those requests separately.

The establish a WebSocket connection algorithm [RFC6455] is modified as follows:

6. Security Considerations

6.1. Interaction with HSTS

The upgrade-insecure-requests directive does not replace the Strict-Transport-Security HTTP response header [RFC6797]. Authors who serve their site over secure transport SHOULD send that header with an appropriate max-age in order to ensure that users are not subject to SSL stripping attacks by maliciously active network attackers.

6.2. CSP Violation Reports

When sending a violation report for an upgraded resource, user agents MUST target the Document or Worker that triggered the request, rather than the Document or Worker on which the upgrade-insecure-requests directive was set. Due to §3.3 Policy Inheritance, the latter might be a cross-origin ancestor of the former, and sending violation reports to that set of reporting endpoints could leak data in unexpected ways.

Likewise, the SecurityPolicyViolationEvent MUST NOT target any Document other than the one which triggered the request, for the same reasons.

7. Authoring Considerations

7.1. Legacy Clients

Legacy clients which do support mixed content blocking [MIX], but do not support the upgrade-insecure-requests directive will continue to have a suboptimal experience on pages containing a priori insecure URLs. Authors SHOULD ensure that they collect violation reports in order to determine which resources are most problematic for their users, and SHOULD use that information to prioritize fixes for URLs in legacy content that users will most likely request.

7.2. Relation to HSTS

The mechanism specified here deals only with the security policy for a specific protected resource. It does not deprecate, replace, or in any way reduce the value of the Strict-Transport-Security HTTP response header [RFC6797]. Authors can and should continue to use that header to ensure that their users are not subject to SSL stripping downgrade attacks, as the upgrade-insecure-requests directive will not ensure that users visiting your site via links on third-party sites will be upgraded to HTTPS for the top-level navigation.

Likewise, the Strict-Transport-Security header does not imply the behavior that upgrade-insecure-requests activates. It only ensures that resources requested from an origin will never hit the network insecurely.

Therefore, we recommend that authors who wish to ensure that their users have a secure experience do the following:

  1. Redirect incoming traffic from HTTP to HTTPS by serving a Location header along with a 302 status code.
  2. Respond to HTTPS requests with a Strict-Transport-Security header with a reasonable max-age.
  3. Work with user agent vendors to add their sites to those user agent’s HSTS Preload Lists (for example, by visiting hstspreload.appspot.com).
  4. Opt-into either this document’s upgrade-insecure-requests mechanism, or into Mixed Content’s strict mode in order to ensure that insecure content is never loaded.

8. IANA Considerations

The "HTTP Preferences" registry should be updated with the following alteration to the existing registration for the return preference [RFC7240]:

Preference

return

Value

One of "minimal", "representation", or "secure-representation".

Description

When the value is "minimal", it indicates that the client prefers that the server return a minimal response to a request. When the value is "representation", it indicates that the client prefers that the server include a representation of the current state of the resource in response to a request. When the value is "secure-representation" it indicates that the client prefers to be redirected to a secure version of the resource (and that the client supports the upgrade-insecure-requests mechanism).

Reference

Section 4.2 of RFC7240 defines the "minimal" and "representation" values. This specification (see §3.2.1 The return=secure-representation Preference ) defines the secure-representation value.

9. Acknowledgements

Anne van Kesteren helped ensure that the initial draft of this document was sane. Peter Eckersley and Daniel Kahn Gillmor clarified the problem space, and helped point out the impact.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words "for example" or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word "Note" and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Conformance Classes

A conformant user agent must implement all the requirements listed in this specification that are applicable to user agents.

A conformant server must implement all the requirements listed in this specification that are applicable to servers.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[ABNF]
Dave Crocker; Paul Overell. Augmented BNF for Syntax Specifications: ABNF. RFC. URL: http://www.ietf.org/rfc/rfc5234.txt
[CSP]
Mike West; Dan Veditz. Content Security Policy. WD. URL: https://w3c.github.io/webappsec/specs/content-security-policy/
[FETCH]
Anne van Kesteren. Fetch. Living Standard. URL: https://fetch.spec.whatwg.org/
[MIX]
Mike West. Mixed Content. ED. URL: https://w3c.github.io/webappsec/specs/mixedcontent/
[RFC6455]
Ian Fette; Alexey Melnikov. The WebSocket Protocol. RFC. URL: http://www.ietf.org/rfc/rfc6455.txt
[URL]
Anne van Kesteren. URL. Living Standard. URL: https://url.spec.whatwg.org/
[dom]
Anne van Kesteren; et al. W3C DOM4. 10 July 2014. LCWD. URL: http://www.w3.org/TR/dom/
[html5]
Robin Berjon; et al. HTML5. 28 October 2014. REC. URL: http://www.w3.org/TR/html5/
[rfc2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: http://www.ietf.org/rfc/rfc2119.txt
[rfc7240]
J. Snell. Prefer Header for HTTP. June 2014. Proposed Standard. URL: http://www.ietf.org/rfc/rfc7240.txt
[workers]
Ian Hickson. Web Workers. 1 May 2012. CR. URL: http://www.w3.org/TR/workers/

Informative References

[BBC-ARCHIVE]
Neil McIntosh. Labelling BBC Online's archived websites. URL: http://www.bbc.co.uk/blogs/internet/entries/f7126d19-2afa-3231-9c4e-0f7198c468ab
[NYT-HTTPS]
Eitan Konigsburg; Rajiv Pant; Elena Kvochko. Embracing HTTPS. URL: http://open.blogs.nytimes.com/2014/11/13/embracing-https/
[RFC6797]
Jeff Hodges; Collin Jackson; Adam Barth. HTTP Strict Transport Security (HSTS). RFC. URL: http://www.ietf.org/rfc/rfc6797.txt
[WEB-HTTPS]
Mark Nottingham. Securing the Web. TAG Finding. URL: http://www.w3.org/2001/tag/doc/web-https

Issues Index

In a thread on public-webappsec@, Peter Eckersley suggested modifying this to allow whitelisting specific hosts, rather than upgrading everything: "That way, if you have N third parties on a site, and (say) two of them provide images only, and don’t support HTTPS at all, you can use the upgrade mechanism for scripts on the other N - 2 origins."
This should be called from Fetch, probably after the existing step #3.
This isn’t correct for nested documents. We need to pass the policy-setting document’s host down to ensure that we’re only attempting "first-party" upgrades.