This specification defines an API extending the HTMLMediaElement
that enables controlling remote playback of media from a web page.
Status of This Document
This section describes the status of this
document at the time of its publication. A list of current W3C
publications and the latest revision of this technical report can be found
in the W3C technical reports index at
https://www.w3.org/TR/.
This document builds on the group's experience on presenting web
content on external presentation-type displays, and re-uses patterns
and design considerations from the Presentation API specification
whenever appropriate [PRESENTATION-API].
Although this document is still a work in progress and
is subject to change, the Working Group believes that the API surface is
stable. Most of the remaining issues listed on the issue tracker are
considered minor at this stage except for
Issue #41.
Issue #41
discusses the set of media playback features that remote playback
devices are expected to support. The group will seek further developer
feedback and implementation experience to identify any interoperability
issues around these features when used during remote playback, and will
further clarify the specification based on feedback received.
For other issues or concerns, it is possible to file a bug or
send an email to the mailing
list. For small editorial changes like typos, sending a pull request
is appreciated.
The Working Group invites everyone to review this document, and will
work with relevant groups at W3C to conduct horizontal reviews on
accessibility, internationalization, privacy, security and technical
architecture principles.
No feature has been identified as being at risk.
The Second Screen Working Group will develop a test suite for the
Remote Playback API during the Candidate Recommendation period and
prepare an implementation report. For this specification to advance to
Proposed Recommendation, two independent, interoperable implementations
of each feature must be demonstrated, as detailed in the Candidate Recommendation exit
criteria section.
Publication as an Editor's Draft does not
imply endorsement by W3C and its Members.
This is a draft document and may be updated, replaced or obsoleted by other
documents at any time. It is inappropriate to cite this document as other
than work in progress.
This document was produced by a group
operating under the
W3C Patent
Policy.
W3C maintains a
public list of any patent disclosures
made in connection with the deliverables of
the group; that page also includes
instructions for disclosing a patent. An individual who has actual
knowledge of a patent which the individual believes contains
Essential Claim(s)
must disclose the information in accordance with
section 6 of the W3C Patent Policy.
As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.
The key words MAY, MUST, MUST NOT, RECOMMENDED, and SHOULD in this document
are to be interpreted as described in
BCP 14
[RFC2119] [RFC8174]
when, and only when, they appear in all capitals, as shown here.
This specification defines conformance criteria that apply to a single
product: the user agent that implements the interfaces that
it contains.
Implementations that use ECMAScript to expose the APIs defined in this
specification MUST implement them in a manner consistent with the
ECMAScript Bindings defined in the Web IDL specification [WEBIDL].
2.
Introduction
This section is non-normative.
This specification aims to make remote playback devices such as
connected TVs, projectors or audio-only speakers, available to the Web
and takes into account playback devices that are attached using wired
(HDMI, DVI, or similar) and wireless technologies (Miracast,
Chromecast, DLNA, AirPlay, or similar).
Devices with limited screen size or quiet speakers lack the ability to
playback media content to a larger audience, for example, a group of
colleagues in a conference room, or friends and family at home. Playing
media content on an external larger and/or louder remote playback
device helps to improve the perceived quality and impact of the
played media.
At its core, this specification enables a page that acts as the
browsing context to initiate and control remote playback of a
particular media element on a selected remote playback device.
How the remoting is initiated and controlled is left to the UA in order
to allow the use of remote playback devices that can be attached
in a wide variety of ways. For example, when a remote playback
device is attached using HDMI or Miracast, the same UA that acts as
the browsing context renders the remote media. Instead of
playing the media on the same device, however, it can use whatever
means the operating system provides for using the external remote
playback device. In such a case, both the browsing context
and the media player run on the same UA and the operating system is
used to route the player output to the remote playback device.
This is commonly referred to as the media
mirroring case. This specification imposes no requirements on
the remote playback devices connected in such a manner.
If the remote playback device is able to fetch and play the media
and communicate with the browsing context, the browsing
context does not need to fetch or render the remoted media. In this
case, the UA acts as a proxy that requests the remote playback
device to play the media itself by passing the necessary data like
the media source. This is commonly referred to as the media
flinging case. This way of attaching to displays could be
enhanced in the future by defining a standard protocol for delivering
these types of messages that remote playback devices could choose to
implement.
The API defined here is intended to be used with UAs that attach to
remote playback device devices through any of the above means.
3.
Use cases and requirements
This section is non-normative.
The use cases and requirements of this specification are captured in a
separate document available here.
4.
Examples
This section shows code examples that highlight the usage of the main
features of the Remote Playback API. In these examples,
player.html implements the player page controlling the
remote playback and media.ext is the media file to be
played remotely. Both the page and the media are served from the domain
https://example.org. Please refer to the comments in the
code examples for further details.
4.1
Monitor availability of remote playback devices example
<!-- player.html --><!-- The video element with custom controls that supports remote playback. --><videoid="videoElement"src="https://example.org/media.ext" /><buttonid="deviceBtn"style="display: none;">Pick device</button><script>// The "Pick device" button is visible if at least one remote playback device is available.const deviceBtn = document.getElementById("deviceBtn");
const videoElem = document.getElementById("videoElement");
functionavailabilityCallback(available) {
// Show or hide the device picker button depending on device availability.
deviceBtn.style.display = available ? "inline" : "none";
}
videoElem.remote.watchAvailability(availabilityCallback).catch(() => {
// Availability monitoring is not supported by the platform, so discovery of// remote playback devices will happen only after remote.prompt() is called.// Pretend the devices are available for simplicity; or, one could implement// a third state for the button.
deviceBtn.style.display = "inline";
});
</script>
<!-- player.html --><script>
deviceBtn.onclick = () => {
// Request the user to select a remote playback device.
videoElem.remote.prompt()
// Update the UI and monitor the connected state.
.then(updateRemotePlaybackState);
// Otherwise, the user cancelled the selection UI or no screens were found.
};
<script>
<!-- player.html --><script>// The remote playback may be initiated by the user agent,// so check the initial state to sync the UI with it.if (videoElem.remote.state == "disconnected")
switchToLocalUI();
else
switchToRemoteUI();
videoElem.remote.onconnecting = switchToRemoteUI;
videoElem.remote.onconnect = switchToRemoteUI;
videoElem.remote.ondisconnect = switchToLocalUI;
// Handles both 'connecting' and 'connected' state. Calling more than once// is a no-op.functionswitchToRemoteUI() {
// Indicate that the state is 'connecting' or 'connected' to the user.// For example, hide the video element as only controls are needed.
videoElem.style.display = "none";
// Stop monitoring the availability of remote playback devices.
videoElem.remote.cancelWatchAvailability();
};
functionswitchToLocalUI() {
// Show the video element.
videoElem.style.display = "inline";
// Start watching the device availability again.
videoElem.remote.watchAvailability(availabilityCallback);
};
<script>
5.
API
5.1
Common idioms
A local playback device is the device the browsing
context is running on along with the default video/audio outputs
the device has.
Note
A local playback device might have extra outputs, like an
external display or speakers/headphones. As long as the switch of
what output to use happens outside of the user agent on the
system level, the playback is considered to happen on a local
playback device for the purpose of this spec.
A media element state is the set of all single
media element
properties observable by the page and/or the user via the
user agent implementation. The new properties introduced by
this spec are not considered part of the media element state
for convenience.
For example, the paused attribute or the pause/resume
button reflecting that state on the default controls of the media
element would be a part of media element state.
The user agent MUST keep track of the set of availability
callbacks registered with each media element through
the watchAvailability()
method. The set of availability callbacks for each
RemotePlayback object is represented as a set of tuples
(callbackId, callback),
initially empty, where:
5.2.1.2
The list of available remote playback devices
The user agent MUST keep a list of available remote
playback devices. This list contains remote playback
devices and is populated based on an implementation specific
discovery mechanism. It is set to the most recent result of the
algorithm to monitor the list of available remote playback
devices or an empty list if the algorithm hasn't been run
yet.
Some remote playback devices may only be able to play a
subset of media resources
because of functional, security or hardware limitations. Examples
are set-top boxes, smart TVs or networked speakers capable of
rendering only certain formats of video and/or audio. We say that
such a device is a compatible remote playback device
for a media resource
if the user agent can reasonably guarantee that the
remote playback of the media specified by the resource will
succeed on that device.
A simple algorithm for assigning callbackId values is
to keep a counter for each browsing context and
incrementing it in step 6.
Note
To avoid leaking information that could fingerprint the user, the
user agent is expected not to assign a callbackId that
uses any persistent information from the browser profile or a
remote playback device.
5.2.1.4
Monitoring the list of available remote playback devices
If there is already an unsettled promise from a previous call to
prompt() for the same media element or
even for the same browsing context, the user agent MAY reject
promise with an OperationError exception and abort all
remaining steps.
Note
The rationale here is that the user agent might show a dialog
that's modal to either the media element or the
browsing context. In such a case, the second call to
prompt() would not be able to show any UI.
An example of this situation is when the user agent only
supports media flinging, and the media element's source
is not a URL that can be passed to a
remote playback device.
If remote playback is unavailable and will remain so
before the request for user permission is complete,
reject promise with a NotFoundError exception and
abort all remaining steps.
Request user permission to change remote playback
state.
The user may have already selected a remote playback
device for a related purpose, for example, to mirror the
contents of the display or the current page. In that case, the
user agent may choose to skip the UI to select a device and
proceed immediately to the next step.
If the user picked a remote playback devicedevice to initiate remote playback with, the
user agent MUST run the following steps:
Otherwise, the user is considered to deny permission to
use the device, so reject promise with NotAllowedError
exception and hide the UI shown by the user agent.
Note
The details of implementing the UI and device selection are left to
the user agent; for example it can show the user a dialog and allow
the user to select an available device (granting
permission), or cancel the selection (denying
permission). In most cases, available devices will advertise
a user friendly name along with locale and text direction
information for that name. The user agent is encouraged to render
this user friendly name, using its locale and text direction when
they are known.
5.2.3
The state attribute
The state attribute represents the
RemotePlayback connection's current state. It can take one of
the values of RemotePlaybackState depending on the connection
state:
connecting means
that the user agent is attempting to initiate remote
playback with the selected remote playback device. This
is the initial state when the promise returned by
prompt() is fulfilled. The local playback of
the media element continues in this state and media commands still
take effect on the local playback state.
connected means that
the transition from local to remote playback has finished and all
media commands now take effect on the remote playback
state.
disconnected means
that the remote playback has not been initiated, has failed to initiate or has been
stopped. All media commands will take effect on the local
playback state. The remote playback can be initiated through a
call to prompt().
5.2.4
Establishing a connection with a remote playback device
When the user agent is to establish a connection with
the remote playback device, it MUST run the following steps:
Input
remote, the RemotePlayback object that is to be
connected.
The mechanisms that are used to connect the user agent with the
remote playback device, select the remote playback
source, and initiate playback are all implementation choices of
the user agent. The connection will likely have to provide a two-way
messaging abstraction capable of carrying media commands to the
remote playback device and receiving media playback state in order
to keep the media element state and remote playback
state in sync (unless media mirroring is used).
Note
The user agent is encouraged to pass locale and text direction
information to the remote playback device when possible, so
that the remote playback device can adjust its user interface
and operational functions to locale-specific attributes that reflect
the user's preferences. For example, the remote playback
device can use that information to choose the same default text
track as the user agent, as well as to set the HTTP
Accept-Language header it sends to fetch media resources.
Note
The user agent should not render output from the media element while
it is in a "connected" state and its content
is being rendered on a remote playback device.
5.2.5
Browser initiated remote playback
A user agent MAY support connecting
to a remote playback device from the browser. This can be done by
adding appropriate media controls to the user interface that is
exposed to the user, or when the user activates system-wide
mirroring of the display. This feature is known as browser
initiated remote playback. A user agent that supports
browser initiated remote playbackSHOULD initiate the remote
playback only when the user has expressed an intention to do so via
a user gesture, for example by clicking a button in the browser.
A user agent that implements browser initiated remote playback
should consider how it interacts with other browser policies that
affect media playback, such as which media elements
are allowed to play
and background media playback optimizations.
The remote playback device can implement a subset
of the capabilities of the playback engine of the user
agent, and some HTMLMediaElement APIs do not always
make sense to use during remote playback. In this case, the
local playback state is expected to reflect as closely
as possible the actual remote playback state after a
media command that is not supported during remote playback.
The remote playback device might not actually stop playback of the
media when requested by the user agent; it depends on the
implementation of the user agent and the remote playback device.
In this case, stopping remote playback means that the user agent
merely disconnects from the remote playback device and the
media element switches to the disconnected state.
Some pages may wish to disable remote playback of a media
element; for example, they may prefer to use a PresentationRequest
to present a complete document on a presentation screen. To support
this use case, a new disableRemotePlayback attribute is added to
the list of content attributes for audio and video elements.
A corresponding disableRemotePlayback IDL attribute which
reflects the value of each element's disableRemotePlayback
content attribute is added to the HTMLMediaElement interface.
The disableRemotePlayback IDL attribute MUST
reflect the content attribute of the same name.
5.3.2
Disabling remote playback
If the disableRemotePlayback attribute is present
on the media element, the user agent MUST NOT play
the media element remotely or present any UI to do so.
Firing the callback provided via the
watchAvailability() method reveals one bit of
information about the presence (or non-presence) of a
remote playback device typically discovered through the local
area network. This could be used in conjunction with other
information for fingerprinting the user. However, this information is
also dependent on the user's local network context, so the risk is
minimized. Also, by design, the human readable name of a remote
playback device is not revealed to the page.
Display of the origin requesting remote playback will help the
user understand what content is making the request, especially
when the request is initiated from a child navigable. For
example, embedded content may try to convince the user to click to
trigger a request to start an unwanted remote playback.
Showing the origin that will be presented will help the user know
if that content is from a potentially trustworthy origin
(e.g., https:), and corresponds to a known or expected site.
6.3
Device Access
The Remote Playback API abstracts away what "local" means for
displays, meaning that it exposes network-accessible displays as
though they were local displays. The Remote Playback API requires
user permission for a page to access any display to mitigate issues
that could arise, such as showing unwanted content on a display
viewable by others.
6.4
Messaging between the local and remote playback devices
This specification will not mandate communication protocols between
the local playback device and the remote playback
device, but the user agent should set some guarantees of message
confidentiality and authenticity between them.
For this specification to be advanced to Proposed Recommendation, there
must be at least two independent, interoperable implementations of each
feature. Each feature may be implemented by a different set of
products, there is no requirement that all features be implemented by a
single product. Additionally, implementations must demonstrate support
for the media remoting and media flinging cases, either
within the same product or within different products.
For the purposes of these criteria, we define the following terms:
Independent
Each implementation must be developed by a different party, and
cannot share, reuse, or derive from code used by another qualifying
implementation. Sections of code that have no bearing on the
implementation of this specification are exempt from this
requirement.
Interoperable
Passing the respective test case(s) in the official test suite.
Implementation
A user agent which:
implements the specification.
is available to the general public. The implementation may be a
shipping product or other publicly available version (i.e., beta
version, preview release, or "nightly build"). Non-shipping product
releases must have implemented the feature(s) for a period of at
least one month in order to demonstrate stability.
is not experimental (i.e. a version specifically designed to
pass the test suite and not intended for normal usage going
forward).