This document defines how a user's display, or parts thereof, can be used as the source of a media stream using {{MediaDevices/getDisplayMedia}}, an extension to the Media Capture API [[GETUSERMEDIA]].

This document is not complete. It is subject to major changes and, while early experimentations are encouraged, it is therefore not intended for implementation.

Introduction

This document describes an extension to the Media Capture API [[GETUSERMEDIA]] that enables the acquisition of a user's display, or part thereof, in the form of a video track. In some cases system, application or window audio is also captured which is presented in the form of an audio track. This enables a number of applications, including screen sharing using WebRTC [[WEBRTC]].

This feature has signficant security implications. Applications that use this API to access information that is displayed to users could access confidential information from other origins if that information is under the control of the application. This includes content that would otherwise be inaccessible due to the protections offered by the user agent sandbox.

This document concerns itself primarily with the capture of video and audio [[GETUSERMEDIA]], but the general mechanisms defined here could be extended to other types of media, of which depth [[MEDIACAPTURE-DEPTH]] is currently defined.

This specification defines conformance criteria that apply to a single product: the user agent that implements the interfaces that it contains.

Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[!WEBIDL]], as this specification uses that specification and terminology.

Example

The following example demonstrates a request for display capture using the navigator.mediaDevices.getDisplayMedia method defined in this document.

try {
  let mediaStream = await navigator.mediaDevices.getDisplayMedia({video:true});
  videoElement.srcObject = mediaStream;
} catch (e) {
  console.log('Unable to acquire screen capture: ' + e);
}

Terminology

This document uses the definition of {{MediaStream}}, {{MediaStreamTrack}} and {{ConstrainablePattern}} from [[!GETUSERMEDIA]].

Screen capture encompasses the capture of several different types of screen-based surfaces. Collectively, these are referred to as display surfaces, of which this document defines the following types:

This document draws a distinction between two variants of each type of display surface:

Some operating systems permit windows from different applications to occlude other windows, in whole or part, so the visible display surface is a strict subset of the logical display surface.

The source pixel ratio of a display surface is 1/96th of 1 inch divided by its vertical pixel size.

The devicechange event is defined in [[GETUSERMEDIA]].

Capturing Displayed Media

Capture of displayed media is enabled through the addition of a new {{MediaDevices/getDisplayMedia}} method on the {{MediaDevices}} interface, that is similar to {{MediaDevices/getUserMedia()}} , except that it acquires media from one display device chosen by the end-user each time.

MediaDevices Additions

partial interface MediaDevices {
  Promise<MediaStream> getDisplayMedia(optional DisplayMediaStreamOptions options = {});
};
getDisplayMedia

Prompts the user for permission to live-capture their display.

The user agent MUST let the end-user choose which [=display surface=] to share out of all available choices every time, and MUST NOT use any {{MediaTrackConstraints}} in options.video or options.audio to limit that choice.

The user agent MAY use the presence of the {{displaySurface}} constraint and its value to influence the presentation to the user of the sources to pick from. The user agent MUST still offer the user unlimited choice of any [=display surface=]. The user agent is strongly recommended to steer users away from sharing a monitor, as this poses risks to user privacy.

Any {{MediaTrackConstraints}} in options.video or options.audio MUST be applied to the media chosen by the user only after the user has made their selection.

In the case of audio, the user agent MAY present the end-user with audio sources to share. Which choices are available to choose from is up to the user agent, and the audio source(s) are not necessarily the same as the video source(s). An audio source may be a particular [= display surface/window =], [= display surface/browser =], the entire system audio or any combination thereof. Unlike {{MediaDevices/getUserMedia()}} with regards to audio+video, the user agent is allowed not to return audio even if the audio constraint is present. If the user agent knows no audio will be shared for the lifetime of the stream it MUST NOT include an audio track in the resulting stream. The user agent MAY accept a request for audio and video by only returning a video track in the resulting stream, or it MAY accept the request by returning both an audio track and a video track in the resulting stream. The user agent MUST reject audio-only requests.

In addition to drawing from a different set of sources and requiring user selection, {{MediaDevices/getDisplayMedia}} also differs from {{MediaDevices/getUserMedia()}} in that {{PermissionState/"granted"}} permissions cannot be persisted.

When the {{MediaDevices/getDisplayMedia()}} method is called, the user agent MUST run the following steps:

  1. Let mediaDevices be [=this=].

  2. Let controller be options.controller if present, or null otherwise.

  3. If controller is not null, run the following steps:

    1. If controller.{{CaptureController/[[IsBound]]}} is true, return a promise [=reject|rejected=] with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

    2. Set controller.{{CaptureController/[[IsBound]]}} to true.

  4. If the [=relevant global object=] of [=this=] does not have [=transient activation=], return a promise rejected with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

  5. Let options be the method's first argument.

  6. Let constraints be [options.audio, options.video].

  7. If constraints.video is false, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

  8. For each [= map/exist | existing =] member in constraints whose value, CS, is a dictionary, run the following steps:

    1. If CS contains a member named advanced, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

    2. If CS contains a member whose name specifies a constrainable property applicable to [=display surface=]s, and whose value in turn is a dictionary containing a member named either min or exact, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

    3. If CS contains a member whose name specifies a constrainable property applicable to [=display surface=]s, and whose value in turn is a dictionary containing a member named max, and that member's value in turn is less than the constrainable property's [=floor value=], then let failedConstraint be the name of the member, let message be either undefined or an informative human-readable message, and return a promise [=reject|rejected=] with a new OverconstrainedError created by calling OverconstrainedError(failedConstraint, message).

  9. Let requestedMediaTypes be the set of media types in constraints with either a dictionary value or a value of true.

  10. If the current settings object's [=relevant global object=]'s [=associated `Document`=] is NOT [=Document/fully active=] or does NOT have focus, return a promise [=reject|rejected=] with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

  11. Let p be a new promise.

  12. Run the following steps [=in parallel=]:

    1. For each media type T in requestedMediaTypes,

      1. If no sources of type T are available, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotFoundError}}.

      2. Read the current [= permission state=] for obtaining sources of type T in the current browsing context. If the permission state is {{PermissionState/"denied"}}, jump to the step labeled PermissionFailure below.

    2. Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled Permission Failure below.

    3. [=Prompt the user to choose=] a display device, for a {{PermissionDescriptor}} with its {{PermissionDescriptor/name}} set to "display-capture", resulting in a set of provided media.

      The provided media MUST include precisely one video track.

      The provided media MUST include at most one audio track. This audio track MUST NOT be included if audio was not specified in requestedMediaTypes, or if it was specified as false.

      The devices chosen MUST be the ones determined by the user. Once selected, the source of a {{MediaStreamTrack}} MUST NOT change, unless the user permits it through their interaction with the user agent.

      User agents are encouraged to warn users against sharing [= display surface/browser =] display devices as well as [= display surface/monitor =] display devices where browser windows are visible, or otherwise try to discourage their selection on the basis that these represent a significantly higher risk when shared.

      If the result of the request is {{PermissionState/"granted"}}, then for each device that is sourcing the provided media, using a stable and private id for the device, deviceId, set [[\devicesLiveMap]][deviceId] to true, if it isn’t already true, and set the [[\devicesAccessibleMap]][deviceId] to true, if it isn’t already true.

      The user agent MUST NOT store a {{PermissionState/"granted"}} permission entry.

      If the result is {{PermissionState/"denied"}}, jump to the step labeled Permission Failure below. If the user never responds, this algorithm stalls on this step.

      If the user grants permission but a hardware error such as an OS/program/webpage lock prevents access, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotReadableError}} and abort these steps.

      If the result is {{PermissionState/"granted"}} but device access fails for any reason other than those listed above, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{AbortError}} and abort these steps.

    4. Let stream be a {{MediaStream}} object.

    5. For each source that the user granted permission to, run the following steps:

      1. Let track be the result of [=create a MediaStreamTrack|creating a MediaStreamTrack=] with source and mediaDevices.

      2. Add track to stream's track set.

      3. [=Tie track source to MediaDevices=] with source and mediaDevices.

    6. Run the [=ApplyConstraints algorithm=] on all tracks in stream with the appropriate constraints. Should this fail, let failedConstraint be the result of the algorithm that failed, and let message be either undefined or an informative human-readable message, and then reject p with a new OverconstrainedError created by calling OverconstrainedError(failedConstraint, message).

    7. This invocation of {{MediaDevices/getDisplayMedia()}} is now considered to have produced a new capture-session.

    8. If controller is not null, run the following steps:
      1. Set controller.{{CaptureController/[[Source]]}} to stream's video track's [[\Source]].

      2. Set controller.{{CaptureController/[[DisplaySurfaceType]]}} to the to stream's video track's {{DisplayCaptureSurfaceType}}.

      3. Queue a task to run the [=finalize focus decision algorithm=] on controller.

    9. Resolve p with stream and abort these steps.

    10. Permission Failure: [=Reject=] p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotAllowedError}}.

  13. Return p.

When the top-level document loses focus, run the following steps on all {{CaptureController}} objects in that document and in documents of its nested [=browsing contexts=]:

  1. If {{CaptureController/[[Source]]}} is undefined, abort these steps.

  2. Set {{CaptureController/[[FocusChangeDisabled]]}} to true.

The user agent MUST NOT capture content that's behind a partially transparent captured display surface.

For the newly created {{MediaStreamTrack}}, the user agent MUST NOT capture the prompt that was shown to the user.

Information that is not currently rendered to the screen SHOULD be obscured in captures unless the application has been specifically authorized to access that content (e.g. through means such as elevated permissions).

The user agent MUST NOT share audio without active user consent, for example if the capture of the video of a [= display surface/window =] is accompanied by capture of the audio of the entire system, including applications unrelated to that window.

Closed and Minimized Display Surfaces

A display surface that is being shared may temporarily or permanently become inaccessible to the application because of actions taken by the operating system or user agent. What makes a display surface considered inaccesible is outside the scope of this specification, but examples MAY include a [= display surface/monitor =] disconnecting, [= display surface/window =] or [= display surface/browser =] closing or becoming minimized, or due to an incoming call on a phone.

User agents ultimately control what inaccesible means in this context, but are encouraged to only fire mute and unmute events for interruptions that have external reasons.

When display surface enters an inaccessible state that is not necessarily permanent, the user agent MUST queue a task that [= set a track's muted state|sets the muted state =] of the corresponding media track to true.

When display surface exits an inaccessible state and becomes accessible, the user agent MUST queue a task that [= set a track's muted state|sets the muted state =] of the corresponding media track to false.

When a display surface enters an inaccessible state that is permanent (such as the source [= display surface/window=] closing), the user agent MUST queue a task that [= track/ended | ends =] the corresponding media track.

A stream that was just returned by {{MediaDevices/getDisplayMedia}} MAY contain tracks that are muted by default. Audio and video tracks belonging to the same stream MAY be muted/unmuted independently of one another.

Unconstrained Display Surface Selection

Not accepting constraints for source selection means that {{MediaDevices/getDisplayMedia}} only provides fingerprinting surface that exposes whether audio, video or audio and video display sources are present. (This is a fingerprinting vector.)

Note that accepting the {{displaySurface}} constraint does not limit user selection.

Constrainable Properties for Captured Display Surfaces

Constraints serve a different purpose in {{MediaDevices/getDisplayMedia}} than they do in {{MediaDevices/getUserMedia()}}. They do not aid discovery, instead they are applied only after user-selection.

This section define which constraints apply to {{MediaDevices/getDisplayMedia}} tracks; constraints defined for {{MediaDevices/getUserMedia()}} do not apply unless listed here.

Some of these constraints enable user agent processing like downscaling and frame decimation, as well as display-specific features. Others enable observation of inherent properties of a user-selected display surface, as capabilities and settings.

The following new and existing {{MediaStreamTrack}} Constrainable Properties are defined to apply to the user-selected video display surface, with the following behavior:

Property Name Type Behavior
width {{unsigned long}} The width, in pixels. As a capability, max MUST reflect the display surface's width, and min MUST reflect the width of the smallest aspect-preserving representation available through downscaling by the user agent.
height {{unsigned long}} The height, in pixels. As a capability, max MUST reflect the display surface's height, and min MUST reflect the height of the smallest aspect-preserving representation available through downscaling by the user agent.
frameRate {{double}} The frame rate (frames per second). As a capability, max MUST reflect the display surface's frame rate, and min MUST reflect the lowest frame rate available through frame decimation by the user agent.
aspectRatio {{double}} The exact aspect ratio (width in pixels divided by height in pixels, represented as a double rounded to the tenth decimal place) or aspect ratio range. As a setting, represents width / height. As a capability, min and max both MUST be the current setting value, rendering this property immutable from the application viewpoint.
resizeMode {{DOMString}} This string is one of the members of {{VideoResizeModeEnum}}. As a setting, {{VideoResizeModeEnum/"none"}} means the {{MediaStreamTrack}} contains all bits needed to render the display in full detail, which if the source pixel ratio > 1, means width and height will be larger than the display's appearance from an end-user viewpoint would suggest, whereas {{VideoResizeModeEnum/"crop-and-scale"}} means the {{MediaStreamTrack}} contains an aspect-preserved representation of the display surface that has been downscaled by the user agent, but not cropped. As a capability, the values {{VideoResizeModeEnum/"none"}} and {{VideoResizeModeEnum/"crop-and-scale"}} both MUST be present.
displaySurface {{DOMString}}

This string is one of the members of {{DisplayCaptureSurfaceType}}.

As a setting, indicates the type of [=display surface=] that is being captured.

As a capability, the setting value MUST be the lone value present, rendering this property immutable from the application viewpoint.

As a constraint, the value signals the application's preference of a particular [=display surface=] type to the user agent; the user agent MAY reorder the options offered to the user according to that preference. This constraint is ignored for all other purposes, and can therefore not cause any side effects (such as being the cause of OverconstrainedError).

logicalSurface {{boolean}} As a setting, a value of true indicates capture of a [=logical display surface=], whereas a value of false indicates a capture of a [=visible display surface=]. As a capability, this same value MUST be the lone value present, rendering this property immutable from the application viewpoint.
cursor {{DOMString}} This string is one of the members of {{CursorCaptureConstraint}}. As a setting, indicates if and when the cursor is included in the captured [=display surface=]. As a capability, the user agent MUST include only the set of values from {{CursorCaptureConstraint}} it is capable of supporting for this [=display surface=].

The following new and existing {{MediaStreamTrack}} Constrainable Properties are defined to apply to the user-selected audio sources, with the following behavior:

Property Name Type Behavior
restrictOwnAudio {{boolean}}

As a setting, this value indicates whether or not the user agent is applying own audio restriction to the source.

As a constraint, this property can be constrained resulting in a source with own audio restriction enabled or disabled.

When own audio restriction is applied, the user agent MUST attempt to remove any audio from the audio being captured that was produced by the document that performed {{MediaDevices/getDisplayMedia}}. If the user agent is not able to remove the audio through processing it SHOULD remove the audio by excluding the document's audio from being captured. If this results in no audio being captured, the user agent MUST keep the track muted until it is able to capture audio again.

suppressLocalAudioPlayback {{boolean}}

As a setting, this value indicates whether or not the application instructed the user agent to apply local audio playback suppression to the source.

As a constraint, this value is only meaningful if the user selects capturing a [= display surface/browser=] [=display surface=]. In that case, a value of true indicates that the user agent SHOULD perform local audio playback suppression on the captured [= display surface/browser=] [=display surface=].

When local audio playback suppression is applied, the user agent SHOULD stop relaying audio to the local speakers, but that audio MUST still be captured by any ongoing audio-capturing [=capture-sessions=]. This suppression MUST NOT be observable to the captured document. Furthermore, the capturing document may only observe whether it is applying suppressLocalAudioPlayback; not whether that suppression is having an effect (i.e. can't observe if the user is overriding this in the user agent).

When a [= display surface/browser=] [=display surface=] is subject to multiple concurrent captures, local audio playback suppression SHOULD be applied as long as at least one active audio-capturing [=capture-session=] is constraining suppressLocalAudioPlayback to true.

When inherent properties of the underlying source of a user-selected display surface change, for example in response to the end-user resizing a captured window, and these changes render the capabilities and/or settings of one or more constrainable properties outdated, the user agent MUST queue a task to run the following step:

  1. Update all affected constrainable properties at the same time.

    If this causes an "overconstrained" situation, then the user agent MUST ignore the culprit constraints for as long as they overconstrain. The user agent MUST NOT mute the track.

While min and exact constraints produce TypeError on getDisplayMedia(), this specification does not alter the track.applyConstraints() method. Therefore, they may instead produce OverconstrainedError or succeed depending on values, and therefore potentially be present to cause this "overconstrained" situation. The max constraint may also cause this, e.g. with aspectRatio. This spec considers these to be edge cases that aren't useful.

Downscaling and Frame Decimation

For the purposes of the [=SelectSettings=] algorithm, the user agent SHOULD consider all possible combinations of downscaled dimensions that preserve the aspect ratio of the original display surface (to the nearest pixel), and frame rates available through frame decimation, as available [= settings dictionaries =].

The downscaling and decimation effects of constraints is then effectively governed by the [= fitness distance =] algorithm.

The intent is for the user agent to produce output that is close to the ideal width, ideal height, and/or ideal frameRate when these are specified, while at all times preserving the aspect ratio of the original display surface.

The user agent SHOULD downscale by the source pixel ratio by default, unless otherwise directed by applied constraints.

The user agent MUST NOT crop the captured output.

The user agent MUST NOT upscale the captured output, or create additional frames, except as needed to preserve high resolutions and frame rates in an aggregated display surface.

The max constraint type lets a web application provide a maximum envelope for constrainable properties like width and height. This is helpful to limit extreme aspect ratios, should the end-user resize a [= display surface/window =] or [= display surface/browser =] surface to such an extreme while it is being captured.

For each constrainable property of positive numeric type in this specification, the user agent MUST establish a floor value, representing the smallest allowable value supported by the user agent regardless of source. This value MUST be constant and MUST be greater than 0. The user agent is encouraged to support all values above the floor value regardless of source.

The purpose of the floor value is to help user agents avoid failing {{MediaDevices/getDisplayMedia()}} with OverconstrainedError after the user has already been prompted, and avoid leaking information about the user's system.

CaptureStartFocusBehavior

Describes whether an application invoking {{CaptureController/setFocusBehavior()}} would like the user agent to focus the [=display surface=] associated with that {{CaptureController}}'s [=capture-session=].

            enum CaptureStartFocusBehavior {
              "focus-capturing-application",
              "focus-captured-surface",
              "no-focus-change"
            };    
          
Enumeration description
focus-capturing-application The application prefers to be focused.
focus-captured-surface The application prefers that the [=display surface=] associated with this {{CaptureController}}'s [=capture-session=] be focused.
no-focus-change The application prefers that the user agent not change focus, leaving focus with whichever surface last had focus following the user's interaction with the user agent and/or operating system.
The Working Group is investigating the possibility of deprecating {{CaptureStartFocusBehavior/"no-focus-change"}}.

CaptureController

A {{CaptureController}} object may be associated with a [=capture-session=]. It would be used to expose functionality that's associated with the [=capture-session=] itself, rather than with the call to {{MediaDevices/getDisplayMedia()}} or its resulting stream or tracks.

Any given [=capture-session=] is associated with at most one {{CaptureController}}.

At most one {{CaptureController}} is associated with any given [=capture-session=].

            [Exposed=Window, SecureContext]
            interface CaptureController : EventTarget {
              constructor();
              undefined setFocusBehavior(CaptureStartFocusBehavior focusBehavior);
            };
          

{{CaptureController}} does not yet define event handlers, so it is not required to inherit from {{EventTarget}}. This is for the benefit of future specifications that extend {{CaptureController}} with event handler attributes; if inheritance is not used, it can be removed.

constructor
Create a new {{CaptureController}} object with the following internal slots:
Internal Slot Initial value Description (non-normative)
[[\IsBound]] false Whether an application has attempted to associate [=this=] with a [=capture-session=].
[[\Source]] null The source of the associated [=capture-session=].
[[\DisplaySurfaceType]] null Once capture starts, this will be set to the type of the captured [=display surface=].
[[\FocusChangeDisabled]] false Whether focus-change has been disabled by an external event or a user agent consideration.
[[\FocusDecisionFinalized]] false Set to true when the focus decision is finalized.
[[\FocusBehavior]] null The focus behavior desired by the application.

The user agent MAY set {{CaptureController/[[FocusChangeDisabled]]}} to true at any moment based on its own logic.

setFocusBehavior

Run the following steps:

  1. Let focusBehavior be the method's first argument.

  2. If [=this=].{{CaptureController/[[Source]]}} is null, set [=this=].{{CaptureController/[[FocusBehavior]]}} to focusBehavior and abort these steps.

  3. If [=this=].{{CaptureController/[[Source]]}} has been stopped, [=exception/throw=] an "{{InvalidStateError}}" {{DOMException}}.

  4. If [=this=].{{CaptureController/[[DisplaySurfaceType]]}} is neither {{DisplayCaptureSurfaceType/"browser"}} nor {{DisplayCaptureSurfaceType/"window"}}, [=exception/throw=] an "{{InvalidStateError}}" {{DOMException}}.

  5. If [=this=].{{CaptureController/[[FocusDecisionFinalized]]}} is true, [=exception/throw=] an "{{InvalidStateError}}" {{DOMException}}.

  6. Set [=this=].{{CaptureController/[[FocusBehavior]]}} to focusBehavior.

  7. Run the [=finalize focus decision algorithm=] on [=this=].

The finalize focus decision algorithm, given a controller, consists of running the following steps:

  1. If too much time has elapsed since the [=capture-session=] started, the user agent SHOULD set {{CaptureController/[[FocusDecisionFinalized]]}} to true. The timespan is left up to the user agent, but it is recommended that a value of one second be used.

  2. If controller.{{CaptureController/[[FocusDecisionFinalized]]}} is true, abort these steps.

  3. Set controller.{{CaptureController/[[FocusDecisionFinalized]]}} to true.

  4. If controller.{{CaptureController/[[FocusChangeDisabled]]}} is true, abort these steps.

  5. If controller.{{CaptureController/[[DisplaySurfaceType]]}} is neither {{DisplayCaptureSurfaceType/"browser"}} nor {{DisplayCaptureSurfaceType/"window"}}, abort these steps.

  6. Let focusBehavior be controller.{{CaptureController/[[FocusBehavior]]}}.

  7. Run the following steps [=in parallel=]:

    1. If focusBehavior is {{CaptureStartFocusBehavior/"focus-capturing-application"}}, focus the [=display surface=] representing the capturing document.

    2. If focusBehavior is {{CaptureStartFocusBehavior/"focus-captured-surface"}}, focus the [=display surface=] referred to by controller.{{CaptureController/[[Source]]}}.

SelfCapturePreferenceEnum

Describes the different hints an application can provide about whether the [=display surface=] the application is in, should be among the choices offered to the user.

            enum SelfCapturePreferenceEnum {
              "include",
              "exclude"
            };
          
Enum value Description
include The application prefers the surface be included among the choices offered.
exclude The application prefers the surface be excluded from the choices offered.

SystemAudioPreferenceEnum

Describes whether an application invoking {{MediaDevices/getDisplayMedia()}} would like the user agent to include system audio among the audio sources offered to the user.

            enum SystemAudioPreferenceEnum {
              "include",
              "exclude"
            };
          
Enumeration description
include The application prefers that options to share system audio be offered to the user for [=display surface/monitor=] [=display surfaces=].
exclude The application prefers that options to share system audio not be offered to the user.

SurfaceSwitchingPreferenceEnum

Describes whether an application invoking {{MediaDevices/getDisplayMedia()}} would like the user agent to offer the user an option to dynamically switch the source [=display surface=] during the capture.

            enum SurfaceSwitchingPreferenceEnum {
              "include",
              "exclude"
            };
          
Enumeration description
include The application prefers that an option to dynamically switch the source [=display surface=] during the capture be offered to the user.
exclude The application prefers that an option to dynamically switch the source [=display surface=] during the capture NOT be offered to the user.

MonitorTypeSurfacesEnum

Describes whether the application would like the user agent to offer the user the option to choose [=display surfaces=] whose type is [=display surface/monitor=].

            enum MonitorTypeSurfacesEnum {
              "include",
              "exclude"
            };
          
Enumeration description
include The application prefers that the [=display surfaces=] presented to the user include those of type [=display surface/monitor=].
exclude The application prefers that the [=display surfaces=] presented to the user exclude those of type [=display surface/monitor=].

DisplayMediaStreamOptions

The DisplayMediaStreamOptions dictionary is used to instruct the user agent what sort of {{MediaStreamTrack}}s may be included in the {{MediaStream}} returned by {{MediaDevices/getDisplayMedia}}.

dictionary DisplayMediaStreamOptions {
  (boolean or MediaTrackConstraints) video = true;
  (boolean or MediaTrackConstraints) audio = false;
  CaptureController controller;
  SelfCapturePreferenceEnum selfBrowserSurface;
  SystemAudioPreferenceEnum systemAudio;
  SurfaceSwitchingPreferenceEnum surfaceSwitching;
  MonitorTypeSurfacesEnum monitorTypeSurfaces;
};

Dictionary DisplayMediaStreamOptions Members

video of type (boolean or {{MediaTrackConstraints}}), defaulting to true

If true, it requests that the returned {{MediaStream}} contain a video track. If a Constraints structure is provided, it further specifies desired processing options to be applied to the video track rendition of the display surface chosen by the user. If false, the request will be [=rejected=] with a {{TypeError}}, as per the getDisplayMedia algorithm.

audio of type (boolean or {{MediaTrackConstraints}}), defaulting to false

If true, it signals an interest that the returned {{MediaStream}} contain an audio track, if supported and audio is available for display surface chosen by the user. If a Constraints structure is provided, it further specifies desired processing options to be applied to the audio track. If false, the {{MediaStream}} will not contain an audio track.

controller of type {{CaptureController}}

If present, this {{CaptureController}} object will be associated with the [=capture-session=]. Through the methods exposed on this object, the [=capture-session=] can be manipulated.

selfBrowserSurface of type {{SelfCapturePreferenceEnum}}
If present, signals application preference for whether the [=display surface/browser=] [=display surface=] which is associated with [=this=]'s [=relevant global object=]'s [=associated Document=]'s [=top-level browsing context=], should be among the choices offered to the user. The user agent MAY ignore this hint.
systemAudio of type {{SystemAudioPreferenceEnum}}
If present, signals whether the application would like system audio to be included among the possible audio sources offered to the user. The user agent MAY ignore this hint.
surfaceSwitching of type {{SurfaceSwitchingPreferenceEnum}}
If present, signals whether the application would like the user agent to offer the user an option to dynamically switch the captured [=display surface=]. The user agent MAY ignore this hint.
monitorTypeSurfaces of type {{MonitorTypeSurfacesEnum}}

If present, signals whether the application would like the user agent to include [=display surfaces=] whose type is [=display surface/monitor=] among the choices offered to the user. The user agent MAY ignore this hint.

The user agent may still offer to the user the option to capture [=display surfaces=] of type [=display surface/monitor=]. Applications are therefore encouraged to still check the {{MediaTrackSettings/displaySurface}} setting of the tracks they receive.

Extensions to {{MediaTrackSupportedConstraints}}

{{MediaTrackSupportedConstraints}} is extended here with the list of constraints that a user agent recognizes.

partial dictionary MediaTrackSupportedConstraints {
  boolean displaySurface = true;
  boolean logicalSurface = true;
  boolean cursor = true;
  boolean restrictOwnAudio = true;
  boolean suppressLocalAudioPlayback = true;
};
displaySurface of type {{boolean}}, defaulting to true

Whether {{displaySurface}} constraint is recognized.

logicalSurface of type {{boolean}}, defaulting to true

Whether {{logicalSurface}} constraint is recognized.

cursor of type {{boolean}}, defaulting to true

Whether {{cursor}} constraint is recognized.

restrictOwnAudio of type {{boolean}}, defaulting to true

Whether {{restrictOwnAudio}} constraint is recognized.

suppressLocalAudioPlayback of type {{boolean}}, defaulting to true

Whether {{suppressLocalAudioPlayback}} constraint is recognized.

Extensions to {{MediaTrackConstraintSet}}

{{MediaTrackConstraintSet}} is used for reading the current status of constraints.

partial dictionary MediaTrackConstraintSet {
  ConstrainDOMString displaySurface;
  ConstrainBoolean logicalSurface;
  ConstrainDOMString cursor;
  ConstrainBoolean restrictOwnAudio;
  ConstrainBoolean suppressLocalAudioPlayback;
};
displaySurface of type {{ConstrainDOMString}}

The type of [=display surface=] that is being captured. This assumes values from the {{DisplayCaptureSurfaceType}} enumeration.

logicalSurface of type {{ConstrainBoolean}}

A value of true indicates capture of a [=logical display surface=]; a value of false indicates a capture of a [=visible display surface=].

cursor of type {{ConstrainDOMString}}

Assumes values from the {{CursorCaptureConstraint}} enumeration that determines if and when the cursor is included in the captured display surface.

restrictOwnAudio of type {{ConstrainBoolean}}

This constraint is only applicable to audio tracks. See {{restrictOwnAudio}}.

suppressLocalAudioPlayback of type {{ConstrainBoolean}}

This constraint is only applicable to audio tracks. See {{suppressLocalAudioPlayback}}.

Extensions to {{MediaTrackSettings}}

When the {{MediaStreamTrack/getSettings()}} method is invoked on a video stream track, the user agent must return the extended {{MediaTrackSettings}} dictionary, representing the current status of the underlying user agent.

partial dictionary MediaTrackSettings {
  DOMString displaySurface;
  boolean logicalSurface;
  DOMString cursor;
  boolean restrictOwnAudio;
  boolean suppressLocalAudioPlayback;
};
displaySurface of type {{DOMString}}

The type of [=display surface=] that is being captured. This assumes values from the {{DisplayCaptureSurfaceType}} enumeration.

logicalSurface of type {{boolean}}

A value of true indicates capture of a logical display surface; a value of false indicates a capture capture of a visible display surface.

cursor of type {{DOMString}}

Assumes values from the {{CursorCaptureConstraint}} enumeration that determines if and when the cursor is included in the captured display surface.

restrictOwnAudio of type {{boolean}}

Indicates whether the restrictOwnAudio constraint is applied (true) or not (false).

suppressLocalAudioPlayback of type {{boolean}}

Indicates whether or not the application instructed the user agent to apply [=local audio playback suppression=] to the source.

Extensions to {{MediaTrackCapabilities}}

When the {{MediaStreamTrack/getCapabilities()}} method is invoked on a video stream track, the user agent must return the extended {{MediaTrackCapabilities}} dictionary, representing the capabilities of the underlying user agent.

partial dictionary MediaTrackCapabilities {
  DOMString displaySurface;
  boolean logicalSurface;
  sequence<DOMString> cursor;
};
displaySurface of type {{DOMString}}

MUST be the same value as is returned by {{MediaStreamTrack/getSettings()}}, rendering this property immutable from the application's viewpoint.

logicalSurface of type {{boolean}}

MUST be the same value as is returned by {{MediaStreamTrack/getSettings()}}, rendering this property immutable from the application's viewpoint.

cursor of type sequence<DOMString>

MUST consist of exactly the set of values from {{CursorCaptureConstraint}} that the user agent is capable of supporting for this track.

DisplayCaptureSurfaceType

The {{DisplayCaptureSurfaceType}} enumeration describes the different types of display surface.

enum DisplayCaptureSurfaceType {
  "monitor",
  "window",
  "browser"
};
Enum value Description
monitor a [=display surface/monitor=] [=display surface=], physical display, or collection of physical displays
window a [= display surface/window =] [=display surface=], or single application window
browser a [=display surface/browser=] [=display surface=], or single browser window

CursorCaptureConstraint

The {{CursorCaptureConstraint}} enumerates the conditions under which the cursor is captured.

enum CursorCaptureConstraint {
  "never",
  "always",
  "motion"
};
Enum value Description
never a {{CursorCaptureConstraint/"never"}} cursor capture constraint omits the cursor from the captured display surface.
always a {{CursorCaptureConstraint/"always"}} cursor capture constraint includes the cursor in the captured display surface.
motion a {{CursorCaptureConstraint/"motion"}} cursor capture constraint includes the cursor in the captured display surface when the cursor/pointer is moved. The captured cursor is removed when there is no further movement of the pointer/cursor for certain period of time, as determined by the user agent.

Device Identifiers

Each potential source of capture is treated by this API as a discrete media source. However, display capture sources MUST NOT be enumerated by {{MediaDevices/enumerateDevices()}}, since this would reveal too much information about the host system.

Display capture sources therefore cannot be selected with the {{MediaTrackSupportedConstraints/deviceId}} constraint, since their {{MediaDeviceInfo/deviceId}}s are not exposed.

This is not to be confused with the stable and private id of the same name used in algorithms to implement privacy indicators.

Permissions Integration

Screen Capture is a [=powerful feature=] which is identified by the [=powerful feature/name=] "display-capture", requiring [=express permission=] to be used.

As required for integration with the [[[Permissions]]] specification, this specification defines the following:

[=powerful feature/permission state constraints=]
Valid values for this descriptor's permission state are {{PermissionState/"prompt"}} and {{PermissionState/"denied"}}. The user agent MUST NOT ever set this descriptor's permission state to {{PermissionState/"granted"}}.

Permissions Policy Integration

This specification defines a [=policy-controlled feature=] identified by the string "display-capture". Its [=policy-controlled feature/default allowlist=] is "self".

A [=document=]'s [=Document/permissions policy=] determines whether any content in that document is allowed to use {{MediaDevices/getDisplayMedia}}. If disabled in any document, no content in the document will be [=allowed to use=] {{MediaDevices/getDisplayMedia}}. This is enforced by the [=prompt the user to choose=] algorithm.

Privacy Indicator Requirements

This specification extends the Privacy Indicator Requirements of {{MediaDevices/getUserMedia()}} to include {{MediaDevices/getDisplayMedia}}.

References in this specification to [[\devicesLiveMap]], [[\devicesAccessibleMap]], and [[\kindsAccessibleMap]] refer to the definitions already created to support Privacy Indicator Requirements for {{MediaDevices/getUserMedia()}}.

For each kind of device that {{MediaDevices/getDisplayMedia}} exposes, using a stable and private id for the device, deviceId, set kind to "Display" + kind, and do the following:

Then, given the new definitions above, the requirements on the user agent are those specified in Privacy Indicator Requirements of {{MediaDevices/getUserMedia()}}.

Even though there's a single permission descriptor for {{MediaDevices/getDisplayMedia}}, the above definitions distinguish by kind to enable user agents to implement privacy indicators that show the end-user the specific kinds of display sources that are being shared at any point.

Since this specification forbids user agents from persisting {{PermissionState/"granted"}} permissions, only the "Live" indicators are significant.

The user agent MUST NOT fire the devicechange event based on changes in the set of available sources from {{MediaDevices/getDisplayMedia}}.

Security and Permissions

This section is informative; however, it notes some serious risks to platform security if the advice it contains are not adhered to.

The risks to user privacy and security posed by capture of displayed content are twofold. The immediate and obvious risk is that users inadvertently share content that they did not wish to share, or might not have realized would be shared.

Display capture presents a less obvious risk to the cross site request forgery protections offered by the browser sandbox. Display and capture of information that is also under the control of an application, even indirectly, can allow that application to access information that would otherwise be inaccessible to it directly. For example, the canvas API does not permit sampling of a canvas, or conversion to an accessible form if it is not origin-clean [[2DCONTEXT]].

This issue is discussed in further detail in [[!RTCWEB-SECURITY-ARCH]] and [[!RTCWEB-SECURITY]].

Display capture that includes browser windows, particularly those that are under any form of control by the application, risks violation of these basic security protections. This risk is not entirely contained to browser windows, since control channels between browser applications and other applications, depending on the operating system. The key consideration is whether the captured display surface could be somehow induced to present information that would otherwise be secret from the application that is receiving the resulting media.

Capturing Logical or Visible Display Surfaces

Capture of logical display surfaces causes there to be a potential for content to be shared that a user is not made aware of. A logical display surface might render information that a user did not intend to expose. This can be more easily recognized if this information is visible. Such means are likely ineffectual against a machine, but a human recipient is less able to process content that appears only briefly.

It is encouraged that information that is not currently rendered to the screen be obscured in captures unless the application has been specifically authorized to access that content through elevated permissions.

How obscured areas of the logical display surface are captured to produce a visible display surface capture MAY vary. Some applications, like presentation software, benefit from having obscured portions of the screen render the image that appeared prior to being obscured. Freezing images can cause visual artifacts for changing content, or hide the fact that content is being obscured. Note that frozen portions of a capture can be incorrectly perceived as a bug. Alternatively, obscured areas might be replaced with content that marks them as being obscured, such as a grey color or hatching.

Some systems may only capture the logical display surface. Devices with small screens, for instance, do not typically have the concept of a [= display surface/window =], and render applications in full screen modes only. These systems might provide a capture of an application that is not currently visible, which could be unusable without capturing the logical display surface.

When capturing a [= display surface/window =] or other display surface that is partially transparent, any content behind it will not be captured.

There is a risk that the user prompt be exposed to the web page for a short amount of time by the newly created {{MediaStreamTrack}}, for instance if the user selects the screen on which the user prompt is displayed. In the case of the user prompt displaying previews of the various surfaces available for selection, those previews will not be captured by the newly created {{MediaStreamTrack}}.

Capturing Audio

{{MediaDevices/getDisplayMedia}} allows capturing audio alongside video, this poses privacy and security concern as this may expose additional information about system applications, and the set of shared audio sources are not necessarily the same as the set of shared video sources. For example, the capture of the video of a [= display surface/window =] that is accompanied by the audio of the entire system, including applications unrelated to that window, will not be shared without active user consent. It is important that the user is aware of what content will be shared, including any possible audio. It is strongly encouraged that the user is allowed to give consent to video but not audio, resulting in a video-only stream. This ensures that the request for audio is always optional and does not restrict the user's choices compared to a video-only request.

Authorizing Display Capture

This document encourages implementations to provide additional limitations on the mechanisms used to affirm user consent. These limitations are designed to mitigate the security and privacy risks that the API poses.

Two forms of consent interaction are described: active user consent and a range of elevated permissions. These are non-normative recommandations only.

Active User Consent

Active user consent is sufficient where there is little or no risk of an application gaining information that the user did not intend to share. These cases can be identified by those where the application that requests capture has no control over what is rendered to the captured display surface.

To prevent applications from limiting the available choices presented to a user with the goal of promoting a particular choice, the {{MediaDevices/getDisplayMedia}} API does not permit the use of constraints to narrow the set of options presented.

Elevated Permissions

It is strongly advised that elevated permissions be required to access any display surface that might be used to circumvent cross-origin protections for content. The key goal of this consent process is not just to demonstrate that a user intends to share content, but to also to determine that the user exhibits an elevated level of trust in the application that is being granted access.

Several different controls might be provided to grant elevated permissions. This section describes several different capabilities that could be independently granted. A user agent might opt to prohibit access to any capability that requires elevated permissions.

If access to these surfaces is supported, it is strongly advised that any mechanism to acquire elevated permissions not rely solely on simple prompts for user consent. Any action needs to ensure that a decision to authorize an application with elevated privileges is deliberate. For instance, a user agent might require a process equivalent to software installation to signify that user consent for elevated permissions is granted.

An elevated permissions experience could allow the user agent to communicate the risks associated with enabling this feature, or at least to convey the need for augmented trust in the application.

Note that elevated permissions are not a substitute for active user consent. It is advised that user agents still present users with the ability to select what is shared, even for applications that have elevated permissions.

Capabilities Depending on Elevated Permissions

Elevated permissions are encouraged as a prerequisite for access to capture of [= display surface/monitor =] or [= display surface/browser =] display surfaces. Note that capture of a complete [= display surface/monitor =] is included because this could include a window from the user agent.

Similarly, elevated permissions are an encouraged prerequisite for access to logical display surfaces, where that would not ordinarily be provided.

It is encouraged that elevated permissions that are granted to an origin be persisted. An elevated permissions process in part relies on its novelty to ensure that it correctly captures user intent.

Feedback and Interface During Capture

Implementations are advised to provide user feedback and control mechanisms similar to those offered users when sharing a camera or microphone, as encouraged in [[GETUSERMEDIA]].

It is important that a user be aware that content is being shared when content is actively being captured. User agents are advised to display a prominent indicator while content is being captured. In addition to an indicator, a user agent is advised to provide a means to learn precisely what is being shared; while this capability is trivially provided by an application by rendering the captured content, this information allows a user to accurately assess what is being shared.

In addition to feedback mechanisms, a means to for the user to stop any active capture is advisable.