MediaDevices Additions

partial interface MediaDevices {
  Promise<MediaStream> getDisplayMedia(optional DisplayMediaStreamOptions options = {});
};

getDisplayMedia

Prompts the user for permission to live-capture their display.

The user agent MUST let the end-user choose which [=display surface=] to share out of all available choices every time, and MUST NOT use any {{MediaTrackConstraints}} in options.video or options.audio to limit that choice.

The user agent MAY use the presence of the {{displaySurface}} constraint and its value to influence the presentation to the user of the sources to pick from. The user agent MUST still offer the user unlimited choice of any [=display surface=]. The user agent is strongly recommended to steer users away from sharing a monitor, as this poses risks to user privacy.

Any {{MediaTrackConstraints}} in options.video or options.audio MUST be applied to the media chosen by the user only after the user has made their selection.

In the case of audio, the user agent MAY present the end-user with audio sources to share. Which choices are available to choose from is up to the user agent, and the audio source(s) are not necessarily the same as the video source(s). An audio source may be a particular [= display surface/window =], [= display surface/browser =], the entire system audio or any combination thereof. Unlike {{MediaDevices/getUserMedia()}} with regards to audio+video, the user agent is allowed not to return audio even if the audio constraint is present. If the user agent knows no audio will be shared for the lifetime of the stream it MUST NOT include an audio track in the resulting stream. The user agent MAY accept a request for audio and video by only returning a video track in the resulting stream, or it MAY accept the request by returning both an audio track and a video track in the resulting stream. The user agent MUST reject audio-only requests.

In addition to drawing from a different set of sources and requiring user selection, {{MediaDevices/getDisplayMedia}} also differs from {{MediaDevices/getUserMedia()}} in that {{PermissionState/"granted"}} permissions cannot be persisted.

When the {{MediaDevices/getDisplayMedia()}} method is called, the user agent MUST run the following steps:

1. Let mediaDevices be [=this=].

2. Let controller be options.controller if present, or null otherwise.

3. If controller is not null, run the following steps:

   1. If controller.{{CaptureController/[[IsBound]]}} is true, return a promise [=reject|rejected=] with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

   2. Set controller.{{CaptureController/[[IsBound]]}} to true.

4. If the [=relevant global object=] of [=this=] does not have [=transient activation=], return a promise rejected with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

5. Let options be the method's first argument.

6. Let constraints be [options.audio, options.video].

7. If constraints.video is false, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

8. For each [= map/exist | existing =] member in constraints whose value, CS, is a dictionary, run the following steps:

   1. If CS contains a member named advanced, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

   2. If CS contains a member whose name specifies a constrainable property applicable to [=display surface=]s, and whose value in turn is a dictionary containing a member named either min or exact, return a promise [=reject|rejected=] with a newly [=exception/created=] {{TypeError}}.

   3. If CS contains a member whose name specifies a constrainable property applicable to [=display surface=]s, and whose value in turn is a dictionary containing a member named max, and that member's value in turn is less than the constrainable property's [=floor value=], then let failedConstraint be the name of the member, let message be either undefined or an informative human-readable message, and return a promise [=reject|rejected=] with a new OverconstrainedError created by calling OverconstrainedError(failedConstraint, message).

9. Let requestedMediaTypes be the set of media types in constraints with either a dictionary value or a value of true.

10. If the current settings object's [=relevant global object=]'s [=associated `Document`=] is NOT [=Document/fully active=] or does NOT have focus, return a promise [=reject|rejected=] with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{InvalidStateError}}.

11. Let p be a new promise.

12. Run the following steps [=in parallel=]:

    1. For each media type T in requestedMediaTypes,

       1. If no sources of type T are available, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotFoundError}}.

       2. Read the current [= permission state=] for obtaining sources of type T in the current browsing context. If the permission state is {{PermissionState/"denied"}}, jump to the step labeled PermissionFailure below.

    2. Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled Permission Failure below.

    3. [=Prompt the user to choose=] a display device, for a {{PermissionDescriptor}} with its {{PermissionDescriptor/name}} set to "display-capture", resulting in a set of provided media.

       The provided media MUST include precisely one video track.

       The provided media MUST include at most one audio track. This audio track MUST NOT be included if audio was not specified in requestedMediaTypes, or if it was specified as false.

       The devices chosen MUST be the ones determined by the user. Once selected, the source of a {{MediaStreamTrack}} MUST NOT change, unless the user permits it through their interaction with the user agent.

       User agents are encouraged to warn users against sharing [= display surface/browser =] display devices as well as [= display surface/monitor =] display devices where browser windows are visible, or otherwise try to discourage their selection on the basis that these represent a significantly higher risk when shared.

       If the result of the request is {{PermissionState/"granted"}}, then for each device that is sourcing the provided media, using a stable and private id for the device, deviceId, set [[\devicesLiveMap]][deviceId] to true, if it isn’t already true, and set the [[\devicesAccessibleMap]][deviceId] to true, if it isn’t already true.

       The user agent MUST NOT store a {{PermissionState/"granted"}} permission entry.

       If the result is {{PermissionState/"denied"}}, jump to the step labeled Permission Failure below. If the user never responds, this algorithm stalls on this step.

       If the user grants permission but a hardware error such as an OS/program/webpage lock prevents access, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotReadableError}} and abort these steps.

       If the result is {{PermissionState/"granted"}} but device access fails for any reason other than those listed above, reject p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{AbortError}} and abort these steps.

    4. Let stream be a {{MediaStream}} object.

    5. For each source that the user granted permission to, run the following steps:

       1. Let track be the result of [=create a MediaStreamTrack|creating a MediaStreamTrack=] with source and mediaDevices.

       2. Add track to stream's track set.

       3. [=Tie track source to MediaDevices=] with source and mediaDevices.

    6. Run the [=ApplyConstraints algorithm=] on all tracks in stream with the appropriate constraints. Should this fail, let failedConstraint be the result of the algorithm that failed, and let message be either undefined or an informative human-readable message, and then reject p with a new OverconstrainedError created by calling OverconstrainedError(failedConstraint, message).

    7. This invocation of {{MediaDevices/getDisplayMedia()}} is now considered to have produced a new capture-session.

    8. If controller is not null, run the following steps:

       1. Set controller.{{CaptureController/[[Source]]}} to stream's video track's [[\Source]].

       2. Set controller.{{CaptureController/[[DisplaySurfaceType]]}} to the to stream's video track's {{DisplayCaptureSurfaceType}}.

       3. Queue a task to run the [=finalize focus decision algorithm=] on controller.

    9. Resolve p with stream and abort these steps.

    10. Permission Failure: [=Reject=] p with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{NotAllowedError}}.

13. Return p.

When the top-level document loses focus, run the following steps on all {{CaptureController}} objects in that document and in documents of its nested [=browsing contexts=]:

1. If {{CaptureController/[[Source]]}} is undefined, abort these steps.

2. Set {{CaptureController/[[FocusChangeDisabled]]}} to true.

The user agent MUST NOT capture content that's behind a partially transparent captured display surface.

For the newly created {{MediaStreamTrack}}, the user agent MUST NOT capture the prompt that was shown to the user.

Information that is not currently rendered to the screen SHOULD be obscured in captures unless the application has been specifically authorized to access that content (e.g. through means such as elevated permissions).

The user agent MUST NOT share audio without active user consent, for example if the capture of the video of a [= display surface/window =] is accompanied by capture of the audio of the entire system, including applications unrelated to that window.

Closed and Minimized Display Surfaces

A display surface that is being shared may temporarily or permanently become inaccessible to the application because of actions taken by the operating system or user agent. What makes a display surface considered inaccesible is outside the scope of this specification, but examples MAY include a [= display surface/monitor =] disconnecting, [= display surface/window =] or [= display surface/browser =] closing or becoming minimized, or due to an incoming call on a phone.

User agents ultimately control what inaccesible means in this context, but are encouraged to only fire mute and unmute events for interruptions that have external reasons.

When display surface enters an inaccessible state that is not necessarily permanent, the user agent MUST queue a task that [= MediaStreamTrack/set a track's muted state|sets the muted state =] of the corresponding media track to true.

When display surface exits an inaccessible state and becomes accessible, the user agent MUST queue a task that [= MediaStreamTrack/set a track's muted state|sets the muted state =] of the corresponding media track to false.

When a display surface enters an inaccessible state that is permanent (such as the source [= display surface/window=] closing), the user agent MUST queue a task that [= MediaStreamTrack/ended | ends =] the corresponding media track.

A stream that was just returned by {{MediaDevices/getDisplayMedia}} MAY contain tracks that are muted by default. Audio and video tracks belonging to the same stream MAY be muted/unmuted independently of one another.

Unconstrained Display Surface Selection

Not accepting constraints for source selection means that {{MediaDevices/getDisplayMedia}} only provides fingerprinting surface that exposes whether audio, video or audio and video display sources are present.

Note that accepting the {{displaySurface}} constraint does not limit user selection.

Constrainable Properties for Captured Display Surfaces

Constraints serve a different purpose in {{MediaDevices/getDisplayMedia}} than they do in {{MediaDevices/getUserMedia()}}. They do not aid discovery, instead they are applied only after user-selection.

This section define which constraints apply to {{MediaDevices/getDisplayMedia}} tracks; constraints defined for {{MediaDevices/getUserMedia()}} do not apply unless listed here.

Some of these constraints enable user agent processing like downscaling and frame decimation, as well as display-specific features. Others enable observation of inherent properties of a user-selected display surface, as capabilities and settings.

The following new and existing {{MediaStreamTrack}} Constrainable Properties are defined to apply to the user-selected video display surface, with the following behavior:

The following new and existing {{MediaStreamTrack}} Constrainable Properties are defined to apply to the user-selected audio sources, with the following behavior:

When inherent properties of the underlying source of a user-selected display surface change, for example in response to the end-user resizing a captured window, and these changes render the capabilities and/or settings of one or more constrainable properties outdated, the user agent MUST queue a task to run the following step:

1. Update all affected constrainable properties at the same time.

   If this causes an "overconstrained" situation, then the user agent MUST ignore the culprit constraints for as long as they overconstrain. The user agent MUST NOT mute the track.

            enum CaptureStartFocusBehavior {
              "focus-capturing-application",
              "focus-captured-surface",
              "no-focus-change"
            };    
          

            [Exposed=Window, SecureContext]
            interface CaptureController : EventTarget {
              constructor();
              undefined setFocusBehavior(CaptureStartFocusBehavior focusBehavior);
            };
          

            enum SelfCapturePreferenceEnum {
              "include",
              "exclude"
            };
          

            enum SystemAudioPreferenceEnum {
              "include",
              "exclude"
            };
          

            enum SurfaceSwitchingPreferenceEnum {
              "include",
              "exclude"
            };
          

            enum MonitorTypeSurfacesEnum {
              "include",
              "exclude"
            };
          

dictionary DisplayMediaStreamOptions {
  (boolean or MediaTrackConstraints) video = true;
  (boolean or MediaTrackConstraints) audio = false;
  CaptureController controller;
  SelfCapturePreferenceEnum selfBrowserSurface;
  SystemAudioPreferenceEnum systemAudio;
  SurfaceSwitchingPreferenceEnum surfaceSwitching;
  MonitorTypeSurfacesEnum monitorTypeSurfaces;
};

partial dictionary MediaTrackSupportedConstraints {
  boolean displaySurface = true;
  boolean logicalSurface = true;
  boolean cursor = true;
  boolean restrictOwnAudio = true;
  boolean suppressLocalAudioPlayback = true;
};

partial dictionary MediaTrackConstraintSet {
  ConstrainDOMString displaySurface;
  ConstrainBoolean logicalSurface;
  ConstrainDOMString cursor;
  ConstrainBoolean restrictOwnAudio;
  ConstrainBoolean suppressLocalAudioPlayback;
};

partial dictionary MediaTrackSettings {
  DOMString displaySurface;
  boolean logicalSurface;
  DOMString cursor;
  boolean restrictOwnAudio;
  boolean suppressLocalAudioPlayback;
};

partial dictionary MediaTrackCapabilities {
  DOMString displaySurface;
  boolean logicalSurface;
  sequence<DOMString> cursor;
};

enum DisplayCaptureSurfaceType {
  "monitor",
  "window",
  "browser"
};

enum CursorCaptureConstraint {
  "never",
  "always",
  "motion"
};

Device Identifiers

Each potential source of capture is treated by this API as a discrete media source. However, display capture sources MUST NOT be enumerated by {{MediaDevices/enumerateDevices()}}, since this would reveal too much information about the host system.

Display capture sources therefore cannot be selected with the {{MediaTrackSupportedConstraints/deviceId}} constraint, since their {{MediaDeviceInfo/deviceId}}s are not exposed.

Capturing Logical or Visible Display Surfaces

Capture of logical display surfaces causes there to be a potential for content to be shared that a user is not made aware of. A logical display surface might render information that a user did not intend to expose. This can be more easily recognized if this information is visible. Such means are likely ineffectual against a machine, but a human recipient is less able to process content that appears only briefly.

It is encouraged that information that is not currently rendered to the screen be obscured in captures unless the application has been specifically authorized to access that content through elevated permissions.

How obscured areas of the logical display surface are captured to produce a visible display surface capture MAY vary. Some applications, like presentation software, benefit from having obscured portions of the screen render the image that appeared prior to being obscured. Freezing images can cause visual artifacts for changing content, or hide the fact that content is being obscured. Note that frozen portions of a capture can be incorrectly perceived as a bug. Alternatively, obscured areas might be replaced with content that marks them as being obscured, such as a grey color or hatching.

Some systems may only capture the logical display surface. Devices with small screens, for instance, do not typically have the concept of a [= display surface/window =], and render applications in full screen modes only. These systems might provide a capture of an application that is not currently visible, which could be unusable without capturing the logical display surface.

When capturing a [= display surface/window =] or other display surface that is partially transparent, any content behind it will not be captured.

There is a risk that the user prompt be exposed to the web page for a short amount of time by the newly created {{MediaStreamTrack}}, for instance if the user selects the screen on which the user prompt is displayed. In the case of the user prompt displaying previews of the various surfaces available for selection, those previews will not be captured by the newly created {{MediaStreamTrack}}.

Capturing Audio

{{MediaDevices/getDisplayMedia}} allows capturing audio alongside video, this poses privacy and security concern as this may expose additional information about system applications, and the set of shared audio sources are not necessarily the same as the set of shared video sources. For example, the capture of the video of a [= display surface/window =] that is accompanied by the audio of the entire system, including applications unrelated to that window, will not be shared without active user consent. It is important that the user is aware of what content will be shared, including any possible audio. It is strongly encouraged that the user is allowed to give consent to video but not audio, resulting in a video-only stream. This ensures that the request for audio is always optional and does not restrict the user's choices compared to a video-only request.

Authorizing Display Capture

This document encourages implementations to provide additional limitations on the mechanisms used to affirm user consent. These limitations are designed to mitigate the security and privacy risks that the API poses.

Two forms of consent interaction are described: active user consent and a range of elevated permissions. These are non-normative recommandations only.

Feedback and Interface During Capture

Implementations are advised to provide user feedback and control mechanisms similar to those offered users when sharing a camera or microphone, as encouraged in [[GETUSERMEDIA]].

It is important that a user be aware that content is being shared when content is actively being captured. User agents are advised to display a prominent indicator while content is being captured. In addition to an indicator, a user agent is advised to provide a means to learn precisely what is being shared; while this capability is trivially provided by an application by rendering the captured content, this information allows a user to accurately assess what is being shared.

In addition to feedback mechanisms, a means to for the user to stop any active capture is advisable.