CropTarget Motivation

The [=cropping mechanism=] presented in this document ({{BrowserCaptureMediaStreamTrack/cropTo}}) relies on [=Crop-session Target=] rather than on direct node references. This serves a dual purpose.

• It allows cropping by one document to coordinates specified in another document.

• [=Tagging=] an {{Element}} as a potential crop-target allows the user agent to avoid unnecessary work on all other elements, like the calculation of bounding boxes and sending such coordinates cross-process.

CropTarget Definition

CropTarget is an intentionally empty, opaque identifier that exposes nothing. Its sole purpose is to be handed to {{BrowserCaptureMediaStreamTrack/cropTo}} as input.

          [Exposed=(Window,Worker), Serializable]
          interface CropTarget {
            // Intentionally empty; just an opaque identifier.
          };
        

To create a CropTarget with element as input, run the following steps:

1. Let cropTarget be a new object of type {{CropTarget}}.

2. Let weakRef be a weak reference to element.

   [=Create a CropTarget|Create=] cropTarget.[[\Element]] initialized to weakRef.

   cropTarget keeps a weak reference to the element it represents. In other words, cropTarget will not prevent garbage collection of its element.

{{CropTarget}} objects are serializable. The [=serialization steps=], given value, serialized, and a boolean forStorage, are:

1. If forStorage is true, throw with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"DataCloneError"}}.

2. Set serialized.[[\CropTargetElement]] to value.[[\Element]].

The [=deserialization steps=], given serialized and value are:

1. Set value.[[\Element]] to serialized.[[\CropTargetElement]].

MediaDevices.produceCropTarget

          partial interface MediaDevices {
            Promise<CropTarget>
            produceCropTarget(Element element);
          };
        

produceCropTarget()

Calling {{MediaDevices/produceCropTarget}} on an {{Element}} of a supported type associates that {{Element}} with a {{CropTarget}}. This {{CropTarget}} may be used as input to {{BrowserCaptureMediaStreamTrack/cropTo}}. We define a valid CropTarget as one returned by a previous call to {{MediaDevices/produceCropTarget()}} in the current [=top-level browsing context=] or any of its descendant browsing contexts.

When {{MediaDevices/produceCropTarget}} is called on a given element, the user agent [=create a CropTarget|creates a CropTarget=] with element as input. The user agent MUST return a {{Promise}} p. The user agent MUST resolve p only after it has finished all the necessary internal propagation of state associated with the new {{CropTarget}}, at which point the user agent MUST be ready to receive the new {{CropTarget}} as a valid parameter to {{BrowserCaptureMediaStreamTrack/cropTo}}.

When cloning an {{Element}} on which {{MediaDevices/produceCropTarget}} was previously called, the clone is not associated with any {{CropTarget}}. If {{MediaDevices/produceCropTarget}} is later called on the clone, a new {{CropTarget}} will be assigned to it.

BrowserCaptureMediaStreamTrack

Recall that, as per [[SCREEN-CAPTURE]], when {{MediaDevices/getDisplayMedia()}} is called, it returns a {{Promise}}<{{MediaStream}}>, and that this {{MediaStream}} contains exactly one video track, whose type is {{MediaStreamTrack}}.

We specify that if the user chooses to capture a [=browser=] [=display-surface=], the user agent MUST instantiate the video track as either {{MediaStreamTrack}}, or as some sub-class of {{MediaStreamTrack}}, and that {{BrowserCaptureMediaStreamTrack/cropTo}} MUST be exposed on this track. For simplicity's sake, this document assumes that a subclass called {{BrowserCaptureMediaStreamTrack}} is used by the user agent.

The track MUST be initially [=uncropped=].

          [Exposed = Window]
          interface BrowserCaptureMediaStreamTrack : MediaStreamTrack {
            Promise<undefined> cropTo(CropTarget? cropTarget);
            BrowserCaptureMediaStreamTrack clone();
          };
        

cropTo()

Calls to this method instruct the user agent to start/stop cropping a [=self-capture video track=] to the bounding client rectangle of cropTarget.[[\Element]]. Since the track is restricted to the visible viewport of the [=display-surface=], the captured area will be the intersection of the visible viewport and the element bounding client rectangle. Whenever {{BrowserCaptureMediaStreamTrack/cropTo}} is invoked, the user agent MUST execute the following algorithm:

1. If [=this=] is not a [=self-capture video track=], the user agent MUST return a new {{Promise}}, [=rejected=] with an {{NotSupportedError}}.

2. The user agent MUST validate cropTarget according to [=this=] track's current [=crop-state=].

   • If [=this=] track is [=uncropped=], the user agent MUST only accept [=valid CropTargets=].
   • If [=this=] track is [=cropped=], the user agent MUST accept either [=valid CropTargets=] or {{undefined}}.

   If the user agent does not accept cropTarget, return a {{Promise}} [=rejected=] with an {{UnknownError}}.

3. Let p be a new {{Promise}}.

4. Run the following steps in parallel:

   1. If cropTarget is neither {{undefined}} nor a [=valid CropTarget=], reject p with a {{NotAllowedError}} and abort these steps.

   2. If cropTarget is either {{undefined}} or a [=valid CropTarget=], the user agent MUST update [=this=] video track's [=crop-state=] according to cropTarget:

      • If cropTarget is set to {{undefined}}, the user agent MUST stop cropping. [=This=] video track reverts to the [=uncropped=] state.
      • If cropTarget is a [=valid CropTarget=], the user agent MUST start cropping [=this=] video track to the contours of the element referenced by this {{CropTarget}}. This means that for each new frame produced on the track, the user agent calculates the bounding box of the pixels belonging to the element, and crops the frame to the coordinates of this bounding box.

   3. Call the track's state before this method invocation PRE-STATE, and after this method invocation POST-STATE. The user agent MUST resolve p when it is guaranteed that no more frames cropped (or uncropped) according to PRE-STATE have been delivered to the application, and that any additional frames delivered to the application will therefore be cropped (or uncropped) according to either POST-STATE or a later state.

      The timing of the cropTo promise resolution and the timing of the actual cropping of video frames is observable to JavaScript through MediaStreamTrack transforms. It is expected that the first newly cropped video frame will be enqueued on the MediaStreamTrack ReadableStream just after the cropTo promise is resolved.

5. Return p.

clone()

When a {{BrowserCaptureMediaStreamTrack}} is cloned, the user agent MUST produce a track which is initially [=uncropped=], regardless of the [=crop-state=] of the original track.

Crop-Session Lifetime