A {{MediaStreamTrack}} is a transferable object. This allows manipulating real-time media outside the context it was requested or created in, for instance in workers or third-party iframes.

To preserve the existing privacy and security infrastructure, in particular for capture tracks, the track source lifetime management remains tied to the context that created it. The transfer algorithm MUST ensure the following behaviors:

1. The context named originalContext that created a track named originalTrack remains in control of the originalTrack source, named trackSource, even when originalTrack is transferred into transferredTrack.

2. In particular, originalContext remains the proxy to privacy indicators of trackSource. transferredTrack or any of its clones are considered as tracks using trackSource as if they were tracks created in and controlled by originalContext.

3. When originalContext goes away, trackSource gets ended, thus transferredTrack gets ended.

4. When originalContext would have muted/unmuted originalTrack, transferredTrack gets muted/unmuted.

5. If transferredTrack is cloned in transferredTrackClone, transferredTrackClone is tied to trackSource. It is not tied to originalTrack in any way.

6. If transferredTrack is transferred into transferredAgainTrack, transferredAgainTrack is tied to trackSource. It is not tied to transferredTrack or originalTrack in any way.

The WebIDL changes to make the track transferable are the following:

[Exposed=(Window,Worker), Transferable]
partial interface MediaStreamTrack {
};

At creation of a {{MediaStreamTrack}} object, called track, run the following steps:

1. Initialize track.`[[IsDetached]]` to false.

The {{MediaStreamTrack}} transfer steps, given value and dataHolder, are:

1. If value.`[[IsDetached]]` is true, throw a "DataCloneError" DOMException.

2. Set dataHolder.`[[id]]` to value.{{MediaStreamTrack/id}}.

3. Set dataHolder.`[[kind]]` to value.{{MediaStreamTrack/kind}}.

4. Set dataHolder.`[[label]]` to value.{{MediaStreamTrack/label}}.

5. Set dataHolder.`[[readyState]]` to value.{{MediaStreamTrack/readyState}}.

6. Set dataHolder.`[[enabled]]` to value.{{MediaStreamTrack/enabled}}.

7. Set dataHolder.`[[muted]]` to value.{{MediaStreamTrack/muted}}.

8. Set dataHolder.`[[source]]` to value underlying source.

9. Set dataHolder.`[[constraints]]` to value active constraints.

10. Set dataHolder.`[[contentHint]]` to value application-set content hint.

11. Set value.`[[IsDetached]]` to true.

12. Set value.[[\ReadyState]] to {{MediaStreamTrackState/"ended"}} (without stopping the underlying source or firing an `ended` event).

{{MediaStreamTrack}} transfer-receiving steps, given dataHolder and track, are:

1. Initialize track.{{MediaStreamTrack/id}} to dataHolder.`[[id]]`.

2. Initialize track.{{MediaStreamTrack/kind}} to dataHolder.`[[kind]]`.

3. Initialize track.{{MediaStreamTrack/label}} to dataHolder.`[[label]]`.

4. Initialize track.{{MediaStreamTrack/readyState}} to dataHolder.`[[readyState]]`.

5. Initialize track.{{MediaStreamTrack/enabled}} to dataHolder.`[[enabled]]`.

6. Initialize track.{{MediaStreamTrack/muted}} to dataHolder.`[[muted]]`.

7. Set track application-set content hint to dataHolder.`[[contentHint]]`.

8. [=Initialize the underlying source=] of track to dataHolder.`[[source]]`.

9. Set track's constraints to dataHolder.`[[constraints]]`.

The underlying source is supposed to be kept alive between the transfer and transfer-receiving steps, or as long as the data holder is alive. In a sense, between these steps, the data holder is attached to the underlying source as if it was a track.

On microphone audio tracks, frame counters allow the application to tell the ratio of audio that is delivered as one quality indicator and the latency metrics measure the input delay from capture to application.

On camera and screenshare video tracks, frame counters allow the application to tell what the frame rate is, which may be lower than the target {{MediaTrackSettings/frameRate}}. For example, if the track is sourced from a camera then the production of frames could be slowed down if it's dark or frames could be dropped if the system is CPU starved. This could impact the total number of frames produced by the source and impact how many frames are delivered, discarded or dropped for other reasons.

partial interface MediaStreamTrack {
  [SameObject] readonly attribute
      (MediaStreamTrackAudioStats or MediaStreamTrackVideoStats)? stats;
};

Let the {{MediaStreamTrack}} have a [[\Stats]] internal slot initialized it to null, unless otherwise specified below.

If the track's is of {{MediaStreamTrack/kind}} "audio", run the following steps:

1. If the {{MediaStreamTrack}} is sourced from getUserMedia(), initialize {{MediaStreamTrack/[[Stats]]}} to a new instance of {{MediaStreamTrackAudioStats}} set up to expose audio stats for this {{MediaStreamTrack}}.

If the track's is of {{MediaStreamTrack/kind}} "video", run the following steps:

1. If the {{MediaStreamTrack}} is sourced from getUserMedia() or getDisplayMedia(), initialize {{MediaStreamTrack/[[Stats]]}} to a new instance of {{MediaStreamTrackVideoStats}} set up to expose video stats for this {{MediaStreamTrack}}.

[Exposed=Window]
interface MediaStreamTrackAudioStats {
  readonly attribute unsigned long long deliveredFrames;
  readonly attribute DOMHighResTimeStamp deliveredFramesDuration;
  readonly attribute unsigned long long totalFrames;
  readonly attribute DOMHighResTimeStamp totalFramesDuration;
  readonly attribute DOMHighResTimeStamp latency;
  readonly attribute DOMHighResTimeStamp averageLatency;
  readonly attribute DOMHighResTimeStamp minimumLatency;
  readonly attribute DOMHighResTimeStamp maximumLatency;
  undefined resetLatency();
  [Default] object toJSON();
};
        

[Exposed=Window]
interface MediaStreamTrackVideoStats {
  readonly attribute unsigned long long deliveredFrames;
  readonly attribute unsigned long long discardedFrames;
  readonly attribute unsigned long long totalFrames;
  [Default] object toJSON();
};
        

Some platforms or User Agents may provide built-in support for video effects triggered by user motion heuristics, in particular for camera video streams. Web applications may either want to control or at least be aware that these heuristics are active and might trigger these effects at the source level. This can for instance allow the web application to update its UI or to turn off these heuristics where having such effects tiggered accidentally might be considered insensitive or inappropriate. For that reason, we extend {{MediaStreamTrack}} with the following properties.

The WebIDL changes are the following:

partial dictionary MediaTrackSupportedConstraints {
  boolean gestureReactions = true;
};

partial dictionary MediaTrackConstraintSet {
  ConstrainBoolean gestureReactions;
};

partial dictionary MediaTrackSettings {
  boolean gestureReactions;
};

partial dictionary MediaTrackCapabilities {
  sequence<boolean> gestureReactions;
};

Some platforms offer functionality for voice isolation: Attempting to remove all parts of an audio track that do not correspond to a human voice. Some platforms even attempt to remove extraneous voices, leaving the "main voice" as the dominant component of the audio. The exact methods used may vary between implementations.

This constraint permits the platform to turn on that functionality, with the desired result being that the "main voice" in the audio signal is the dominant component of the audio.

This will have large effects on audio that is presented for other reasons than to transmit voice (for instance music or ambient noises), so needs to be off by default.

This constraint is a stronger version of noise cancellation, which means that if the "noiseSuppression" constraint is set to false and "voiceIsolation" is set to true, the value of "noiseCancellation" will be ignored.

This constraint has no such relationship with any other constraint; in particular it does not affect echoCancellation.

The WebIDL changes are the following:

partial dictionary MediaTrackSupportedConstraints {
  boolean voiceIsolation = true;
};

partial dictionary MediaTrackConstraintSet {
  ConstrainBoolean voiceIsolation;
};

partial dictionary MediaTrackSettings {
  boolean voiceIsolation;
};

partial dictionary MediaTrackCapabilities {
  sequence<boolean> voiceIsolation;
};
        

Processing considerations

When the "voiceIsolation" setting is set to true by the ApplyConstraints algorithm, the UA will attempt to remove the components of the audio track that do not correspond to a human voice. If a dominant voice can be identified, the UA will attempt to enhance that voice.

When the "voiceIsolation" constraint setting is set to false by the ApplyConstraints algorithm, the UA will process the audio according to other settings in its normal fashion.

The configuration (capabilities and settings) of a {{MediaStreamTrack}} may be changed dynamically outside the control of web applications. One example is when a user decides to switch on background blur through the operating system. Web applications might want to know that the configuration of a particular {{MediaStreamTrack}} has changed. For that purpose, a new event is defined below.

partial interface MediaStreamTrack {
  attribute EventHandler onconfigurationchange;
};

The onconfigurationchange attribute is an [=event handler IDL attribute=] for the `onconfigurationchange` [=event handler=], whose [=event handler event type=] is configurationchange.

When the [=User Agent=] detects a change of configuration in a track's underlying source, the [=User Agent=] MUST run the following steps:

1. If track.{{MediaStreamTrack/muted}} is true, wait for track.{{MediaStreamTrack/muted}} to become false or track.{{MediaStreamTrack/readyState}} to be "ended".

2. [=Queue a task=] on [=current settings object=]'s [=environment settings object/responsible event loop=] to perform the following steps:

   This task will run before any other task that may set track.{{MediaStreamTrack/muted}} to true.

   1. If track.{{MediaStreamTrack/readyState}} is "ended", abort these steps.

   2. If track's capabilities and settings are matching source configuration, abort these steps.

   3. Update track's capabilities and settings according track's underlying source.

   4. [=Fire an event=] named {{configurationchange}} on track.

This example shows how to monitor external background blur changes.

      const stream = await navigator.mediaDevices.getUserMedia({video: true});
      const [track] = stream.getVideoTracks();
      let {backgroundBlur} = track.getSettings();
      applyBlurInSoftwareInstead(!backgroundBlur);

      track.addEventListener("configurationchange", () => {
        if (backgroundBlur != track.getSettings().backgroundBlur) {
          backgroundBlur = track.getSettings().backgroundBlur;
          applyBlurInSoftwareInstead(!backgroundBlur);
        }
      });
    

{{MediaStreamTrack}}'s objects are exposed to workers, so can do {{MediaStream}}'s objects.

[Exposed=(Window,Worker)]
partial interface MediaStream {
};