This document defines a set of extension commands to the WebDriver specification for controlling mock capture devices and access rules to these devices.

This document is not complete. It is subject to major changes and, while early experimentations are encouraged, it is therefore not intended for implementation.

Introduction

This document defines a set of extension commands to the [[WebDriver2]] specification for controlling mock capture devices and access rules to these devices.

This specification defines conformance criteria that apply to a single product: the User Agent that implements the interfaces that it contains.

Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. (In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.)

Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[WEBIDL]], as this specification uses that specification and terminology.

Media Capture Permissions

Control prompt results

A [= session | WebDriver session =] handles two prompt result values:

  1. A getUserMedia prompt result which is either "granted" or "denied". This value is used when {{MediaDevices.getUserMedia()}} is executing the request permission to use step. If the [=permission state=] is {{PermissionState/"prompt"}}, the request permission to use step will return the value of the [=getUserMedia prompt result=]. The default value of [=getUserMedia prompt result=] is "granted".

  2. A getDisplayMedia prompt result which is either "granted" or "denied". This value is used when {{MediaDevices.getDisplayMedia()}} is executing the prompt the user to choose step. If the [= permission state =] is {{PermissionState/"prompt"}}, the prompt the user to choose step will return:

    • "denied" if the value of the [= getDisplayMedia prompt result =] is "denied".
    • a list of mock provided media if the value of the [= getDisplayMedia prompt result =] is "granted".
    The default value of [= getDisplayMedia prompt result =] is "granted".

Another approach is to use [[Permissions]] automation. The [[Permissions]] specification defines permissions to camera and microphone as well as display. It defines in particular the extension command to set each permission of the browsing contexts of a [= session | WebDriver session =].

[[Permissions]] automation is not well suited for {{MediaDevices/getUserMedia()}} and getDisplayMedia APIs. While the [[Permissions]] automation API allows to set each context indepdendently, user agents may grant/deny automatically requests based on past getUserMedia calls of the given or related contexts. The {{PermissionState/"prompt"}} value is also not useful for automation purposes and the value "granted" cannot be used for getDisplayMedia.

Set capture prompt result

HTTP Method [= extension command URI template | URI Template =]
POST /session/{session id}/capture-devices/prompt-result

The set capture prompt result extension command sets the [=getUserMedia prompt result=] and [=getDisplayMedia prompt result=] values. 

enum MockCapturePromptResult {
  "granted",
  "denied"
};

dictionary MockCapturePromptResultConfiguration {
  MockCapturePromptResult getUserMedia;
  MockCapturePromptResult getDisplayMedia;
};

The MockCapturePromptResultConfiguration dictionary is used to get and set the [=getUserMedia prompt result=] and [= getDisplayMedia prompt result =] values.

The remote end steps are:

  1. Let configuration be the command parameters argument, [=converted to idl values | converted to an IDL value =] of {{MockCapturePromptResultConfiguration}}. If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. If configuration.getUserMedia is set, set [=getUserMedia prompt result=] to configuration.getUserMedia.

  5. If configuration.getDisplayMedia is set, set [= getDisplayMedia prompt result =] to configuration.getDisplayMedia.

  6. Return success with data null.

Get capture prompt result

HTTP Method [= extension command URI template | URI Template =]
GET /session/{session id}/capture-devices/prompt-result

The get capture prompt result extension command sets the [=getUserMedia prompt result=] and [= getDisplayMedia prompt result =] values.

The remote end steps are:

  1. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  2. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  3. Let result be a MockCapturePromptResultConfiguration.

  4. Set result.getUserMedia to [=getUserMedia prompt result=].

  5. Set result.getDisplayMedia to [= getDisplayMedia prompt result =].

  6. Return success with data result.

Mock Capture Devices

A mock capture device simulates a real capture device or source of data. This specification defines mock camera and microphone devices.

A session has a set of mock capture devices. This set is used in {{MediaDevices/getUserMedia()}} as available sources and in {{MediaDevices/enumerateDevices()}} as available media decices.

A set of mock capture devices consists in a list of mock cameras and a list of mock microphones.

Mock Capture Device

Mock capture devices are video or audio sources. 

dictionary MockCaptureDeviceConfiguration {
  DOMString label;
  DOMString deviceId;
  DOMString groupId;
};

The MockCaptureDeviceConfiguration dictionary is used to create mock capture devices or get the state of mock capture devices.

Dictionary {{MockCaptureDeviceConfiguration}} Members

label of type {{DOMString}}
The system-wide label of the microphone, exposed as {{MediaDeviceInfo/label}}.
deviceId of type {{DOMString}}
The system-wide identifier of the capture device which allows generating {{MediaTrackSupportedConstraints/deviceId}} values.
groupId of type {{DOMString}}
The system-wide group identifier of the capture device which allows generating {{MediaTrackSupportedConstraints/groupId}} values.

Mock Camera

Mock cameras are video sources. A mock camera provides video frames to {{MediaStreamTrack}} of {{MediaStreamTrack/kind}} "video". 

dictionary MockCameraConfiguration : MockCaptureDeviceConfiguration {
  double defaultFrameRate = 30;
  DOMString facingMode = "user";
  // TODO: Add more capabilities parameters like:
  // ULongRange width;
  // ULongRange height;
  // DoubleRange frameRate;
};

The MockCameraConfiguration dictionary is used to create a mock camera.

Dictionary {{MockCameraConfiguration}} Members

defaultFrameRate of type {{double}}
The {{MediaTrackSupportedConstraints/frameRate}} to use by default when creating a video source if no frame rate constraint is provided by {{MediaDevices.getUserMedia()}}.
facingMode of type {{DOMString}}
The {{MediaTrackSupportedConstraints/facingMode}} value of the mock camera. A mock camera supports a single value.

A mock camera device has an associated configuration of type {{MockCameraConfiguration}}.

A session has a list of mock cameras.

At creation of the session, the [= list of mock cameras =] MUST contain one mock camera, whose configuration is:

{
  defaultFrameRate = 30;
  facingMode = "user";
};

The [= list of mock cameras =] MAY contain other mock cameras by default.

Mock Microphone

Mock microphones are audio sources. A mock microphone provides audio samples to {{MediaStreamTrack}} of {{MediaStreamTrack/kind}} "audio". The MockMicrophoneConfiguration dictionary is used to create a mock microphone. 

dictionary MockMicrophoneConfiguration : MockCaptureDeviceConfiguration {
  unsigned long defaultSampleRate = 44100;
  // TODO: Add more capabilities parameters like:
  // ULongRange sampleRate;
  // sequence echoCancellation;
};

Dictionary {{MockMicrophoneConfiguration}} Members

defaultSampleRate of type {{unsigned long}}
The {{MediaTrackSupportedConstraints/sampleRate}} to use by default when creating an audio source if no sample rate constraint exists when calling {{MediaDevices.getUserMedia()}}.

A mock microphone device has an associated configuration of type MockMicrophoneConfiguration.

A session has a list of mock microphones.

A session has a default mock microphone.

At creation of the session, the [= list of mock microphones =] MUST contain one mock microphone, defined as the [= default mock microphone =], whose configuration is:

{
  defaultSampleRate = 44100;
};

The [= list of mock microphones =] MAY contain other mock microphones at creation of the session.

Extension Commands

Add a mock camera

HTTP Method [= extension command URI template | URI Template =]
POST /session/{session id}/capture-devices/camera

The add mock camera extension command adds a new mock camera.

The remote end steps are:

  1. Let configuration be the command parameters argument, [=converted to idl values | converted to an IDL value =] of type MockCameraConfiguration. If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. Let mockCamera be the mock camera in the [=list of mock cameras=] whose {{ MockCaptureDeviceConfiguration/deviceId }} is equal to the url variable deviceId parameter, or undefined otherwise.

  5. If mockCamera is not undefined, set mockCamera’s configuration to configuration.

  6. Otherwise, run the following steps:

    1. Let mockCamera be a new mock camera.

    2. Set mockCamera’s configuration to configuration.

    3. Add mockCamera to the list of mock cameras.

    4. Run the following step [= in parallel =]:

      1. Execute the 'media input device changed' steps.

  7. Return success with data null.

Delete a mock camera

HTTP Method [= extension command URI template | URI Template =]
DELETE /session/{session id}/capture-devices/camera/{deviceId}

The delete mock camera extension command removes a mock camera from the [=list of mock cameras=].

The remote end steps are:

  1. Let deviceId be the result of getting the property deviceId from the parameters, [=converted to idl values | converted to an IDL value =] of type {{DOMString}}.

    If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. Let mockCamera be the mock camera in the list of mock cameras whose {{MockCaptureDeviceConfiguration/deviceId}} is equal to the url variable deviceId parameter, or undefined otherwise.

  5. If mockCamera is not undefined, run the following steps:

    1. Remove mockCamera from the [= list of mock cameras =].

    2. Run the following step [= in parallel =]:

      1. Execute the 'media input device changed' steps.

  6. Return success with data null.

Add a mock microphone

HTTP Method [= extension command URI template | URI Template =]
POST /session/{session id}/capture-devices/microphone

The add mock microphone extension command adds a new mock microphone or updates an existing one.

The remote end steps are:

  1. Let configuration be the result of getting the property configuration from the parameters, [=converted to idl values | converted to an IDL value =] of type {{MockMicrophoneConfiguration}}.

    If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. Let mockMicrophone be the mock microphone in the [= list of mock microphones =] whose {{MockCaptureDeviceConfiguration/deviceId}} is equal to the url variable deviceId parameter, or undefined otherwise.

  5. If mockMicrophone is not undefined, set mockMicrophone’s configuration to configuration.

  6. Otherwise, run the following steps:

    1. Let mockMicrophone be a new mock microphone.

    2. Set mockMicrophone’s configuration to configuration.

    3. Add mockMicrophone to the list of mock devices.

    4. If the [= default microphone =] is undefined, set [=default microphone=] to mockMicrophone.

    5. Run the following step [= in parallel =]:

      1. Execute the 'media input device changed' steps.

  7. Return success with data null.

Delete a mock microphone

HTTP Method [= extension command URI template | URI Template =]
DELETE /session/{session id}/capture-devices/{deviceId}

The delete mock capture device extension command deletes a new mock capture device.

The remote end steps are:

  1. Let deviceId be the result of getting the property deviceId from the parameters, [=converted to idl values | converted to an IDL value =] of type {{DOMString}}.

    If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. Let mockMicrophone be the mock microphone in the [=list of mock microphones=] whose {{MockCaptureDeviceConfiguration/deviceId}} is equal to the url variable deviceId parameter, or undefined otherwise.

  5. If mockMicrophone is not undefined, run the following steps:

    1. Remove mockMicrophone from the [=list of mock microphones=].

    2. If mockMicrophone is the [=default microphone=], set the [=default microphone=] to the first microphone in the [=list of mock microphones=] if not empty and to undefined otherwise.

    3. Run the following step [= in parallel =]:

      1. Execute the 'media input device changed' steps.

  6. Return success with data null.

Set default mock microphone device

HTTP Method [= extension command URI template | URI Template =]
POST /session/{session id}/capture-devices/default-microphone

The set default mock microphone device extension command sets the default mock microphone device.

The remote end steps are:

  1. Let deviceId be the result of getting the property deviceId from the parameters, [=converted to idl values | converted to an IDL value =] of type {{DOMString}}.

    If this throws an exception, return a [= error | WebDriver error =] with WebDriver error code invalid argument.

  2. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  3. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  4. Let mockMicrophone be the mock microphone in the [=list of mock microphones=] whose {{MockCaptureDeviceConfiguration/deviceId}} is equal to deviceId or undefined otherwise.

  5. If mockMicrophone is not undefined, run the following steps:

    1. Set the default mock microphone to mockMicrophone.

    2. Run the following step [= in parallel =]:

      1. Execute the 'media input device changed' steps.

  6. Return success with data null.

Reset mock capture devices

HTTP Method [= extension command URI template | URI Template =]
DELETE /session/{session id}/capture-devices

The reset mock capture devices extension command resets the list of mock capture devices to the default list of mock capture devices.

The remote end steps are:

  1. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  2. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  3. Set the the [=mock capture device set=] to the default mock capture device set.

  4. Run the following step [=in parallel=]:

    1. Execute the 'media input device changed' steps.

  5. Return success with data null.

Get mock capture devices

HTTP Method [= extension command URI template | URI Template =]
GET /session/{session id}/capture-devices

The get mock capture devices extension command gets the list of mock capture devices.

The remote end steps are:

  1. If the current browsing context is no longer open, return a [= error | WebDriver error =] with WebDriver error code no such window.

  2. Handle any user prompts, and return its value if it is a [= error | WebDriver error =].

  3. Return success with data as the [=mock capture device set=], serialized as JSON.