The features in this specification extend or modify those found in Pointer Events Level 2 that describes events and related interfaces for handling hardware agnostic pointer input from devices including a mouse, pen, touchscreen, etc.

This proposal intends to add new functionalities to a future version of the Pointer Events Level 2.

Introduction

This document reflects incubation for APIs which are intended to be part of a future version of the Pointer Events specification. They are not yet broadly supported by implementations.

The getCoalescedEvents API (similar to Android getHistorical APIs and iOS GetCoalescedTouches API) exposes all the events that were coalesced into a single event. User agents may coalesce a stream of events into a single event sent to javascript. This situation can occur when many events are generated by the hardware in between the requestAnimationFrame boundaries or for long running javascript code that misses the frame boundary. Basically, if the javascript handlers process the events and update the screen contents faster than the screen refresh rate only the last update will render and show in the next frame. So for example, if there were multiple pointermove events happened from the last requestAnimationFrame callback, UA can coalesce all of them into one event and send only that event before the next requestAnimationFrame representing the last state of the pointer. Most applications care only about the last coordinates and not the intermediate coordinates that occurred. However, a class of applications such as drawing applications may wish precise (subframe timing) coordinates in order to draw better detailed lines. The getCoalescedEvents API can be used in those scenarios to get all the coalesced events from the event that was dispatched to the javascript.

The getPredictedEvents API tries to achieve the minimal latency for drawing app similar to other platforms such as iOS prediction APIs. The drawing latency on the web comes from a wide variety of factors such as UA input pipeline latency and multi level bufferings of the output buffer. This API exposes what UA thinks the near future state of pointer will be and the web app can choose to use that state for example to draw those points temporarily for the current frame. That would help to reduce the perception of overall latency when user interacts with the page.

While for most apps getting all the events and their history before rAF and drawing the result on the page is enough, more time sensitive apps care to get the event as soon as they arrive. They might be sending input events over the network or play a musical note that aren't necessarily tied to the frame time and the screen refresh rate. Coalescing events or rAF-aligning them would hurt those apps. The new pointerevent type pointerrawupdate that fires immedinatedly as they arrive introduced in this spec would help those usecases. Note that there are some considerations around this that will be discussed later in this spec.

Extensions to the Pointer Event types

Below is a new pointerrawupdate event type as well as a modification to the pointermove event in the existing pointer event types.

The pointermove event

A user agent MUST fire a pointer event named pointermove when a pointer changes button state. Additionally one pointermove MUST be fired when pointer changes coordinates, pressure, tangential pressure, tilt, twist, or contact geometry (e.g. width and height) and the circumstances produce no other pointer events defined in this specification. These events may be coalesced or aligned to animation frame callbacks based on UA decision. The coalesced events information will be exposed via getCoalescedEvents API for the single dispatched pointermove event. The final coordinates of such events should be used for finding the target of the event.

The pointerrawupdate event

A user agent MUST fire a pointer event named pointerrawupdate when a pointing device attribute (i.e. button state, coordinates, pressure, tangential pressure, tilt, twist, or contact geometry) is changed. As oppose to pointermove which might be aligned to animation callbacks, user agents SHOULD dispatch pointerrawupdate events as soon as possible and as frequent as the javascript can handle the events. The target of pointerrawupdate events might be different from the pointermove events due to the fact that pointermove events might get aligned with animation frame callbacks and get coalesced and the final position of the event which is used for finding the target could be different from its coalesced events. Note that if there is already another pointerrawupdate with the same pointerId that hasn't been dispatched in the task queue user agent MAY coalesce the new pointerrawupdate with that event instead of creating a new task. So this may cause pointerrawupdate to have coalesced events and they will all be delivered as coalesced events of one pointerrawupdate event as soon as the event's turn to get processed reaches in the task queue. See getCoalescedEvents for more information. In terms of ordering of pointerrawupdate and pointermove, if the UA received an update from the platform that causes both pointerrawupdate and pointermove events then the user agent MUST dispatch pointerrawupdate event before the correponding pointermove for it. Other than the target, the concatenation of coalesced events of all dispatched pointerrawupdate events since the last pointermove event is the same as coalesced events of the next pointermove event in terms of the other attributes. The attributes of pointerrawupdate are mostly the same as pointermove with the exception of cancelable which MUST be false for pointerrawupdate. User agent SHOULD not fire compatibility mouse events for pointerrawupdate.

Adding listener for this type of the event might impact the performance of the web page negatively depending on the implementation of user agent. For most of the use cases the other pointerevent types should suffice. A pointerrawupdate listener should only be added if javascript needs high frequency events and can handle them just as fast. In that case there is probably no need to listen to other types of pointer events for most of the use cases.

Extensions to the PointerEvent interface

The following section describes extensions to the existing PointerEvent interface.

partial dictionary PointerEventInit {
    sequence<PointerEvent> coalescedEvents = [];
    sequence<PointerEvent> predictedEvents = [];
};

partial interface PointerEvent {
    sequence<PointerEvent> getCoalescedEvents();
    sequence<PointerEvent> getPredictedEvents();
};
          

A PointerEvent has an associated coalesced event list (a list of zero or more PointerEvents). If this event is a pointermove or pointerrawupdate event, it is a sequence of all PointerEvent that were coasesced into this event; otherwise it is the empty list.

The events in the coalesced event list will have increasing timeStamps ([[!WHATWG-DOM]]), so the first event will have the smallest timeStamp.

The PointerEvent's attributes will be initalized in a way that is best representative of all event in the coalesced event list. For example its movementX and movementY ([[!POINTERLOCK]]) COULD be the sum of those of all the coalesced events.

A PointerEvent has an associated predicted event list (a list of zero or more PointerEvents). If this event is a pointermove event, it is a sequence of PointerEvents the user agent predicts that will follow the events in coalesced event list in the future; otherwise it is the empty list. The number of events in the list and how far they are from the current timestamp are determined by the user agent and the prediction algorithm it uses.

The events in the predicted event list will have increasing timeStamps ([[!WHATWG-DOM]]), so the first event will have the smallest timeStamp which should still be greater than the last event in the coalesced event list.

When a PointerEvent is created, run the following steps for each event in the coalesced event list and predicted event list:

  1. Set event's pointerId, pointerType, isPrimary and isTrusted to the PointerEvent's pointerId, pointerType, isPrimary and isTrusted.

  2. Set event's cancelable and bubbles attributes to false.

  3. Set event's coalesced event list and predicted event list to empty list.

  4. Initialize the rest of the attributes the same way as PointerEvent.

A PointerEvent has an associated coalesced events targets dirty boolean and an associated predicted events targets dirty boolean. When an event is created they must be initialized to false.

When PointerEvent's target is changed, set both coalesced events targets dirty and predicted events targets dirty to true.

getCoalescedEvents

coalesced event list's getter, when invoked, must run these steps:

  1. If the coalesced events targets dirty is true: for each event in the coalesced event list, set the event's target to this PointerEvent's target.
  2. Set the coalesced events targets dirty to false.
  3. Return the coalesced event list

getPredictedEvents

predicted event list's getter, when invoked, must run these steps:

  1. If the predicted events targets dirty is true: for each event in the coalesced event list, set the event's target to this PointerEvent's target.
  2. Set the predicted events targets dirty to false.
  3. Return the predicted event list

Timing of coalesced event firing

Coalescing only happens for pointermove and pointerrawupdate events. Note that user agents don't need to hit-test all the coalesced events as all of them will have the same target as the dispatched event.

When the user agent dispatches any PointerEvent it MUST dispatch all the events held back so far for coalescing, regardless of their pointerIds. The ordering of all these dispatched events should be in a way that resemebles the most with the actual events' ordering. For example if a pointerdown event causes the dispatch for the coalesced pointermove events the user agent SHOULD first dispatch one pointermove event with all those coalesced events of a pointerId followed by the pointerdown event.

Here is an example of the actual events happening in one frame with increasing timestamps and the events dispatched by the user agent:

Actual eventsDispatched events
pointermove with pointerId=2
pointermove with pointerId=1
pointermove with pointerId=2
pointermove with pointerId=2
pointermove with pointerId=1
pointermove with pointerId=2
pointerdown with pointerId=1 pointermove with pointerId=1 and two coalesced events
pointermove with pointerId=2 and four coalesced events
pointerdown with pointerId=1 and zero coalesced events
pointermove with pointerId=2
pointermove with pointerId=2
pointerup with pointerId=1 pointermove with pointerId=2 and two coalesced events
pointerup with pointerId=1 and zero coalesced events

Additional touch-action values

This section augments the definition of touch-action from [[!pointerevents2]] to add the pan-left, pan-left, pan-up, pan-down values.

Name:touch-action
Value:auto | none | [ [ pan-x | pan-left | pan-right ] || [ pan-y | pan-up | pan-down ] ] | manipulation
Initial:auto
Applies to:all elements except: non-replaced inline elements, table rows, row groups, table columns, and column groups.
Inherited:no
Percentages:N/A
Media:visual
Computed value:Same as specified value.

The user agent MAY consider touches that begin on the element only for the purposes of scrolling that starts in any of the directions specified by all of the listed values. Once scrolling is started, the direction may be reversed by the user even if scrolls that start in the reversed direction are disallowed. In contrast, when scrolling is restricted to starting along a single axis (eg. pan-y), the axis cannot be changed during the scroll. In the case of pan-left, pan-right, pan-up and pan-down, the direction is interpreted as the opposite of the physical movement in the local coordinate space. For example, pan-up corresponds to input event sequences where typically screenY is increasing (i.e. an interaction where the user moves a touch point down the screen) when the element has no effective CSS transforms. If the element is effectively rotated by 90 degrees counter-clockwise, pan-up would then correspond to input event sequences where screenX is increasing.

The direction-specific pan values are useful for customizing overscroll behavior. For example, to implement a simple pull-to-refresh effect the document's touch-action can be set to pan-x pan-down whenever the scroll position is 0 and pan-x pan-y otherwise. This allows pointer event handlers to define the behavior for upward scrolls that start from the top of the document.

The direction-specific pan values can also be used for composing a component that implements custom panning with pointer event handling within an element that scrolls natively (or vice-versa). For example, an image carousel may use pan-y to ensure it receives pointer events for any horizontal pan operations without interfering with vertical scrolling of the document. When the carousel reaches its right-most extent, it may change its touch-action to pan-y pan-right so that a subsequent pan operation beyond it's extent can scroll the document within the viewport if possible. It's not possible to change the behavior of a pan in the middle of an operation.

Examples

The following are examples that demonstrate how the APIs in this specification might be used.


var pointerEventInitDict =
{
  bubbles: true,
  cancelable: true,
  composed: true,
  pointerId: 42,
  pointerType: "pen",
  clientX: 300,
  clientY: 500,
};

var p1 = new PointerEvent("pointermove", pointerEventInitDict);
pointerEventInitDict.clientX += 10;
var p2 = new PointerEvent("pointermove", pointerEventInitDict);
pointerEventInitDict.coalescedEvents = [p1, p2];
var event = new PointerEvent("pointermove", pointerEventInitDict);

window.addEventListener("pointermove", function(event) {
  for (let e of event.getCoalescedEvents()) {
    drawPoint(e.pageX, e.pageY);
  }
});


var predicted_points = [];
window.addEventListener("pointermove", function(event) {
  for (let e of predicted_points.reverse())
    clearPoint(e.pageX, e.pageY);
  for (let e of event.getCoalescedEvents())
    drawPoint(e.pageX, e.pageY);
  predicted_points = event.getPredictedevents();
  for (let e of predicted_points)
    drawPoint(e.pageX, e.pageY);
});

<div style="overflow: auto;">
    <div style="touch-action: pan-y pan-left;">
        <div style="touch-action: pan-x;">
            This element receives pointer events when not panning to the left.
        </div>
    </div>
</div>