Long Animation Frames API

1. Introduction

As the page is loading and while the user is interacting with the page afterwards, both the application and browser queue various events that are then executed by the browser -- e.g. user agent schedules input events based on user’s activity, the application schedules callbacks for requestAnimationFrame and other callbacks, etc. Once in the queue, the browser dequeues these events one-by-one and executes them.

However, some tasks can take a long time (multiple frames) and if/when that happens, the UI thread may become blocked and block all other tasks as well. To the user, this is commonly visible as a "locked up" page where the browser is unable to respond to user input; this is a major source of bad user experience on the web today:

Delayed "time to Interactive":

while the page is loading, or even completely visually rendered, long tasks often tie up the main thread and prevent the user from interacting with the page. Poorly designed third-party content is frequently the culprit.

High/variable input latency:

critical user-interaction events (e.g. tap, click, scroll, wheel, etc.) are queued behind long tasks which yields janky and unpredictable user experience.

High/variable event handling latency:

like input, processing event callbacks (e.g. onload events, etc.) delay application updates.

Janky animations and scrolling:

some animation and scrolling interactions require coordination between compositor and main threads; if a long task is blocking the main thread it can affect responsiveness of animations and scrolling.

Some applications (and RUM vendors) are already attempting to identify and track cases where "long tasks" happen. For example, one known pattern is to install a ~short periodic timer and inspect the elapsed time between the successive expirations: if the elapsed time is greater than the timer period, then there is high likelihood that one or more long tasks have delayed execution of the event loop. This approach mostly works but has several bad performance implications: by polling to detect long tasks, the application prevents quiescence and long idle blocks (see requestIdleCallback); it’s bad for battery life; there is no way to know what is causing the delay (e.g. first party or third party code).

The RAIL performance model suggests that applications should respond to user input in less than 100ms (for touch move and scrolling, the threshold is 16ms). The goal of this API is to surface notifications about tasks that may prevent the application from hitting these targets. This API surfaces tasks that take 50ms or more. A website without these tasks should respond to user input in under 100ms: it will take less than 50ms to finish the task that is being executed when the user input is received and less than 50ms to execute the task to react to such user input.

1.1. Usage Example

const observer = new PerformanceObserver(function(list) {
    for (const entry of list.getEntries()) {
        // Process long task notifications:
        // report back for analytics and monitoring
        // ...
    }
});
// Register observer for previous and future long task notifications.
observer.observe({type: "long-animation-frame", buffered: true});
// Long script execution after this will result in queueing
// and receiving "long-animation-frame" entries in the observer.

// Register observer for previous and future long animation frame notifications.
// After this, long periods where the main thread is busy will result in queueing
// and receiving "long-animation-frame" entries in the observer.
observer.observe({type: "long-animation-frame", buffered: true});

1.2. Long Animation Frames vs. Long Tasks

While both long tasks and long animation frames measure congestion and jank, long animation frames provide information that has a better correlation with how users perceive this type of congestion. That’s because long animation frames measure a sequence that begins when the main thread is idle, and end when the frame either renders or the user agents decides there is nothing to render.

The task term is somewhat of an implementation detail, and the long animation frame addition attempts to remedy that by introducing a more user-centric metric of the same phenomenon of main thread congestion/jank.

Because long animation frames are guaranteed to have a maximum of one rendering phase, we can also use them to expose additional information about the rendering phase itself, such as renderStart and styleAndLayoutStart.

2. Long Animation Frame Timing

A long animation frame refers to any of the following occurrences whose duration exceeds 50ms:

• A task, after which updating the rendering is not necessary.

• A task after which updating the rendering is necessary, up until rendering is updated.

Long Animation Frame timing involves the following new interfaces:

2.1. PerformanceLongAnimationFrameTiming interface

[Exposed=Window]
interface PerformanceLongAnimationFrameTiming : PerformanceEntry {
    /* Overloading PerformanceEntry */
    readonly attribute DOMHighResTimeStamp startTime;
    readonly attribute DOMHighResTimeStamp duration;
    readonly attribute DOMString name;
    readonly attribute DOMString entryType;

    readonly attribute DOMHighResTimeStamp renderStart;
    readonly attribute DOMHighResTimeStamp styleAndLayoutStart;
    readonly attribute DOMHighResTimeStamp blockingDuration;
    readonly attribute DOMHighResTimeStamp firstUIEventTimestamp;
    [SameObject] readonly attribute FrozenArray<PerformanceScriptTiming> scripts;
    [Default] object toJSON();
};

A PerformanceLongAnimationFrameTiming has a frame timing info timing info.

The entryType attribute’s getter step is to return "long-animation-frame".

The name attribute’s getter step is to return "long-animation-frame".

The startTime attribute’s getter step is to return the relative high resolution time given this's timing info's start time and this's relevant global object.

The duration attribute’s getter step is to return the duration between this's startTime and the relative high resolution time given this's timing info's end time and this's relevant global object.

The renderStart attribute’s getter step is to return the relative high resolution time given this's timing info's update the rendering start time and this's relevant global object.

The styleAndLayoutStart attribute’s getter step is to return the relative high resolution time given this's timing info's style and layout start time and this's relevant global object.

The firstUIEventTimestamp attribute’s getter step is to return the relative high resolution time given this's timing info's first ui event timestamp and this's relevant global object.

The blockingDuration attribute’s getter steps are:

1. Let sortedTaskDurations be timing info's task durations, sorted in descending order.

2. If this's timing info's update the rendering start time is not zero, then:

   1. Let renderDuration be the duration between this's renderStart and the relative high resolution time given this's timing info's end time.

   2. Increment sortedTaskDurations[0] by renderDuration.

      Note: This makes it so that the longest task duration + render duration would be considered blocking if their total duration is >50ms.

3. Let totalBlockingDuration be 0.

4. For each duration in sortedTaskDurations, if duration is greater than 50 then increment totalBlockingDuration by duration - 50.

5. Return totalBlockingDuration.

The scripts attribute’s getter steps are:

1. Let scripts be a list « ».

2. Let entryWindow be this’s relevant global object.

3. For each scriptInfo in this's frame timing info's scripts:

   1. Let scriptWindow be scriptInfo’s window.

   2. Let scriptEntry be a new PerformanceScriptTiming in this's relevant realm, whose timing info is scriptInfo and whose window attribution is the value corresponding to the first matching statement:

      scriptWindow is undefined

      other

      scriptWindow is entryWindow

      self

      entryWindow’s associated Document's node navigable's ancestor navigables contains scriptWindow’s associated Document's node navigable

      ancestor

      scriptWindow’s associated Document's node navigable's ancestor navigables contains entryWindow’s associated Document's node navigable

      descendant

      entryWindow’s associated Document's node navigable's top-level traversable is scriptWindow’s associated Document's node navigable's top-level traversable

      same-page

      Otherwise

      other.

   3. Append scriptEntry to scripts.

4. Return scripts.

2.2. PerformanceScriptTiming interface

enum ScriptInvokerType {
    "classic-script",
    "module-script",
    "event-listener",
    "user-callback",
    "resolve-promise",
    "reject-promise"
};

enum ScriptWindowAttribution {
    "self", "descendant", "ancestor", "same-page", "other"
};

[Exposed=Window]
interface PerformanceScriptTiming : PerformanceEntry {
    /* Overloading PerformanceEntry */
    readonly attribute DOMHighResTimeStamp startTime;
    readonly attribute DOMHighResTimeStamp duration;
    readonly attribute DOMString name;
    readonly attribute DOMString entryType;

    readonly attribute ScriptInvokerType invokerType;
    readonly attribute DOMString invoker;
    readonly attribute DOMHighResTimeStamp executionStart;
    readonly attribute DOMString sourceURL;
    readonly attribute DOMString sourceFunctionName;
    readonly attribute long long sourceCharPosition;
    readonly attribute DOMHighResTimeStamp pauseDuration;
    readonly attribute DOMHighResTimeStamp forcedStyleAndLayoutDuration;
    readonly attribute Window? window;
    readonly attribute ScriptWindowAttribution windowAttribution;
    [Default] object toJSON();
};

A PerformanceScriptTiming has an associated script timing info timing info.

A PerformanceScriptTiming has an associated ScriptWindowAttribution window attribution.

The entryType attribute’s getter step is to return "script".

The name attribute’s getter step is to return "script".

The invokerType attribute’s getter step is to return this's timing info's invoker type.

The invoker attribute’s getter steps are:

1. Switch on this’s invokerType:

   "`classic-script`"

   "`module-script`"

   Return this’s timing info's source url.

   "`event-listener`"

   1. Let targetName be this’s timing info's invoker name.

   2. If this’s timing info's event target element id is not the empty string, then: Set targetName to the concatenation of « targetName, "#", this’s timing info's event target element id ».

   3. Otherwise, If this’s timing info's event target element src attribute is not the empty string, then: Set targetName to the concatenation of « targetName, '[src=', this’s timing info's event target element src attribute, ']' ».

   4. Return the concatenation of « targetName, ".on", this's timing info's event type ».

   this feels a bit custom, need to discuss name generation.

   "`user-callback`"

   Return this’s timing info's invoker name.

   "`resolve-promise`"

   "`reject-promise`"

   1. If this’s timing info's invoker name is the empty string, then:

      1. If this’s invokerType is "`resolve-promise`", then return "`Promise.resolve`".

      2. Otherwise, return "`Promise.reject`".

   2. Let thenOrCatch be "`then`" if invokerType is "`resolve-promise`"; otherwise "`reject-promise`".

   3. Return the concatenation of « invoker name, ".", thenOrCatch ».

The startTime attribute’s getter step is to return the relative high resolution time given this's timing info's start time and this's relevant global object.

The duration attribute’s getter step is to return the duration between this's startTime and the relative high resolution time given this's timing info's end time and this's relevant global object.

The executionStart attribute’s getter step is to return 0 if this's timing info's execution start time is 0; Otherwise the relative high resolution time given this's timing info's execution start time and this's relevant global object.

The forcedStyleAndLayoutDuration attribute’s getter step is to return an implementation-defined value that represents time spent performing style and layout synchronously, e.g. by calling getComputedStyle() or getBoundingClientRect().

Find a way to make this interoperable/normative. Perhaps mark those functions in WebIDL as requiring synchronous style/layout? Also move to timing info once that’s resolved.

The pauseDuration attribute’s getter step is to return this's timing info's pause duration.

The sourceURL attribute’s getter step is to return this's timing info's source url. The sourceFunctionName attribute’s getter step is to return this's timing info's source function name. The sourceCharPosition attribute’s getter step is to return this's timing info's source character position.

The window attribute’s getter steps are:

1. Let window be the result of calling deref on this's timing info's window.

2. If window is undefined, then return null; Otherwise return window.

The windowAttribution attribute’s getter step is to return this's window attribution.

3. Processing model

Note: A user agent implementing the Long Animation Frame API would need to include "long-animation-frame" in supportedEntryTypes for Window contexts, respectively.

3.1. Frame Timing Info

start time

current task start time

update the rendering start time

style and layout start time

first ui event timestamp

end time

A DOMHighResTimeStamp, initially 0. Note: all the above are [=monotonic clock/unsafe current time=|unsafe=], and should be coarsened when exposed via an API.

task durations

A list of DOMHighResTimeStamp, initially empty.

scripts

A list of script timing info, initially empty.

pending script

Null or a script timing info, initially null.

script timing info is a struct. It has the following items:

invoker type

A ScriptInvokerType.

start time

end time

execution start time

An unsafe DOMHighResTimeStamp, initially 0.

pause duration

A DOMHighResTimeStamp representing a number of milliseconds, initially 0.

invoker name

source url

source function name

event type

event target element id

event target element src attribute

A string, initially the empty string.

source character position

A number, initially -1.

window

A WeakRef to a Window.

A Document has a null or frame timing info current frame timing info, initially null.

3.2. Report Long Animation Frames

3.2.1. Long Animation Frame Monitoring {#loaf-monitoring}

3.2.2. Long Script Monitoring

4. Additions to existing standards

4.1. Monkey-patches to the WebIDL standard

5. Security & privacy considerations

The Long Animation Frames API adheres to the same-origin policy by including origin-safe attribution information about the source of the long task. There is a 50ms threshold for long tasks. Durations are only provided in 1 ms granularity. Together this provides adequate protection against cross-origin leaks.

The Long Animation Frames API provides timing information about the duration and type of tasks executed by the user, as well as attribution such as the browsing context causing the function calls. This could enable an attacker to perform side-channel timing attacks to guess the user’s action, or identify the user. For example, a pattern of long script followed by a long render could be put together to guess user’s interaction with a social widget. Detailed function call attribution would be used to determine the user’s action.

5.1. What is Exposed to Observers?

All observers within the top level page (i.e. all iframes in the page and the main frame) will receive notifications about presence of long animation frames. We expose the start time of the task, its duration (with 1 ms granularity), and a pointer to the culprit frame. This information can already be observed today, and with higher resolution, using setTimeout. An attacker can do this by clearing everything else on the page and adding the vulnerable cross-origin resource to ensure that delays from the setTimeout are caused by that resource. Observers in other different pages (tabs or windows) should not receive notifications, regardless of the architecture of the user agent.

5.2. Attack Scenarios Considered

The following are the timing attacks considered:

1. Traditional timing attacks: using external resource load time to reveal the size of private data. For instance the number of hidden pictures in a gallery, whether username is valid, etc. See an example.

2. Side-channel timing attacks: using time for video parsing, script parsing, App Cache reads or Cache API (service workers) usage to uniquely identify a user, or to create a profile of the user’s age, gender, location, and interests etc. For instance, status updates from a social network can be limited to certain demographic (eg. females of age 20-30) the file size of the permalink page can be used to determine whether the user is in the target demographic.

These scenarios are addressed by the 50ms threshold AND respecting cross-origin boundary i.e. not showing task type or additional attribution to untrusted cross origin observers.

5.3. Additional information exposed by the Long Animation Frames API

To mitigate this, long animation frames are only reported to "participating local roots": only documents that are associated with a work task that contributed to the sequence, or that were rendered as part of the frame, are eligible to observe the long animation frame, and that long animation frame would be available only in their nearest ancestor that is either topmost or has a cross-origin parent.

5.4. PerformanceScriptTiming and opaque scripts