Long Animation Frames API

1. Introduction

As the page is loading and while the user is interacting with the page afterwards, both the application and browser queue various events that are then executed by the browser -- e.g. user agent schedules input events based on user’s activity, the application schedules callbacks for requestAnimationFrame and other callbacks, etc. Once in the queue, the browser dequeues these events one-by-one and executes them.

However, some tasks can take a long time (multiple frames) and if/when that happens, the UI thread may become blocked and block all other tasks as well. To the user, this is commonly visible as a "locked up" page where the browser is unable to respond to user input; this is a major source of bad user experience on the web today:

Delayed "time to Interactive":

while the page is loading, or even completely visually rendered, long tasks often tie up the main thread and prevent the user from interacting with the page. Poorly designed third-party content is frequently the culprit.

High/variable input latency:

critical user-interaction events (e.g. tap, click, scroll, wheel, etc.) are queued behind long tasks which yields janky and unpredictable user experience.

High/variable event handling latency:

like input, processing event callbacks (e.g. onload events, etc.) delay application updates.

Janky animations and scrolling:

some animation and scrolling interactions require coordination between compositor and main threads; if a long task is blocking the main thread it can affect responsiveness of animations and scrolling.

Some applications (and RUM vendors) are already attempting to identify and track cases where "long tasks" happen. For example, one known pattern is to install a ~short periodic timer and inspect the elapsed time between the successive expirations: if the elapsed time is greater than the timer period, then there is high likelihood that one or more long tasks have delayed execution of the event loop. This approach mostly works but has several bad performance implications: by polling to detect long tasks, the application prevents quiescence and long idle blocks (see requestIdleCallback); it’s bad for battery life; there is no way to know what is causing the delay (e.g. first party or third party code).

The RAIL performance model suggests that applications should respond to user input in less than 100ms (for touch move and scrolling, the threshold is 16ms). The goal of this API is to surface notifications about tasks that may prevent the application from hitting these targets. This API surfaces tasks that take 50ms or more. A website without these tasks should respond to user input in under 100ms: it will take less than 50ms to finish the task that is being executed when the user input is received and less than 50ms to execute the task to react to such user input.

1.1. Usage Example

const observer = new PerformanceObserver(function(list) {
    for (const entry of list.getEntries()) {
        // Process long task notifications:
        // report back for analytics and monitoring
        // ...
    }
});
// Register observer for previous and future long task notifications.
observer.observe({type: "long-animation-frame", buffered: true});
// Long script execution after this will result in queueing
// and receiving "long-animation-frame" entries in the observer.

// Register observer for previous and future long animation frame notifications.
// After this, long periods where the main thread is busy will result in queueing
// and receiving "long-animation-frame" entries in the observer.
observer.observe({type: "long-animation-frame", buffered: true});

1.2. Long Animation Frames vs. Long Tasks

While both long tasks and long animation frames measure congestion and jank, long animation frames provide information that has a better correlation with how users perceive this type of congestion. That’s because long animation frames measure a sequence that begins when the main thread is idle, and end when the frame either renders or the user agents decides there is nothing to render.

The task term is somewhat of an implementation detail, and the long animation frame addition attempts to remedy that by introducing a more user-centric metric of the same phenomenon of main thread congestion/jank.

Because long animation frames are guaranteed to have a maximum of one rendering phase, we can also use them to expose additional information about the rendering phase itself, such as renderStart and styleAndLayoutStart.

2. Long Animation Frame Timing

A long animation frame refers to any of the following occurrences whose duration exceeds 50ms:

• A task, after which updating the rendering is not necessary.

• A task after which updating the rendering is necessary, up until rendering is updated.

Long Animation Frame timing involves the following new interfaces:

2.1. PerformanceLongAnimationFrameTiming interface

[Exposed=Window]
interface PerformanceLongAnimationFrameTiming : PerformanceEntry {
    /* Overloading PerformanceEntry */
    readonly attribute DOMHighResTimeStamp startTime;
    readonly attribute DOMHighResTimeStamp duration;
    readonly attribute DOMString name;
    readonly attribute DOMString entryType;

    readonly attribute DOMHighResTimeStamp renderStart;
    readonly attribute DOMHighResTimeStamp styleAndLayoutStart;
    readonly attribute DOMHighResTimeStamp styleDuration;
    readonly attribute DOMHighResTimeStamp layoutDuration;
    readonly attribute DOMHighResTimeStamp blockingDuration;
    readonly attribute DOMHighResTimeStamp firstUIEventTimestamp;
    [SameObject] readonly attribute FrozenArray<PerformanceScriptTiming> scripts;
    [Default] object toJSON();
};

PerformanceLongAnimationFrameTiming includes PaintTimingMixin;

A PerformanceLongAnimationFrameTiming has a frame timing info timing info.

The entryType attribute’s getter step is to return "long-animation-frame".

The name attribute’s getter step is to return "long-animation-frame".

The startTime attribute’s getter step is to return the relative high resolution time given this’s timing info’s start time and this’s relevant global object.

The duration attribute’s getter step is to return the duration between this’s startTime and the relative high resolution time given this’s timing info’s end time and this’s relevant global object.

The renderStart attribute’s getter step is to return the relative high resolution time given this’s timing info’s update the rendering start time and this’s relevant global object.

The styleAndLayoutStart attribute’s getter step is to return the relative high resolution time given this’s timing info’s style and layout start time and this’s relevant global object.

The styleDuration attribute’s getter step is to return this’s timing info’s style duration.

The layoutDuration attribute’s getter step is to return this’s timing info’s layout duration.

For every PerformanceLongAnimationFrameTiming entry, the sum of entry’s styleDuration and layoutDuration MUST be less than or equal to the duration between entry’s styleAndLayoutStart and entry’s paintTime. If entry’s timing info’s update the rendering start time is 0, i.e. the frame did not include a rendering update, then entry’s styleDuration and layoutDuration MUST both be 0.

Note: styleDuration and layoutDuration represent the time spent recalculating styles and updating layout, respectively, as part of the rendering update. Their sum can be less than the duration between styleAndLayoutStart and paintTime, as that period can also include other work that is not included in these durations, such as running ResizeObserver callbacks or updating intersection observations. Style and layout performed synchronously by scripts are not included, and are reported via forcedStyleDuration and forcedLayoutDuration instead.

The firstUIEventTimestamp attribute’s getter step is to return the relative high resolution time given this’s timing info’s first ui event timestamp and this’s relevant global object.

The blockingDuration attribute’s getter steps are:

1. Let sortedTaskDurations be timing info’s task durations, sorted in descending order.

2. If this’s timing info’s update the rendering start time is not zero, then:

   1. Let renderDuration be the duration between this’s renderStart and the relative high resolution time given this’s timing info’s end time.

   2. Increment sortedTaskDurations[0] by renderDuration.

      Note: This makes it so that the longest task duration + render duration would be considered blocking if their total duration is >50ms.

3. Let totalBlockingDuration be 0.

4. For each duration in sortedTaskDurations, if duration is greater than 50 then increment totalBlockingDuration by duration - 50.

5. Return totalBlockingDuration.

The scripts attribute’s getter steps are:

1. Let scripts be a list « ».

2. Let entryWindow be this’s relevant global object.

3. For each scriptInfo in this’s frame timing info’s scripts:

   1. Let scriptWindow be scriptInfo’s window.

   2. Let scriptEntry be a new PerformanceScriptTiming in this’s relevant realm, whose timing info is scriptInfo and whose window attribution is the value corresponding to the first matching statement:

      scriptWindow is undefined

      other

      scriptWindow is entryWindow

      self

      entryWindow’s associated Document’s node navigable’s ancestor navigables contains scriptWindow’s associated Document’s node navigable

      ancestor

      scriptWindow’s associated Document’s node navigable’s ancestor navigables contains entryWindow’s associated Document’s node navigable

      descendant

      entryWindow’s associated Document’s node navigable’s top-level traversable is scriptWindow’s associated Document’s node navigable’s top-level traversable

      same-page

      Otherwise

      other.

   3. Append scriptEntry to scripts.

4. Return scripts.

2.2. PerformanceScriptTiming interface

enum ScriptInvokerType {
    "classic-script",
    "module-script",
    "event-listener",
    "user-callback",
    "resolve-promise",
    "reject-promise"
};

enum ScriptWindowAttribution {
    "self", "descendant", "ancestor", "same-page", "other"
};

[Exposed=Window]
interface PerformanceScriptTiming : PerformanceEntry {
    /* Overloading PerformanceEntry */
    readonly attribute DOMHighResTimeStamp startTime;
    readonly attribute DOMHighResTimeStamp duration;
    readonly attribute DOMString name;
    readonly attribute DOMString entryType;

    readonly attribute ScriptInvokerType invokerType;
    readonly attribute DOMString invoker;
    readonly attribute DOMHighResTimeStamp executionStart;
    readonly attribute DOMString sourceURL;
    readonly attribute DOMString sourceFunctionName;
    readonly attribute long long sourceCharPosition;
    readonly attribute DOMHighResTimeStamp pauseDuration;
    readonly attribute DOMHighResTimeStamp forcedStyleAndLayoutDuration;
    readonly attribute DOMHighResTimeStamp forcedStyleDuration;
    readonly attribute DOMHighResTimeStamp forcedLayoutDuration;
    readonly attribute Window? window;
    readonly attribute ScriptWindowAttribution windowAttribution;
    [Default] object toJSON();
};

A PerformanceScriptTiming has an associated script timing info timing info.

A PerformanceScriptTiming has an associated ScriptWindowAttribution window attribution.

The entryType attribute’s getter step is to return "script".

The name attribute’s getter step is to return "script".

The invokerType attribute’s getter step is to return this’s timing info’s invoker type.

The invoker attribute’s getter steps are:

1. Switch on this’s invokerType:

   "`classic-script`"

   "`module-script`"

   Return this’s timing info’s source url.

   "`event-listener`"

   1. Let targetName be this’s timing info’s invoker name.

   2. If this’s timing info’s event target element id is not the empty string, then: Set targetName to the concatenation of « targetName, "#", this’s timing info’s event target element id ».

   3. Otherwise, If this’s timing info’s event target element src attribute is not the empty string, then: Set targetName to the concatenation of « targetName, "[src=", this’s timing info’s event target element src attribute, "]" ».

   4. Return the concatenation of « targetName, ".on", this’s timing info’s event type ».

   this feels a bit custom, need to discuss name generation.

   "`user-callback`"

   Return this’s timing info’s invoker name.

   "`resolve-promise`"

   "`reject-promise`"

   1. If this’s timing info’s invoker name is the empty string, then:

      1. If this’s invokerType is "`resolve-promise`", then return "`Promise.resolve`".

      2. Otherwise, return "`Promise.reject`".

   2. Let thenOrCatch be "`then`" if invokerType is "`resolve-promise`"; otherwise "`reject-promise`".

   3. Return the concatenation of « invoker name, ".", thenOrCatch ».

The startTime attribute’s getter step is to return the relative high resolution time given this’s timing info’s start time and this’s relevant global object.

The duration attribute’s getter step is to return the duration between this’s startTime and the relative high resolution time given this’s timing info’s end time and this’s relevant global object.

The executionStart attribute’s getter step is to return 0 if this’s timing info’s execution start time is 0; Otherwise the relative high resolution time given this’s timing info’s execution start time and this’s relevant global object.

The forcedStyleAndLayoutDuration attribute’s getter step is to return the sum of this’s timing info’s forced style duration and forced layout duration.

The forcedStyleDuration attribute’s getter step is to return this’s timing info’s forced style duration.

The forcedLayoutDuration attribute’s getter step is to return this’s timing info’s forced layout duration.

Note: forcedStyleDuration and forcedLayoutDuration represent the time spent recalculating styles and updating layout, respectively, synchronously while the script was running, e.g. when it called getComputedStyle() or getBoundingClientRect(). See § 3.2.3 Style and Layout Monitoring.

The pauseDuration attribute’s getter step is to return this’s timing info’s pause duration.

The sourceURL attribute’s getter step is to return this’s timing info’s source url. The sourceFunctionName attribute’s getter step is to return this’s timing info’s source function name. The sourceCharPosition attribute’s getter step is to return this’s timing info’s source character position.

The window attribute’s getter steps are:

1. Let window be the result of calling deref on this’s timing info’s window.

2. If window is undefined, then return null; Otherwise return window.

The windowAttribution attribute’s getter step is to return this’s window attribution.

3. Processing model

Note: A user agent implementing the Long Animation Frame API would need to include "long-animation-frame" in supportedEntryTypes for Window contexts, respectively.

3.1. Frame Timing Info

start time

current task start time

update the rendering start time

style and layout start time

first ui event timestamp

end time

A DOMHighResTimeStamp, initially 0. Note: these timestamps are unsafe, and should be coarsened when exposed via an API.

style duration

A DOMHighResTimeStamp representing a number of milliseconds, initially 0.

layout duration

A DOMHighResTimeStamp representing a number of milliseconds, initially 0.

style and layout stack

A stack of "`style`", "`layout`", and "`script`" strings, initially empty. It represents the style recalculations, layouts, and scripts that are in progress, with the innermost one at the top.

style and layout stack change time

A DOMHighResTimeStamp, initially 0, representing the coarsened time at which the style and layout stack was last modified.

style and layout periods

A list of style and layout periods, initially empty. It holds the style recalculations and layouts that were performed while no script was running.

cross-origin isolated capability

A boolean, initially false, used to coarsen the times from which style and layout durations are computed.

task durations

A list of DOMHighResTimeStamp, initially empty.

scripts

A list of script timing info, initially empty.

pending script

Null or a script timing info, initially null.

script initiated

A boolean, initially false.

script timing info is a struct. It has the following items:

invoker type

A ScriptInvokerType.

start time

end time

execution start time

An unsafe DOMHighResTimeStamp, initially 0.

pause duration

forced style duration

forced layout duration

A DOMHighResTimeStamp representing a number of milliseconds, initially 0.

invoker name

source url

source function name

event type

event target element id

event target element src attribute

A string, initially the empty string.

source character position

A number, initially -1.

window

A WeakRef to a Window.

A style and layout period is a struct. It has the following items:

type

"`style`" or "`layout`".

start time

end time

A coarsened DOMHighResTimeStamp.

A Document has a null or frame timing info current frame timing info, initially null.

3.2. Report Long Animation Frames

3.2.1. Long Animation Frame Monitoring {#loaf-monitoring}

3.2.2. Long Script Monitoring

3.2.3. Style and Layout Monitoring

The user agent calls begin style recalculation and end style recalculation given a Document whenever it starts and finishes recalculating the styles of that Document, and begin layout and end layout given a Document whenever it starts and finishes updating the layout of that Document. This includes style recalculation and layout performed as part of update the rendering, and style recalculation and layout performed synchronously while a script is running, e.g. when the script calls getComputedStyle() or getBoundingClientRect(). When exactly style recalculation and layout are performed, and therefore when these algorithms are called, is implementation-defined.

Find a way to make this interoperable/normative. Perhaps mark those functions in WebIDL as requiring synchronous style/layout?

Note: Style recalculation and layout can be nested, e.g. when evaluating container size queries requires recalculating styles in the middle of layout. Elapsed time is always attributed to the innermost style recalculation, layout, or script in progress, so the same time is never counted twice, and time spent running scripts is not counted as style or layout. Style recalculation and layout performed while a script is running count towards that script’s forced style duration and forced layout duration. Otherwise, they only count towards the frame’s style duration and layout duration if they are performed as part of the rendering update, after styleAndLayoutStart.

Note: Recalculating styles or updating layout for one Document can require doing so for other documents, e.g. for an ancestor whose layout determines the size of a descendant’s viewport. In that case, these algorithms are called for each of these documents. As time is attributed to each document’s relevant frame timing info, the style recalculation and layout of documents with a different nearest same-origin root are never attributed to the script that caused them, and only count towards their own root’s frame if they are performed as part of its rendering update.

4. Additions to existing standards

4.1. Monkey-patches to the WebIDL standard

5. Security & privacy considerations

The Long Animation Frames API adheres to the same-origin policy by including origin-safe attribution information about the source of the long task. There is a 50ms threshold for long tasks. Timestamps and durations are coarsened according to [HR-TIME], with a resolution that depends on the cross-origin isolated capability. Style and layout durations are sums of many short periods, so they are computed only from coarsened times (see accumulate style and layout time), which ensures that summing them up doesn’t expose timing information at a higher resolution than the timestamps. Together this provides adequate protection against cross-origin leaks.

The Long Animation Frames API provides timing information about the duration and type of tasks executed by the user, as well as attribution such as the browsing context causing the function calls. This could enable an attacker to perform side-channel timing attacks to guess the user’s action, or identify the user. For example, a pattern of long script followed by a long render could be put together to guess user’s interaction with a social widget. Detailed function call attribution would be used to determine the user’s action.

5.1. What is Exposed to Observers?

All observers within the top level page (i.e. all iframes in the page and the main frame) will receive notifications about presence of long animation frames. We expose the start time of the task, its duration (coarsened according to [HR-TIME]), and a pointer to the culprit frame. This information can already be observed today, and with higher resolution, using setTimeout. An attacker can do this by clearing everything else on the page and adding the vulnerable cross-origin resource to ensure that delays from the setTimeout are caused by that resource. Observers in other different pages (tabs or windows) should not receive notifications, regardless of the architecture of the user agent.

5.2. Attack Scenarios Considered

The following are the timing attacks considered:

1. Traditional timing attacks: using external resource load time to reveal the size of private data. For instance the number of hidden pictures in a gallery, whether username is valid, etc. See an example.

2. Side-channel timing attacks: using time for video parsing, script parsing, App Cache reads or Cache API (service workers) usage to uniquely identify a user, or to create a profile of the user’s age, gender, location, and interests etc. For instance, status updates from a social network can be limited to certain demographic (eg. females of age 20-30) the file size of the permalink page can be used to determine whether the user is in the target demographic.

These scenarios are addressed by the 50ms threshold AND respecting cross-origin boundary i.e. not showing task type or additional attribution to untrusted cross origin observers.

5.3. Additional information exposed by the Long Animation Frames API

To mitigate this, long animation frames are only reported to "participating local roots": only documents that are associated with a work task that contributed to the sequence, or that were rendered as part of the frame, are eligible to observe the long animation frame, and that long animation frame would be available only in their nearest ancestor that is either topmost or has a cross-origin parent.

styleDuration, layoutDuration, forcedStyleDuration, and forcedLayoutDuration only include the style recalculation and layout of documents that share the same nearest same-origin root (see § 3.2.3 Style and Layout Monitoring). They don’t expose how much time cross-origin documents spend on style and layout, beyond what the overall rendering timings described above already expose.

5.4. PerformanceScriptTiming and opaque scripts