Participants

Noam Helfman, Yoav Weiss, Nic Jansma, Steven Bougon, Thomas Kelly, Gilles Dubuc, Annie Sullivan, Ian Clelland, Michal Mocny, Michelle Vu, Nicolás Peña, Dominic Farolino, Benjamin De Kosnik, Alex Christensen, Ilya Grigorik, Philip Walton

Next Meeting

July 2nd @ 10am PST / 1pm EST

Presentations

Abandonment (explainer, slides)

Nicolás: We've talked about abandonment for a while
... Want to enable analytics providers and site owners to get visibility about users that leave the page early (before analytics have had time to register themselves)
… that’s a current blind spot
... Want sites to be able to correlate against business metrics
... Abandon definition:

User has explicitly navigated away from the page (redirects don't count)
Page did not reach First Contentful Paint (FCP)
Page was in the foreground at some point

... Non-abandons:

User explicitly navigates away
Page reached FCP
Page was in foreground at some point

... Need non-abandons to have denominator, as some browsers won't support this API and analytics needs to know abandon rate
... Why FCP? If not reached, the user didn't get value. Reaching FCP doesn’t guarantee user value, but understanding user value is beyond scope for this proposal.
... Not using onload (unrelated to user experience), LCP (we don’t know we reached LCP when the user abandons) or lack of input (there are pages that don’t require interaction)
... What should we expose? Counts for abandons and non-abandons
... Maybe: navstart-to-abandon-time, foreground-duration
... How to expose this information? Reporting API. Network-centered reporting (via HTTP headers)

Would be better to base that on Origin Policy, but unclear when it will be ready
NEL-like headers only solution won’t capture first-view cases where abandonment happened before headers arrived

Ilya: NEL currently explicitly excludes user-abandoned actions such as back-button or tab close
Nic: Would this be "sticky" on the domain, where once you register, it would apply to future navigations?
Ian: Yes, and it's per-origin not per-file
Steven: Can you go over how abandonment might be useful?
Nicolás: Let's say you have a normally operating website, and FCP improves. Is it because users were abandoning in the worse-case scenarios?

Also, if you see a smaller number of users reported - is it because more users abandon before you register them?

... Can also correlate to business metrics
Ilya: There's a non-trivial amount of abandoned navigations across the web, where there's no visibility today. We want to expose that.
Annie: 6-8% of navigations are abandoned on mobile, varies by site. Sites that didn’t meet Core Web Vitals were 24% more likely to be abandoned.
Ben: Does the 6-8% account for network errors or just abandons?
Annie: Just user abandons, excluded redirects or other detectable network errors in the numerator or denominator
Michelle: From Pinterest, we are interested in this as we've just done an abandonment analysis, but are probably missing cases that don’t register. Were there other metrics besides FCP considered, like TTI?
Annie: TTI might be too late, LCP is a continuous metric that may always increase over time
Yoav: One thought is to have FCP as default, but the website could declare a certain JS variable that if it was never defined by JS, the page was abandoned. For the site to say what it considers a non-abandonment
Nicolás: We could definitely have a parameter that enables sites measure their own definition of abandonment
Philip: Could we report abandons that hadn't reached onload versus abandons that haven't reached FCP? Because mostly by onload the page is configured and tracked by other analytics
Nicolás: What are we trying to incentivize. Onload can be made arbitrarily fast, but that doesn’t improve the UX.
… Worth considering if there’s another point in time where we’re sure analytics are registered, but don’t know what that point in time would be. Might be better to have it be site provided.
Philip: Scenario where a user sees a loading screen, but it hasn't reached onload or loaded analytics
Nicolás: Could that be covered by the site-provided proposal?
Ilya: Might be worthwhile to split up 2 cases - Up to FCP analytics could not have run. After FCP, the page is being constructed but analytics may be running.
Michal: What Michelle was pointing out was more along User Experience abandons, that they want to measure when visitor is interacting with the page and abandons it.
... Which is separate from the un-measurable experiences today (before FCP was reached). Maybe worthwhile to use a different name than “abandonment”
Annie: There’s also “cart abandonment”
Ilya: One question was dependency on Reporting API / origin-policy and how that would affect our timeline
Nicolás: Origin-policy should be going forward and not affect our timeline

Back-Forward Cache and RUM metrics

Back-forward cache and RUM metrics
Yoav: We've talked about this in the past but didn't really reach any conclusions
... Browser may implement or improve bfcache
... Sites can change to be "BF Cache eligible" by changing event registrations (unload)
... Can result in user experience changes
... What can we do about it?
... Update old metrics? Seems destructive. Might not be web-compatible for current analytics scripts.
... Fire new metrics? For those we care about, and have analytics scripts capture those new entries.
... Which metrics? NavigationTiming, Paint Timings.

But we’d also need to correlate other metrics to page loads.

... Web compat: Size of NavigationTiming timing array today is usually 1 and if we add new entries it may cause compat concerns.
... Similar for Paint Timings.
... Correlate ResourceTiming, ElementTiming, EventTiming, Layout Instability to page loads. Need to be able to split those out from one another.
... Multiple Time Origins? All metrics are based out of a single time origin today. Should we have a metric that shows after N milliseconds in, this is a new page load. Allow for filtering of entries by navigation.
Nic: Would each new NavigationTiming entry be the "start" of each new time origin
Yoav: That would be possible, or something similar with Visibility Observer
... Right now the method to know whether your page has gone into or out of bfcache is through the pageshow / pagehide events, which aren't necessarily integrated into the Observer interfaces.
... The other thing we need to consider is SPA navigations. Do they affect time origins?
Ilya: BFCache is new to Chromium, but not other engines. How do existing analytics providers treat it?
Nic: mPulse doesn’t measure it today
Michal: Do existing analytics providers take visibility state into account today?
Nic: For mPulse/Boomerang, we track visibility state (and last time it was hidden/visible) on the beacon, and let our customer segment their data on that state
Steven: Yes visibility, no BFCache
Michal: Like the comparison to visibility. What about background tabs?

If you have separate “foreground sessions” to a certain tab, does the analytics get split up?

Nic: mPulse tracks visibility state and customers can split their data based on that. Most people don’t, but we’re currently promoting that.
Yoav: Folks running web properties - is this something you consider today?
Steven: Not BF cache, but we do consider visibility.
Michelle: Same here, we just started taking visibility into account last year
Gilles: Same. But we don’t like the current approach to visibility of simply filtering it out. Would be better to still measure those scenarios and be able to identify them.
Philip: Our concern is that when Chrome ships BFCache, things that used to be back-forward navigations that were fast, will no longer be full navigations, so those won't be represented in metrics.
… i.e. compared to other browsers, today Chrome reports additional navigations because there's no BFCache, but that will soon change.
... How do existing analytics providers show data for BFCache navigations?
Nic: I can only remember a few times our customers have asked about BFCache, so the industry doesn't seem to be well educated about it. It's probably on analytics providers to highlight these types of navigations better.
Michal: Similarly it is related to SPAs. SPAs that manually handle the back button, won’t be counted as navigation as well.
Yoav: So generally people don’t measure BF cache navigation just yet, but it would be good to start?
Gilles: How does one know it's a BFCache, is it in navigation.type?
Yoav: It's actually not represented, the DOM is the same as from before, i.e. navigation.type is the same as when the page first loaded. You’re just getting a pageshow event
Philip: Discussed at TPAC last year - Firefox were considering updating the type in that case
Yoav: There’s a “history navigation” type, but it’s not stated if it’s a BF cache one or a regular history navigation. And in the case of BF cache it’s not updated. We’d likely mint a new type.
Michal: Important to track BF cache navigation, but the total number of navigations a user commits will affect their metrics depending on how you track it. Many things can influence that. User-behavior and how many top-level navigations they commit during a session is important for tracking the user experience.
Yoav: UX can vary based on many factors. We’re trying to minimze disruption by the BF cache introduction, but would be good to measure things holistically.
Michal: Comparison to Visibility is interesting. If the user chooses to “open in new tab” and then go back, is that significantly different than navigating to the new page and then clicking “back”?
… We’re hoping that BF cache navigation is super fast, but so is moving a tab from background to foreground. The overall perf story seems very similar
Yoav: So you want moving to BG and then FG to also count as a “fast navigation”?
Michal: If BF cache counts, so should this
Yoav: BF cache is somewhat different, as if you’re not in the cache, the experience is slower. But that’s also potentially true for BG, e.g. if the browser threw your process out of memory, forcing a new load for the page.
Ilya: If I'm hearing you properly, there are many factors that affect navigations. In the particular context of BFCache, the industry is currently ignoring that.
…When we ship this in Chrome, it may look like performance has regressed (because the denominator has changed), where the true story is that it has improved.
... Because sites aren't looking at this dimension, they aren't optimizing for it. If you don't know that, you're excluding an easy win for your site.
Michal: Good summary. This is one of many examples where you can influence the total number of navigations. I don’t see why this case is special
Yoav: Would be good to consider all of them holistically, and plan for an extensible solution that can fit these future types.

Next Call

Proposal: Maybe worthwhile to move to Tuesday June 30th to avoid holidays. Yoav to check.