Participants
- Yoav Weiss, Noam Helfman, Andy Davies, Nic Jansma, Ane Diaz De Tuesta, Guohui Deng, Jenna Sasson, Josè Dapena Paz, Olga Gerchikov, Rahul Arekeri, Adrian de la Rosa, Jacob Groß, Alex Christensen, Michal Mocny, Ben Schouten, Benoît Zugmeyer, Joone Hur, Barry Pollard,
Admin
- Next meeting: July 31st
- TPAC registration
- Suggest topics, preferences, etc
Minutes
Animation Smoothness - Microsoft (Jenna, Rahul)
- Jenna: PM at Edge team
- … Olga and Rahul on call as well
- … Proposal for more precise way to measure animation smoothness on the web
- … Critical to UX, but to improve it we need to measure it accurately
- … Current metrics (frame rate, frame latency, don’t fully capture what user perceives)
- … Trying to close gap
- … New API that would give developers a more accurate view of smoothness
- … Original thought was to reflect main and compositor thread activity
- … Maybe only main thread is sufficient
- … Want API to be lightweight and performant
- … Allow developers to define their own measurement windows
- … Looking at browser’s entire window content
- … Identified 5 main use-cases
- … 1. On Demand Animations
- … 2. Measuring animation and graphics of browsers (rAF)
- … 3. Gaming, where smoothness directly impacts gameplay
- … 4. Testing animation performance on different hardware
- … 5. Improving animation libraries which rely on custom logic or FPS tracking
- … Existing approaches, with strengths and limitations
- … rAF is easy to use, but can be energy intensive, not accurate of perceived smoothness
- … Looked into Long Animation Frames which lack specifics
- … For RequestVideoFrameCallback, it’s video-specific and can be inconsistent between browsers
- … Edge had a FPS emitter library, built off rAF, similar limitations
- … Right now we’re exploring 4 API designs
- … No proposed front-runners yet
- … Option 1: window.frameInfo() or something similar
- … Option 2: Start and End markers, when duration is too long, might know there’s an issue, similar to Frame Timing API
- … Option 3: Event Listener, triggers events when frames drop below certain threshold (not a lot of detail, probably looking at this the least)
- … Option 4: Performance Observer, similar to LoAF and PaintTiming, observer for specific performance events, trigger alerts when performance dips
- … Open comments and questions in explainer
- … No finalized API yet, we’re in the discussion phase.
- … Goal is to choose shape and design of API
- Rahul: We’ve gotten good feedback
- … Ask if anyone has prototypes or past experience solving this, we can take a look at that
- … Trying to experiment and look at different things
- Jenna: https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/AnimationSmoothness/explainer.md
- Bas: At start of presentation, after consultation, suggested only looking at main thread would be sufficient for measurement
- Jenna: Got feedback on PR 1003, compositor that struggles keeping up will apply pressure on the main thread as well
- Bas: “Compositor animations are expected to be fast” is not a good argument. As browsers move filters into them etc. they can get slow. So we shouldn’t ignore the compositor there
- … backpressure depends on the pipeline, which varies by browser (e.g. Safari has none in some cases)
- Michal: Agree that we try to push more things to the compositor, so it could jank up
- … I may have given that feedback that very low level info on the compositor would be hard
- … LoAF may be able to be expanded to include the main thread as well
- … For the compositor, maybe we need something like the compute pressure API. Otherwise, it could get very complicated, and expressing that to developers could be hard and would be hard for developers to do something with it
- … Maybe just exposing the point in time can help developers investigate this in the lab
- NoamH: need to expose cases where the compositor TODO[a] the main thread. We saw cases where we know we have a problem but we don’t know if it happens in production. Important to understand when the compositor is faster than the main thread. Content depends on the main thread
- Olga: Want to expand on what Noam was saying. The API should reflect frame rate and frame completeness perceived by users. Should focus on details, but on the final state of the pipeline
- … Frame completeness can be comprised of: checker board, partial from main thread, partial from compositor thread, iframes, etc
- … So maybe not focus on implementation but align on user perception
- Bas: That feels hard because none of this is standardized. Only the main thread loop is. Could vary between implementation, so could be hard to bridge gaps
- Olga: Maybe can still be expressed in deterministic terms
- Bas: That would require better specs for the painting pipeline if we wanted this to be interoperable
- Michal: A fan of this goal. With CWV hat-on, capturing real-user experiences is very complicated. Users conflate slow-loading, with smoothness, CLS, etc. Sites can be janky with a clean paint pipeline.
- … A simpler problem is when the browser janks in ways we don’t want. That narrow use-case is easier to standardize.
- … rAF polling has a ton of problems, as it triggers paints. LoAF enables observing frames without impacting them. Maybe we can expand it to report frame info every 50 ms.
- NoamH: Tried to compare LoAF and rAF polling. Seems to correlate pretty well, but still missing key aspects, especially outside the main thread.
- Michal: Scrolling is an interesting example. In Chromium, fling-scroll can throttle main thread frames to save work to the compositor. But that can be good for the user
- … The web platform can measure rAF in web workers (for off-screen canvas). Backpressure would also impact worker rAF. It could be possible to measure rAF in a worker for benchmarks (TODO[b] - is that correct?)
- Olga: The API proposal evolved from MotionMark
- Michal: There’s also a distinction between being able to produce frames and making sure those frames actually have the content
- Olga: That’s where frame completeness comes in
- Jenna: Good feedback. I’ll send a link to the GH issues where we can continue this!
Should all values be exposed in CSS pixels or Physical Pixels? - Michal
- Michal: Layout shift attribution - what pixels?
- … once per animation frame you get a summary of what content shifted, with a list of sources and relevant rects
- … In Chromium it’s recorded in physical pixels, where all other APIs report CSS pixels
- … Can be annoying to convert so there’s a proposal to change this
- … The more important value is the “value” attribute, and that won’t change because it’s a ratio
- … So only the attribution list will change
- … Questions - who is using this? Would that break things?
- Yoav: Do we have use-counters on who is touching this information?
- Michal: Need to look it up
- … Concerned there’s a lot of accidental triggers, not clear who is reading it is using it
- Yoav: Would be an upper-bound, if the number is zero, there’s a high risk of accidental
- Michal: Doubt it’s zero, reading and summarizing
- Nic: We’re using it, and not necessarily showing it in smart ways
- Andy: We use it in RUM and also extract it from Chrome traces for film strips
- Michal: Can we change it in place? How would you know what the unit is?
- Jose: Similar problem with element timing
- Michal: I think it’s CSS pixels already. Didn’t check size for LCP
- Barry: May be some tools that rely on it (e.g. DebugBear)
- Michal: Reached out to the devtools team and they worked around this
- … Maybe we’d need to expose both values in traces, but not necessary for the RUM API?
- Ane: discovered this issue while building a troubleshooting tool in DataDog. Saw that the different screens saw different results, where the rects we were trying to draw where skewed, based on DPR
- … We are not aware of the DPR, so very annoying to resolve this
SoftLCP → InteractionContentfulPaint - Michal
- Michal: Chromium is working on SoftNavigations for a while. Iterated on this in the last year, but wanted to discuss some changes
- … Heuristics - user initiated an interaction which creates a context that carries on to new tasks spawned from the relevant event handlers
- … Then we observe history changes and DOM changes, and attribute those to the relevant interaction
- … Attribute the page update to the context as well.
- … When that happens, we’d start reporting a new PerformanceEntry called InteractionContentfulPaint
- … Slightly related to ContainerTiming
- … The whole page’s paints are now attributed to the contexts that changed them
- … When the criteria are create, we also emit a SoftNavigationEntry.
- … So you can use both entries together
- … Expect this in an Origin Trial near you.
- .. For future exciting opportunities - this currently matches LCP, but under the hood we have to do more, because we have to measure all parts of the page.
- … so we could expand the LCP semantics to do that
- … Any time there’s an interaction, we could report all the paints that happened and related to that.
- Nic: Excited to experiment with this. We attempted to do similar things in the past, but this will help our SPA customers.
- … Seems like the OT implementation won’t be drastically different
- Barry: Updated WebVitals to this and this was a single line code change
- Yoav: Soft Nav entry didn’t change, still relies on navigationId? LCP now doesn’t re-fire, interaction-contentful-paint instead.
- … Linking LCP to relevant navigation there’s still a navigationId linking
- Michal: Correct. Some debate if that’s needed?
- … You could still use timestamps
- … If everything on PerformanceTimeline was time-organized, you could use order of entries (Not always the case)
- … We did leave navigationId in,
- … We won’t issue ICP entries until after soft-nav
- … This is nice and layered, but interactions might paint before the navigation is committed
- … In theory, we’re observing it’s happening but we don’t report it to timeline
- … I’d like ICP to use beyond soft navigation
- … When you have a navigation, the Layout Shifts in-between, which are they for?
- … Working on that
- Benjamin: New API, opportunity to do more, take it
- Michal: We copied LCP entry intersectionRect, we could also expose
- Noam: Great to see progress on this, keep wondering if this would allow us to replace current mechanism, timestamp capturing, should based on the description
- … One challenge I see is accuracy of LCP, in case not the right thing you want to capture changes
- … Something started before, example from TPAC, right-click and a context-menu appears
- … Interested in interaction started from the click in the context menu
- Michal: In current we have a restriction is a URL change happens first
- … Make sure it doesn’t regress Chromium performance
- … Shortly you should be able to measure all interactions
- … Also we’re constrained to a subset of interaction types, but I want to expand it
- … Interop project this year to consolidate what is an interaction
- … If you’re trying to do a right-click context menu, won’t work right now, but we could in the future
- … Testing just clicks and interactions that update URL
- ... LCP useful, but we could report everything, visually complete
[a]@noam.helfman@gmail.com can you complete what you said here?
[b]@mmocny@google.com can you take a look?