WebPerfWG call - August 1 2024

Participants

Nic Jansma, Yoav Weiss, Rick Klein, Andy Luhrs, Patrick Meenan, Noam Helfman, Sean Feng, Aoyuan Zuo, Andres Olivares, Bas Schouten, Benjamin De Kosnik, Michal Mocny, Barry Pollard,

Admin

Next call - Aug 29 1pm EST / 10am PST (skipping Aug 15)
Still looking for ideas for TPAC - put them on the agenda

Minutes

Add a non-normative format for the detail param that allows developers to customize the presentation of timings in devtools (UT 111)

Andres: working on Chrome perf tooling, lighthouse, devtools
… Received plenty of requests to allow users to customize how user timing is displayed in devtools
… also allow users to extend the performance panel with their own data
… UT has been very popular to do this, so we thought it can be extended and allow to add richer data to it
… Decided to go with that instead of an API for devtools extension and browser extensions as it’s more cumbersome
… Since UT has a details field we thought we could use that to allow users to pass metadata and then interpret and show it in devtools
… We have a doc on it TODO: link
… Expecting a devtools object defined in details with this TS object
… you can use the datatype to specify track entry and allows us to add more features in the future
… can determine color, track name, subtracks through track group, properties with extra details that’s shown when focusing, as well as tooltip text.
… All of it is optional
… Chatted with frameworks that are likely to use this, and sent PRs to angular and react implementing this, to make their data visible in devtools (instead of forcing devs to install framework-specific extensions)
… avoids context-switching between perf panel and the framework-specific tools as well
… This needs to be performant as it’s code in hotpaths. At the same time, perf.measure and perf.mark have an overhead and have been traditionally slow
… That’s due to the semantics - they add an object and queue an entry
… Hard to optimize this to get frameworks to adopt this in production builds, so we don’t expect it to get adopted as is in production builds
… It’s still very valuable and useful
… The Angular PR recently landed, but it’d be more useful for webdevs if this was available in other browsers
… So wanted to kick off the standard discussion around this
Bas: you mentioned the concern of mark/measure in hot paths. I understand that you want to add custom start/add timestamps.
… If there was a version of the API that had start and end timestamps, we could have avoided that performance overhead.
… Both chrome and firefox take timestamps in critical places, so that’s fine
Anders: This API is intended to be plugged in into the performance timeline. There’s machinery that’s not needed for devtools
Bas: And privacy/security concerns are not the same. Not concerned about the precision of the timestamps in the same way
Anders: there’s `console.timestamp` and it’s specified to specifically add a timestamp to the performance tool
… If you’re not profiling, it could be zero cost
… This might be a better option for that perspective
… We could standardize that, but we should see in the next topic if this all converges
… So maybe we shouldn’t standardize the format, but just say that devtools can use the timestamps already used in UT
NoamH: wonder if it’s practical for this property only be interpreted when the profiler is on. Also the profiler has options to disable things - maybe this can be the same
… Developers may want more granular control RE their willingness to observe this
Michal: Starting to get into the next conversation. The producer is passing the data and if the consumer doesn’t need the data we don’t need to do the work. But perf timeline is a consumer
… Maybe the entry creator can say this is not intended for buffering
… But it may not be as good as it can get using a different approach
Yoav: Seems we want slightly similar things for DevTools and UserTiming, but semantics are different i.e. for some things devs don't want UserTiming entry, but they do want DevTools to show if profiling is on
... For other things we want UserTiming entries, but we don't want them in the PerformanceTimeline. We've had requests in the past where users could filter different types of entries in DevTools.
... If you have these timestamps in React/Angular, developers who have both on their page may only want to see one or the other.
... I think we should carry on with the next session, but I think there's a chance for us to create at worst, two different APIs with similar signature, but it wouldn't be the same API
Sean: Want to register personal concern, do we have anything similar for other web-exposed APIs? i.e. into DevTools? Seems unusual
Andres: Performance Panel you have stuff, console APIs, but you don't have control over the presentation. UserTimings mark/measure add an entry to Performance Panel.
... You can also use console.time and .timestamp
... You can also log a table
... There's also Stack Tagging API that adds better support for async stack traces in Sources panel
Sean: For Mozilla, generally we have an API, and browser vendors decide what to show up in DevTools. Vendor choice. For UserTiming, all vendors are showing something in DevTools. Maybe there's a browser that doesn't want to show it. Feels unusual.
Andres: Implementation detail in the end, but I thought if other browsers added support for this, it would allow other players in the ecosystem to use the API, and for developers to benefit.
... Don't need to go back and forth between custom tooling and dev tools
Michal: Plenty of good precedent here
... Developer tools give much more insight when using locally, observing timeline
... Things that wouldn't be available to the site. We have Network timing, in rich detail, and the site can include cookies and headers that aren't observable by site in the field, and it's rich in information.
... PerformanceTimeline has plenty of info. Way for developers to add more annotation. We don't want to stuff PerformanceTimeline. Could be used to augment Performance Panel.
... Chrome Dev Tools team will use these entirely optional thing. Seems less optional than defining what that list should be
Bas: I think in general this is very appealing
... When framework creators have to make specific tools for specific browsers
... Advantages the largest browsers first
... I understand Sean's concern.
... I don't see anything from perspective of Firefox that wouldn't line up with how our Dev Tools show information
... What if I'm a new UA and I'm creating a new Performance Panel, how well would this match up with that? To me that seems more of an intellectual concern than a practical concern.
Yoav: What is being proposed is not a user-facing feature, it's a developer-facing feature
... In terms of interop, no concerns impacting users
... At worst, this will cause some developer grumpiness when using Dev Tools of one browser than another
... Significantly less severe than anything else happening in web standards

[LoAF] Allow for annotations for long animation frame to further help debugging (LOAF 3)

Michal: a different issue that came from a separate place. Presenting in-lieu of NoamR
… LOAF#3 so it was pretty obvious that this is a LoAF issue
… When you have a LoAF it would report the individual scripts that were long.
.. . <quick demo>
… two functions that take 500ms, LoAF of 1000ms is reported, and the LoAF entry contains those 2 functions
… LoAF only reports for frames above 50ms and individual scripts need to take more than 5 ms. Reporting is per individual scripts
… It’s reporting tasks inside the frame - calling it a script is a misnomer
… e.g. invoker was an event-listener
… But if we inject a wrapper library around event listeners, which is something that a lot of RUM libraries do, the attribution goes to the wrapper, which did no work, but it became the new entry point
… That kinda breaks LoAF attribution, which is a huge issue with LoAF usefulness
… Also common with how react does event scheduling, so it’s very common
… So developers would e.g. blame react/RUM when it’s their code that’s at fault
… We need a way to annotate that this is just a wrapper
… If I’m a developer, I want to tell the browser what function to measure
… Maybe we need a dedicated API that would trace time spent in functions and mark timelines that are not function scoped
… And when the consumers are not running, this would not have no overhead
… So devtools would be one consumer, but LoAF *in the middle of a long frame* would be another
… Proposes a new type of object called a PerformanceTrack
… There should be low overhead so no reason not to use it
… It’s a mini-perf-timeline, but entries are not buffered in the regular per timeline
… relevant entries can buffer and report them in their corresponding entries
… TODO
…
… We could have marks and measures, but we can bikeshed the specifics
… Maybe add a console API that would enable us to add the track with details
… and potentially bind the track to specific functions, and they’d get included as an entry point in LoAF
… Every observer may need to specifically attach a track they are interested in, to reduce overhead. But it’s unclear if this is needed.
… Benefits: doesn’t create a dictionary, created a natural namespace
… having this on performance can confuse developers and it’s not added to the timeline
… using bind doesn’t guarantee attribution because it doesn’t include microtasks
… using console apis to augment this could be confusing
… Even for EventTiming we heard folks wanting to know what was the long interaction, where wrappers got in the way
… In Chrome if you also have LoAF it has all the data, but that’s not always the case.
Bas: One question is the `bind` part. C++ does that all over the place. I suspect there some overhead there in terms of implementation.
… Would that be harder to optimize for JS engines compared to start/stop function calls that won’t involve extra stack instrumentation
Michal: Maybe you have measure start and end, but you could imagine this bind function as a wrapper that marks the start and end
Bas: If it’s defined as a wrapped maybe there are different implications
Michal: Noam felt that this would enable stronger attribution without regressing performance
Jared: OP. Measure is cool, but even that is more than we need. We just need a way to say that this came from “doSomething” or “doSomethingElse”.
… So we just need to give the entry the right name
Michal: Let’s say from the field you’re getting a LoAF attributed to wrapper, but if you know feature was marked that’s enough? That’s good feedback
Jared: For us the top attribution is react render, which is not useful
… but knowing which component caused this would help me figure it out from there
Michal: That’s exactly how this connects to the previous discussion. TODO vs. in the field you’re constrained in how you measure but TODO
Barry: Wrappers are not always innocent. So we should probably already keep their timing
Michal: It’d be neat to know if all the time was in the component or in the framework handling it. True.
NoamH: For the bind function, does it make sense to unbind in case you want to stop tracking? (in case it generates too much noise)
Michal: bind returns a new function reference, but you still have the original function reference. So the developer can decide if they want to call the bound one or the original one
Noam: But that’s harder ergonomically as you need to make a code decision for it. Stopping binding sounds easier
Michal: The intention was for the consumer to decide if the functions should be bound. If profiling is enabled the TODO
NoamH: seems like there’s some overlap with usertiming. Why wouldn’t we want to use performance mark and measure?
Michal: adding to the buffer has memory and processing overhead. The details field is a dictionary that gets GCed
… We had older issues where developers were using UT in order to reflect them in devtools
NoamH: Could we extend UT to do this without the overhead?
Michal: Maybe we can constrain the API somehow..
Bas: Advantage of the API being different is to allow different attribution functions TODO
… Would attribution always go to the lowest stacked frame if this is nested?
Michal: I’m not sure.
Bas: This is time “bla” and this is its parent “foo” would enable to extract more times, but it complicates things