Smoothness Reporting for Animations and Scrolling

Michal Mocny (mmocny@google.com), TPAC 2020

Why?

Animations and Scrolling are a big part of UX!

“Stuttering” tends to be a very visible problem to users, leaves a bad impression.

Today: hard for developer to know when, what, why hiccups happen

Topics covered

• How exactly do we want to measure smoothness?
• Options for reporting the data
• A few Open Questions

Measuring Smoothness

Measuring Smoothness

​

​

Folks tend to think of Smoothness in terms of FPS, but this has some flaws:

• A perfectly static website is perfectly smooth, even when not generating any new frames.�
• With multi-threaded rendering, separate system compositors, variable screen refresh rates, etc, what does FPS even measure?
• Showing stale frames on time is still bad…
• Partial updates...

Measuring Smoothness

​

​

What really matters:

Missed opportunities to show expected animation updates

​

Aka: “Dropped Frames”

What do we consider Animations?

• Animations:
• Scroll
• Pinch/Zoom
• JS driven Animations (i.e. rAF loops, or repeated updates to animating properties)
• CSS & Web Animations
• ...Canvas, Video, Continuous Input handling...
• Not animations:
• Insertion of new content
• Clicking on a button takes a long time to produce UX in response
• Form inputs appearance changes, responding to input
• Background loading

Defining: Dropped Frames

• When an Animation is expected to produce an update for a vsync, yet the page (or the browser) does not do so, we have a dropped frame�
• Animations are not always expected to produce an update for every vsync
• For scrolling, we expect to update the scroll position to a certain point with each frame

Examples...

rAF animation that conditionally updates

var frameCount = 0;

function animationStep() {

if (++frameCount % 2 == 0) {

anim.style.transform = 'translate(' + frameCount + 'px)';

}

requestAnimationFrame(animationStep);

}

requestAnimationFrame(animationStep);

CSS animation with idle periods defined

<style>

div {

[...]

animation: mymove 5s infinite;

}

@keyframes mymove {

0% {transform: translate(0px)}

25% {transform: translate(100px)}

75% {transform: translate(100px)}

100% {transform: translate(200px)}

}

</style>

Scroll-linked animations

document.body.addEventListener('mousewheel', () => {

const now = new Date();

while ((new Date() - now) < 1000) {}

}, {passive: false});

​

​

​

​

window.addEventListener('scroll', () => {

const now = new Date();

while ((new Date() - now) < 1000) {}

document.querySelector('#top').style.top =

window.scrollY + 'px';

});

Here, a non-passive listener delays scroll start… but scroll is smooth otherwise.

Here, scroll handler is passive and so scroll itself is smooth, but the animation run within the handler is delayed.

Options for reporting the data

Diagram: Timeline w/ animations & scroll

Option 1: Report all the raw data points

Note: Fabricated data to serve as example. Reality likely much less dense.

Option 1: Report all the raw data points

Concerns:

• Ergonomics
• Performance (data sizes)
• Security / Privacy

Option 2: Report per animation (summaries)

• Since animations can be defined using different strategies, may start at different times, and in response to distinct events… it is convenient to just generate summary reports per animation.
• Focusing on the animation simplifies attribution
• The source of new smoothness problems is immediately apparent
• API strawman:
• For each animation, at the end of animation (or N-seconds after start)
• Report # expected frames, # produced frames, duration, as well as some metadata
• You can turn this into a %smooth, or an average effective FPS
• (I presented an earlier variant of this option at a Web Perf meeting this year)

Option 2: Strawman API

interface PerformanceAnimationTiming : PerformanceEntry {

entryType: “animation”,

type: <one of “css-animation”, “scroll”, “js-animation”, …>,

startTime: <when it started>,

duration: <how long did it last>,

interval: <average vsync interval during this animation>,

framesProduced: <# of frames produced during this animation>,

framesExpected: <# of frames that were expected during this animation>,

element: <the animating element, if/when possible>,

id: <the animation id, if present>

};

Option 3: Report per frame (on timeline)

• “Flatten” all animations along a single timeline, and report similar to the original Frame Timing API proposal (Note: data differs substantially from original proposal).
• In the end, this is a good overall summary of what the user experienced.
• API strawman:
• For each vsync:
• For all animations currently running, sum up #expected frames, #produced frames

Option 3: Strawman API

interface PerformanceFrameTiming : PerformanceEntry {

entryType: “frame”,

startTime: <when it started was presented, possibly needs fuzzing>,�updatesProduced: <# of animation updates produced during this frame>,

updatesExpected: <# of animation updates that were expected since the last frame>,

};

Note: may have 0 updatesExpected if there are no animations active.�Alternatively: could replace counts with state? COMPLETE, PARTIAL, IDLE

Option 4: Report a single summary

• Similar to proposal for Final-LCP or Final-CLS (?), report a Final-Smoothness
• Or, perhaps at key moments, and every time visibility changes?
• API strawman:
• Similar to Option 2, but the whole page is treated as an animation

Option 4: Strawman API

interface PerformanceFrameTiming : PerformanceEntry {

entryType: “animation”,

type: < “page” or “summary” >,

startTime: <when it started>,

duration: <how long did it last>,

interval: <average vsync interval during this animation>,

framesProduced: <# of frames produced during this animation>,

partialFramesProduced: <# of partial frames produced during this animation>,

framesExpected: <# of frames that were expected during this animation>,

};

Here we treat the overall page timeline as an animation, and report it using almost the same format as Option 2.

Mix and match?

• The options aren’t necessarily mutually exclusive.
• Could offer a single page-level summary, but still give access to details per frame or per animation (either via PerformanceObserver or some property on the PerformanceEntry)

Open Questions

Open Questions

• [How] Do other venders currently measure animation smoothness?
• Agreement on Dropped Frames perspective?
• Agreement on defining Animations?
• Especially, JS updates to animatable properties outside of rAF loops��
• Is “Dropped Frame” a boolean signal, a count, or does it need weights? (i.e. “impact region”)
• Smoothness during page load?
• We expect more smoothness hiccups during load. Should we report smoothness during load distinctly?
• i.e. Goal: minimize the time until page is smooth, then make sure it stays smooth afterwards?
• Can Responsiveness metrics track latency to the start of an animation?