Blink Animations:�The Interpolation Stack

Casually responsible for over half the files in core/animation

smcgruer@, 2019/08/13

Disclaimer

1. This is all code archaeology; I didn't write the interpolation stack.
2. It's very possible there's documentation somewhere I missed.
3. This is going to be pretty dense. I hope you brought coffee.

I wasn't kidding about the number of files

$ find renderer/core/animation/ -name *.cc -o -name *.h | wc -l

264

​

$ find renderer/core/animation/ -name *.cc -o -name *.h | xargs wc -l

...

41426 total

​

$ find renderer/core/animation/ -name *interpolation* | wc -l

142

​

$ find renderer/core/animation/ -name *interpolation* | xargs wc -l

...

16229 total

So what is interpolation all about?

​

• Animations on the web have keyframes, which tell us what the value of the animation is at specific points in time.�
• But in-between those points, we have to interpolate between pairs of keyframes.
• (1 - f) × A + f × B ⇒ interpolated value�
• Easy, right?

The problem with animating CSS (and SVG)

​

• There are a lot of CSS properties (not to mention SVG attributes!)

The problem with animating CSS (and SVG)

• Some properties are easy to interpolate.

The problem with animating CSS (and SVG)

• Some properties cannot be interpolated!

font-family: serif;

font-family: emoji;

@ 50%

font-family: ???;

The problem with animating CSS (and SVG)

• Some properties are tricky to interpolate.

visibility: visible;

visibility: hidden;

transform: rotate(180deg) translateX(20px)

@ 50%

transform: scale(1.5) rotate(90deg) translateX(10px)

transform: scale(2) rotate(0);

Blink Interpolation's big secret

• The 'core' interpolation stack only knows how to interpolate numbers and lists.
• See InterpolableNumber::Interpolate and InterpolableList::Interpolate.�
• All CSS and SVG types are broken down into some structure of InterpolableNumbers/InterpolableLists.
• After interpolation, they are built back into CSS/SVG values to be applied.�
• The logic for doing this is owned by the (many) subclasses of InterpolationType.

Some simple examples

How many pixels?

How many percent?

Getting into the weeds

Diving right in

• Starting at the top, each KeyframeEffect has a KeyframeEffectModel representing all of its keyframes.�
• A KeyframeEffectModel has an InterpolationEffect, which stores precomputed Interpolations for each adjacent pair of keyframes in the animation.�
• There are two types of Interpolation; InvalidatableInterpolation (used for CSS and web animations) and TransitionInterpolation (used for CSS transitions).
• For this talk, we only care about InvalidatableInterpolation.

InvalidatableInterpolation (1/4)

• The obvious method to look at is Interpolate.
• Called whenever we 'sample' a KeyframeEffect, either at the beginning of frame or when style is cleaned and the animation is outdated. �
• Not so fast!
• Most of the time, we defer actual interpolation to later in Style calculation, in a method called EnsureValidConversion.
• The exception is if we already have a valid 'conversion' for the current fraction.

InvalidatableInterpolation (2/4)

• So what is a 'conversion'?�
• InvalidatableInterpolation knows about PropertySpecificKeyframes (a keyframe representation with only one CSS/SVG property in it).�
• These are converted to TypedInterpolationValues.
• Consist of an InterpolationType (remember that?) and an InterpolationValue.
• An InterpolationValue is a pair of an InterpolableValue and a NonInterpolableValue (remember those?).

InvalidatableInterpolation (2/4)

TypedInterpolationValue

InterpolationValue

InterpolationType

InterpolableValue

NonInterpolableValue

PropertySpecificKeyframe

EnsureValidConversion

CSS/SVG Value

InvalidatableInterpolation (3/4)

• EnsureValidConversion: it all depends on the current fraction.�
• 0 or 1? Take the start/end keyframe of the pair, find an InterpolationType for it, and ask the InterpolationType to break down the CSS/SVG value.
• There must be at least one matching type, but there can be more than one!�
• Otherwise, take both keyframes, and try to find a matching InterpolationType.
• There might not be one that can handle both keyframes!
• This produces a PairwisePrimitiveInterpolation or FlipPrimitiveInterpolation, and immediately interpolates it.

An aside: InterpolationEnvironment

• How do we know what the right InterpolationType is for a CSS/SVG property?�
• This is the job of the InterpolationEnvironment.
• As usual, there are subclasses for CSS and for SVG.
• For CSS, the subclass returns a CSSInterpolationTypesMap, where you can find the mapping logic from CSS property to InterpolationType subclass(es).�
• The InterpolationEnvironment also stores other data necessary for the related InterpolationTypes to do their job.
• For example the CSS one stores the StyleResolverState, etc.

InvalidatableInterpolation (4/4)

• Recap: We have a TypedInterpolationValue, representing some fraction between two property-specific keyframes.
• The property-specific keyframes have (if possible) been broken down into InterpolableValues.
• If the fraction was in (0, 1), the pair of InterpolableValues have been interpolated into a single resultant InterpolableValue.�
• It's time to apply our interpolation back to CSS/SVG!

Applying interpolated values

• Again this is the responsibility of the InterpolationType classes.�
• For CSS, CSSInterpolationType overrides InterpolationType::Apply, and defines an ApplyStandardPropertyValue method for its subclasses.�
• How complex this is all comes down to the specific type.
• CSSFontWeightInterpolationType - InterpolableNumber to blink::FontSelectionValue
• CSSColorInterpolationType - nested set of InterpolableLists for visited/unvisited, then various embedded values (R, G, B, A, currentcolor, …).
• CSSTransformInterpolationType - this one is special...

CSSTransformInterpolationType

• Turns out transforms are complex to interpolate.
• "rotate(90deg) scaleX(2)" and "scaleX(2) rotate(90deg)" have different semantics.
• So what does the author want if they interpolate between these?
• Spec says: pad the lists to the same length with matching identity functions, then do pairwise interpolation - falling back to matrix interpolation.
• How do we represent this as an InterpolableValue? We cheat!
• Store + interpolate an InterpolableNumber, which is just the progress fraction.
• During ApplyStandardPropertyValue, retrieve the progress fraction and call TransformOperations::Blend to do the complex work.

Finished... right… ?

• Nope! What happens if you do:
• target.animate({width:['0px', '10px']}, 1000);
• target.animate({width:['50%', '100%']}, 1000);
• This is actually simple; the second effect just replaces the first.�
• But the Web Animations API also allows us to composite animations:
• target.animate({width:['0px', '10px']}, 1000);
• target.animate({width:['50%', '100%']}, {duration: 1000, composite: 'add'});
• This gets trickier; we need to be able to combine the outputs.

Additive composition, by the spec

• Let's break down our example.�
• {width: '0px'} => {width: '10px'}, @ 0.2 (linear)
• Produces width: 2px
• {width: '50%'} => {width: '100%'}, @ 0.8 (linear)
• 0.2(2px + 50%) + 0.8(2px + 100%)
• 0.4px + 10% + 1.6px + 80%
• 2px + 90%�
• Notice anything interesting?

Compute underlying value first

Then apply it to both keyframes before interpolating

Spec authors hate this one weird trick!

• (U + A)(1 - f) + (U + B)f
• ⇒ U(1 + f - f) + A(1 - f) + Bf
• ⇒ U + A(1-f) + Bf�
• This holds even if only one of the keyframes is composite: add!
• add, replace → multiply U by 1-f
• replace, add → multiply U by f
• replace, replace → multiply U by 0�
• But, it also only holds if the operations involved are proper addition.
• So it fails for things like, oh, CSS transform lists (where have we heard that before…)

We call these the 'underlying fraction'

Additive composition, in the code

• The first step is to determine what interpolations we have to care about.
• EffectStack::ActiveInterpolations sorts the effects into the correct 'composite order' (it's a spec thing), then calls CopyToActiveInterpolationsMap.
• The copy logic is meant to discard the appropriate effects, and leave only ones we need to care about. Making this presentation, I found a bug!

composite: 'replace'

composite: 'add'

effect_3

effect_4

Additive composition, in the code

• Next: InvalidatableInterpolation::ApplyStack takes the set of relevant interpolations and resolves them to a single value.�
• Important concept here: the UnderlyingValueOwner
• The 'U' in each calculation of U + A(1-f) + Bf�
• ApplyStack performs the U + A(1-f) + Bf behavior for each interpolation.
• For each interpolation (reminder: keyframe pair), we call InterpolationType::Composite to combine its (already computed!) interpolated value with the underlying value.

InterpolationType::Composite examples

• CSSBasicShapeInterpolationType - check shapes are compatible. If so, scale underlying by underlying fraction and add new value. If not, replace underlying with new value.
• Many types operate like this - scale and add if compatible, else replace.�
• CSSTransformInterpolationType
• As mentioned, here we cannot apply our mathematical trick.
• Instead, we apply the underlying value to each keyframe (if additive), then interpolate the results at the appropriate progress.
• This is the full (U + A)(1-f) + (U + B)f behavior.

Things I Didn't Mention (but probably should have)

• Partial keyframes, aka animate({width: '100px'}, 1000)
• Insert the missing keyframe with an additive composite mode and a 'neutral value for composition' (aka an identity function). This causes it to pick up the underlying value.
• ConversionCheckers
• We cache the conversion from PropertySpecificKeyframes → TypedInterpolationValue.
• But as the underlying value changes, the conversion for a given keyframe pair might no longer be valid.
• ConversionCheckers exist to invalidate the cache before we try to use it.
• There are a lot of these checkers, and they are sadly poorly documented. I'm sure it was obvious at the time :(.
• UnderlyingValueOwner is actually a subclass of an 'UnderlyingValue' hierarchy!
• I only discovered this an hour before this presentation.
• I have no idea what the hierarchy is for.

​

Open Questions

• Should we bother with InterpolableValues at all?
• We could just ask the InterpolationTypes to perform interpolation when required, operating on their underlying data structures.
• Do we really need all this caching?
• Would the code be simpler without it?
• Is the mathematical trick 'worth it'?
• Harder to map to the spec.
• There are (a few) more exceptions than just CSS transforms!

The End

(Congratulations.)