Allow animations to be given a target framerate

[jakearchibald]

Problems:

We're seeing more devices that have screen refresh rates of 90, 120 and 144hz. Apple in particular are worried that allowing higher rendering rates by default will unnecessarily drain battery life. As such, on their 120hz devices, requestAnimationFrame is throttled to 60hz, whereas CSS animations run at 120hz. This doesn't really fit with the current spec, which expects CSS animations and requestAnimationFrame to run at the same rate.

On iOS native, a view can hint towards a particular framerate, and this defaults to 60hz, so developers must opt into higher rates.

It's currently difficult to lower the frame rate of an animation for stylistic reasons, eg 12fps for 'drawn' animation, or 24fps for film. With requestAnimationFrame you have to deliberately 'do nothing' for some callbacks, or try to wrap setInterval around requestAnimationFrame. CSS animations provide step(), but you can't mix this with other timing functions.

It's currently difficult to lower the frame rate to achieve a consistent rate across an animation. Eg, animation at 30fps rather than fluctuating between 30 & 60. With requestAnimationFrame you have to deliberately 'do nothing' for some callbacks. CSS animations don't provide a method here, although the browser can do it automatically.

It's difficult to synchronise animations that run on the compositor thread with animations that run on the main thread. If the main thread becomes busy, the main thread animation will have a different frame rate to the compositor-driven animation. This is usually ok, and maybe not something we should try to solve.

Goals:

• Allow developers to set a framerate for a set of animations.
• These animations may be of different types. Eg CSS, web animations, frame callbacks, animation worklets, SVG animations.
• The framerate may be a hint, which may be snapped to a multiple of the refresh rate. Eg, a desired rate of 30fps may become 25 on a 50hz screen.
• The framerate may not be a hint. 24fps may have been chosen to synchronise with a 24fps video.
• Different sets of animations may have different rates within the same document. A cartoon animation may run at 12fps, whereas other UI may be 60fps.
• The rate of animations may be changed during the animation, in reaction to an inconsistent rate.

Twitter thread: https://twitter.com/grorgwork/status/1185258811109433344

[jakearchibald]

Web Animation's DocumentTimeline feels like a good home for this.

It could be given properties for the target framerate, and whether it's a strict rate. These values could be changed throughout the life of the timeline.

const timeline = new DocumentTimeline();
timeline.targetFPS = 12;
timeline.strictFPS = true;

The timeline would provide a way to get a callback the next time its time updates. We could maybe explain requestAnimationFrame on top of this.

timeline.onFrame(() => { … });

// Maybe:
function requestAnimationFrame(cb) {
  document.timeline.onFrame(cb);
}

Developers could create additional timelines if they want groups of animations to have different rates.

const cartoonTimeline = new DocumentTimeline();
timeline.targetFPS = 12;
timeline.strictFPS = true;

const smoothTimeline = new DocumentTimeline();
timeline.targetFPS = 120;

document.timeline is the default timeline, so changing rate values here would cover all animations on the page, unless they were specifically put into a different timeline.

document.timeline.targetFPS = 90;

Browsers would be able to give default values to document.timeline, or even change the values unless they were explicitly set by a developer. This means a browser could limit animations to 60fps on a 120hz device, or even throttle to 30fps in battery saving modes.

This doesn't cater for the current iOS behaviour where CSS animations run at a different rate to rAF, but maybe that behaviour should change?

cc @birtles.

[AshleyScirra]

It would be good to also have a way of saying "use the display rate of the device" instead of having to hard-code an arbitrary FPS in that case.

[rniwa]

Not endorsing or opposing this idea but in the case of matching movie frame rate of 24fps, we’d also need to take motion blur into account:
https://www.red.com/red-101/shutter-angle-tutorial
https://beyondthetime.net/cinematic-motion-blur-180-rule/

[jakearchibald]

@AshleyScirra a target rate of Infinity should handle that.

[jakearchibald]

@rniwa there's some discussion over at w3c/csswg-drafts#3837.

Edit: But I think the motion blur stuff is mostly separate. Some things benefit from blur, but if the style is 'cartoon', you don't want traditional motion blur there.

[birtles]

CSS animations provide step(), but you can't mix this with other timing functions.

This particular part can be done with GroupEffects (where you put the step timing function on the parent, and the other timing function on the child). That said, that only helps Web Animations (and presumably CSS animations in future) so it doesn't help with rAF or CSS transitions so I think this proposal makes sense. DocumentTimeline does feel like a suitable place for this.

A slightly related feature which has been come up and which may be worth considering at the same time is supporting frame-based animations, that is, animations where animation time progresses at a constant interval, regardless of the actual elapsed wallclock time between frames (i.e. prefer slow-down over dropping frames). For example, see "Duration based vs frame-count based animation" from:

• https://product.voxmedia.com/2013/11/25/5426880/polygon-feature-design-svg-animations-for-fun-and-profit

For that use case presumably the author would not want the animation to run faster on devices that can animate above 60fps. This might suggest an API like targetFPS but rather than having strictFPS, have an enum value that has different modes?

[Yay295]

For that use case presumably the author would not want the animation to run faster on devices that can animate above 60fps. This might suggest an API like targetFPS but rather than having strictFPS, have an enum value that has different modes?

Something like timeline.fps instead of timeline.targetFPS and timeline.fpsMode instead of timeline.strictFPS, and then timeline.fpsMode could be one of strict, max, min?, etc.

For the syncronizing-with-video use-case there's this proposal, though it hasn't had any activity since July.

[jakearchibald]

@birtles

A slightly related feature which has been come up and which may be worth considering at the same time is supporting frame-based animations, that is, animations where animation time progresses at a constant interval, regardless of the actual elapsed wallclock time between frames

Interesting! I think something like that is complementary, but seperate. As in, it shouldn't be a goal here, except to ensure we don't prevent it from happening later. It feels like this feature would be a new unit for animation-duration. So instead of 3s it could be 200frames or whatever.

[jakearchibald]

There are a couple of things missing from my proposal:

• element.animate doesn't allow animations to be put on another timeline. So all animations will end up on document.timeline. You could solve this by adding an option to set the timeline, like you can when constructing an Animation. But maybe there's a reason this isn't already there? (@birtles?)
• There's no way to set the timeline for a CSS animation. There's already a proposal for animation-timeline, but it would need a way to access a developer-created timeline. That probably means we'd need some way to define a named timeline in CSS, similar to how @keyframes works.
• Opting into higher framerates requires JavaScript. Given that JS isn't required for animations, it seems sad that opt-in is JS only. I guess we'd also want a CSS way to influence the document timeline.

However, even without the above, we have a way to explain how browsers throttle the default timeline, and it provides a way for developers to opt into higher framerates.

[flackr]

You can approximate custom frame timing using animationiteration events: https://jsbin.com/cagehit/edit?js,output

I don't have a 120Hz iOS device to test whether the above demo allows JS 120Hz animations but I believe this also has the nice timing characteristics of firing at the same part of the lifecycle as requestAnimationFrame due animation events being fired when animations are updated: https://www.w3.org/TR/web-animations-1/#update-animations-and-send-events

[Yay295]

I have a 144Hz monitor. Your flicker() code was off by 1000 (seconds instead of ms), but otherwise it works. I added a FPS counter to the log: https://jsbin.com/bizuwuleja/edit?js,output

[birtles]

@jakearchibald

Interesting! I think something like that is complementary, but seperate. As in, it shouldn't be a goal here, except to ensure we don't prevent it from happening later.

Agreed. I think the frame-based animation feature boils down to two things:

a. A minimum interval in wallclock time between frames (i.e. maximum frame rate). This is the part that overlaps with this issue.
b. Causing the reported timeline time / rAF time to increment by exactly the interval specified in (a). This is orthogonal.

I believe both those things should happen on the timeline, not as units on an animation.

• element.animate doesn't allow animations to be put on another timeline. So all animations will end up on document.timeline. You could solve this by adding an option to set the timeline, like you can when constructing an Animation. But maybe there's a reason this isn't already there? (@birtles?)

Correct. element.animate() was introduced as a shortcut to create an animation that (a) is playing, (b) has a KeyframeEffect as its target effect, and (c) is attached to the DocumentTimeline.

If you want to do anything different to that you either need to use the Animation() constructor or mutate the Animation returned from element.animate().

It's easy enough to add parameters to KeyframeAnimationOptions if it proves necessary, however. (If the parameter takes an arbitrary timeline, as opposed to a framerate, it might take a little more work because we need to work out how to handle inactive timelines in that case.)

• There's no way to set the timeline for a CSS animation. There's already a proposal for animation-timeline, but it would need a way to access a developer-created timeline. That probably means we'd need some way to define a named timeline in CSS, similar to how @keyframes works.

This is possible. We've talked about introducing @timeline in the past and I believe a version of the scroll-linked animations spec might have included it at one point.

[mdrejhon]

Founder of Blur Busters / Inventor of www.testufo.com here.

Lone Standout Situation (Apple): animations stuck at 60fps on 120Hz iPads

Earlier, W3C accepted my commit for fps=Hz requestAnimationFrame() in HTML 5.2 section 7.1.4.2. However, it appears WHATWG does not have a firm consensus about this behavior, creating a lone-standout problem (Apple 120Hz iDevices don't work properly with www.testufo.com ) when everything else works (Microsoft, Google, Samsung, OnePlus, HTC, Opera, FireFox, etc).

FAIL:

• Apple Safari on 120Hz iPads (WebKit)
• Old Microsoft Edge (old EdgeHTML)

PASS:

• Chrome (all; mobile and desktop)
• FireFox (all; mobile and desktop)
• Opera (all; mobile and desktop)
• Edge (all; desktop and tablet)
• Chromium (all popular forks)
• WebKit (most non-Apple forks)

Even the new Samsung 120 Hz phones (S11/S20) works perfectly in TestUFO (Google Analytics shows visits going green-120Hz from beta phones already). All other non-Apple high-Hz phones work.

I have submitted a WHATWG issue:

whatwg/meta#158

Everything that animates need to consider high Hz, much like all APIs had to be updated to be retina-resolution-aware. CSS animations cannot serve all purposes, there's some very, very good reason (many unrelated to me or Blur Busters too!) that high-Hz support needs to be inclued in requestAnimationFrame() too.

Defacto Standard of High-Hz requestAnimationFrame()

Fixing this rAF() standardization hole at WHATWG is becoming urgent because of the upcoming boom of mainstream-brand 120 Hz smartphones (next iPhone and next Galaxy both support 120 Hz).

As a reminder for fps=Hz requestAnimationFrame()

Works -- All current versions of all desktop browsers
Works -- Razer 120 Hz phones
Works -- Pixel 90 Hz phones
Works -- OnePlus 90 Hz phones
Works -- Xiaomi 144 Hz phones
Works -- Samsung 120 Hz Galaxy (Google Analytics shows Samsung useragents green at 120Hz!)
DOES NOT WORK -- Apple 120 Hz iDevices

Need Solution Before 120 Hz Mainstream Boom

120 Hz becomes much more mainstream with upcoming 120 Hz iPhones and Galaxy launches this year. Even a temporary solution is urgently needed given the emerging tsunami of 120 Hz.

Before the 120 Hz iPhone arrives, maybe a temporary Safari-specific "vendor prefix" solution should be delivered as soon as possible (e.g. custom parameter like suggested) if Apple prefer to follow a differnt path to enabling 120fps with requestAnimationFrame()

[MarjaE2]

At a user end, user css options to throttle frame rates would help motion-sensitive users avoid motion sickness.

Currently Firefox has a browser-level option, but it only goes to 1 frame/second, which is too high for some sites (Wikipedia since the Vector redesign, the Internet Archive since the latest redesign, MDN Web Docs, etc.) and too low for others (for Youtube on videos in safe serieses).

A user css option would allow different frame rates on different sites.

[mdrejhon]

At a user end, user css options to throttle frame rates would help motion-sensitive users avoid motion sickness.

Currently Firefox has a browser-level option, but it only goes to 1 frame/second, which is too high for some sites (Wikipedia since the Vector redesign, the Internet Archive since the latest redesign, MDN Web Docs, etc.) and too low for others (for Youtube on videos in safe serieses).

A user css option would allow different frame rates on different sites.

I agree too.

I'm a fan of Hollywood Filmmaker Mode 24fps 24Hz

Both flexibilities are needed. As long as users can set frame rates higher too.

Higher Hz can (for others) be ergonomic/assistive benefit

Give users the choice. The motionsick double-edged sword goes both ways.

Some of us also get motion blur headaches, so some of us need higher frame rates (too) as an ergonomic or assistive benefit. Motion sickness can go down with higher frame rates, because for some people the headaches keys on the motionblur, rather than from vertigo.

That's why Apple Vision use pulse-based motion blur reducton to simulate 500Hz via BFI (2ms flashes), because in VR, motion blur creates headaches. But displays are getting bigger and even direct-view 2D display motion blur can cause problems. So framerate/Hz based motion blur reduction (via brute Hz) is the ergonomic benefit for some of us.

Do not neglect BOTH ends of the spectrum please.

1. There should be browser configurability to change framerate cap (but unlimited configurability)
2. There should be css framerate decimation features for "Reduce Motion" features.

Effective framerate would be max(item1,item2). However, in all cases, there should be a way to discover the display refresh rate much like there is a way to discover the display resolution. That way, I can at least inform the user that TestUFO motion tests are not running at native refresh rate. Discovery is important.

Permission API or Same-Origin for Embed Animations

I have a refresh rate detector that embeds www.testufo.com/refreshrate into the middle of a page at https://blurbusters.com/oh-no-im-at-the-wrong-refresh-rate/ but it only works on Chrome & Opera & Edge & Brave & FireFox but not Safari. I own both blurbusters and testufo and should have the right to uncap frame rate (it was for slowing down ad banners, but unintended consequences). I can't even properly discover refresh rate on Safari, so, I can't even inform Safari users.

Safari 120Hz+ Quirks Note

BTW, Apple Safari now supports 120fps on MacOS and iPadOS via enabling Developer flags (sometimes not needed) and configuring "Prefer Page Rendering Updates near 60fps".

Now, it's not all unicorns or rainbows:

• Safari on iPhones don't let you do 120fps (okaaay...)
• MacOS Safari won't go above 120fps even on a 144Hz gaming monitor (stutters really bad)

It seems like there's still an internal 120fps cap in Safari, when disabling "Prefer Page Rendering Updates near 60fps".

I am launching TestUFO 2.0 HDR

This will quickly increase popularity of TestUFO among Apple users, as I also have various HDR tests for DCI-P3 and rec.2020

See my sneak pre-announcement at https://github.com/blurbusters/testufo-public#readme - it even supports brighter-than-#FFFFFF for display testing purposes.

cc: @jakearchibald (please distribute internally)