Touch events while scrolling in chromium:
The throttled async touchmove model
rbyers@chromium.org - Apr 17, 2014
This document proposes to change how chromium sends touch events when touch-scrolling in order to improve website compatibility and give web developers additional control while avoiding any substantial performance impact in common scenarios.
Interaction with browser-native overscroll effects
Interaction with touch ACK timeout
Behavior in multi-touch scenarios
Browser behavior differs substantially in how touch events are handled when touch scrolling occurs. For details see “Touching while scrolling”. Chromium is unique in using the touchcancel model, in which the beginning of a scroll gesture triggers a ‘touchcancel’ event to be sent to the page, and the suppression of all touch events until all fingers have lifted. This model was chosen for maximum scrolling performance (little risk of accidental DOM modification etc. during a scroll) while still enabling applications to differentiate between a stationary finger and a scroll.
We’ve been close to changing this behavior previously (eg. see here) but hadn’t quite convinced ourselves that any of the options we considered were universally better. We attempted to ship a new model in Chrome 35 called “touchmove absorption”, but had to back off due to compatibility issues. This document describes an additional modification to touchmove absorption which should address the compatibility impact as well as enabling richer effects.
There are two main classes of problems with Chromium’s current touchcancel behavior. First, many websites (and Android WebView apps) are built for and tested exclusively with other browsers like Mobile Safari and the Android browser, and can be badly broken on Chrome. We’ve had a steady stream of complaints around this over the years (despite lots of evangelism), for example see:
The main classes of issues are:
Secondly, there are many common mobile UI effects which cannot be implemented in JavaScript in this model without re-implementing all of scrolling. Note that we want to focus on adding new platform primitives instead of specific browser features to implement each of these effects directly - see “A Rational Web Platform”.
Note that some of the advanced UI scenarios may require additional functionality that are orthogonal to the proposal in this document. For example, delivering ‘scroll’ events synchronously (which would apply after the finger is lifted for fling, and other input types). I believe the web compatibility benefits alone (goal #1) justify the change here, and so this work should proceed in parallel with efforts to fully address the advanced UI scenarios.
During active scrolling, touchmove events are throttled, sent asynchronously and exposed as uncancellable events to JavaScript. Otherwise touch events should fire as normal and synchronously determine when scroll updates occur.
The simplest and most powerful model is the synchronous touchmove model where every touchmove event is sent and only those that are not cancelled are used to generate a scroll update. This model is terrible for scrolling performance due to constantly blocking scrolling on the main JavaScript thread.
To avoid blocking when scrolling is occuring, we will send touchmove events that occur during active scrolling asynchronously with any scroll updates they trigger (as Safari and Firefox typically do). To make it clear to the application that these events are different and do no support preventDefault, we will set Event.cancelable to false. To reduce the risk of extra CPU consumption causing scroll jank, we will throttle these events using some heuristic (see below). This is similar to touchmove coalescing that we do in response to jank, except that we’re proactively avoiding jank in a perf-critical scenario.
The first touchmove in a touch sequence is always sent synchronously (to give the application a chance to suppress scrolling just before it starts). If activate scrolling is (or becomes) not possible, we will return to sending touchmove event synchronously. For example, if a user scrolls and hits the scroll limit then the browser will receive a GestureScrollUpdate event ACK marked NOT_CONSUMED and we’ll switch to sending subsequent touchmove events synchronously - giving the application a chance to control when scrolling may be resumed.
Note that I’m avoiding re-hashing all the details of how different browsers behave and their pros and cons (and why none of them are good enough). The details from the last time we had this discussion are captured here.
The throttling heuristic gives us a knob we can tune to trade-off richness and performance. It should be set high enough to not trigger scroll jank in common scenarios, but low enough that most existing code relying on touchmove events during a scroll functions adequately. This section describes an initial heuristic that seems reasonable, but I expect we’ll need to iterate and tweak it over time.
First, we’ll throttle (somewhat arbitrarily for now) to one touchmove per 200ms. Chance are we can go more frequent without causing issues (since the events are async), but to minimize the risk we’ll start conservatively. 5 touch events/second should be small compared to the ~60 scroll events per second we dispatch to JS during a scroll.
Secondly, many existing libraries attempt to detect tap gestures by watching for touchend that occurs before a touchmove beyond a short distance from the touchstart. For example, the popular FastClick library does this here using a default (but configurable) threshold of 10 CSS px, and mobile GMail was using a threshold of 12 CSS px (see bug 360614 for details). Therefore our heuristic will also send an async touchmove event whenever a 15 CSS px “application slop region” is exceeded.
What should happen when the first touchmove event in a sequence is consumed but latter events are not? The W3C TouchEvents specification says: “If the preventDefault method is called on the first touchmove event of an active touch point, it should prevent any default action caused by any touchmove event associated with the same active touch point, such as scrolling”. So we should probably not allow any scrolling in this case. This means an application wishing to handle touchmove events at the start but still retain the option of scrolling later in the sequence (eg. for pull to refresh) must take care to treat the first touchmove specially and avoid consuming it.
However, mobile safari already violates the spec here. For the (rarely used) case of an overflow scroll div without the ‘-webkit-overflow-scrolling: touch’ property, it allows scrolling to be suppressed from the start and then started later when touchmove events are released. If we decided it was important, we could probably do this as well without it being a major problem. See public discussion here.
Chrome on Android has an “overflow glow” which is drawn as part of overscroll. Other Chome platforms and Chromium builds have other effects. These effects are all triggered as a result of scroll update events that continue to fire after scrolling is no longer possible, and so are suppressed or re-enabled depending on the disposition of the touchmove events. However, depending on the precise timing, some amount of overscroll may occur before the page gets a chance to prevent it (see below).
The precise timing of switching between sending and absorbing touchmove events could be important. If we allow touchmove events to continue to be turned into GestureScrollUpdate events while we’re waiting for a GestureScrollUpdate ACK, then (if the renderer process is very busy) there could be an arbitrary amount of overscrolling that occurs before the application finally has the chance to suppress it. We’ll probably never completely solve cosmetic race-condition issues without letting the app opt-in to single-threaded scrolling, but we should do our best to reduce the impact of such issues in the default mode.
Therefore we should perhaps wait for any pending scroll update ACK before allowing a touchmove to be absorbed. This shouldn’t be a problem for performance because as soon as a touchmove is absorbed, it quickly turns into a GestureScrollUpdate event that’s blocked on any outstanding gesture event ACK, but it may be awkward to implement.
Even then, there will still be a minimum lag of at least one frame. The scroll delta of the event which first disabled absorption, as well as any unused delta in the previous (consumed) scroll event will still get counted as an overscroll. The common overscroll effects we use are subtle enough (or require a large enough trigger threshold) that this might not be a problem. If it is, then we could consider generating overscroll notifications only for scroll events that were generated from non-absorbed move events. To ensure there’s no gap between scrolling and overscrolling, we could keep track of the last absorbed touchmove and re-send it immediately upon learning that an overscroll event would have been triggered for it. This is all getting pretty complex though, and is perhaps best left to a single-threaded scrolling mode out of scope of this document (which will be necessary to give apps reliable control over fling behavior, for example).
Does consuming a touchmove that generates as scroll update cause the scroll delta to be lost permanently, or do the deltas still accumulate so that a future non-consumed touchmove “catches up” to the current finger position?
This question can be important for some pull to refresh scenarios. In particular, in Android Gmail-style pull-to-refresh (where a header animates but does not stretch out) after pulling down, pushing back up can start scrolling more quickly or immediately (before the finger has returned to the overscroll starting location). If scroll offsets were accumulated then when the app asked to start scrolling, we may still have a net-negative offset that could trigger a native overscroll effect.
Therefore scroll deltas should NOT accumulate. Consuming a touchmove event prevents the movement for that event from ever impacting the scroll position (this happens to be much cleaner from an implementation perspective as well). Of course an application can still manually adjust the scroll offset when desired.
We need to define what exactly should happen when the user lifts their finger while still moving. We’re going to send a ‘touchend’ event, but if we’re actively scrolling and we wait for the disposition of that event then a busy main thread would result in an ugly pause before fling occurs. Since this could be a common scenario (eg. on heavy desktop sites that happen to have touch handlers), I believe we need to make the touchend event in this case asynchronous and ignore it’s disposition.
However, if the page is currently consuming all touchmove events (so no scrolling is occurring), it may feel wrong if a fling gesture scrolls the content. In some cases (like pull to refresh) this may be desired, but in others - like custom manipulation of a side-drawer) it may not. Therefore in this case the touchend should be dispatched synchronously and it’s disposition used to determine if fling should occur. I.e. touchend is sent async if-and-only-if touchmoves are currently being absorbed. Note that touchend must be sent sync in non-scroll scenarios, eg. to allow a ‘tap’ gesture to be cancelled.
Chrome Android will sometimes wait only up to 200ms for a response from a touchevent before proceeding (eg. with a scroll). Since touch events are now possible after scroll has started, we need to define which events the timeout applies to.
Once any touch event has been consumed, we should continue to disable the timeout. But what if no move has been consumed? Should every move event be subject to a timeout, or only the first?
I propose that each move event continues to be subject to timeout in order to mitigate the risk of browser native overscroll jank (see risk of impact on scroll performance below).
TODO: Is a touchstart for a second finger during scrolling send sync or async? If sync, what exactly happens?
The following is a list of potential concerns, and what we can do to help mitigate them.
Since this proposal would send events and block on the main thread in some additional scenarios, there’s some risk of regressing performance (primarily in edge-case scenarios). The async touchmove events could also hurt performance just due to additional CPU usage or by triggering expensive actions (eg. layout) on poorly written sites.
However, in the most common cases of main thread jank (heavy desktop sites) there often aren’t any touch event handlers, and so our compositor touch hit testing optimization will ensure there is still no performance impact of touch events during a scroll. Nonetheless, there will still be scenarios with touch handlers and a janky main thread. Ultimately our plans for touch-action and ‘scroll-delay: none’ can still be leveraged to enable free scrolling in the face of touch event handlers.
A key scenario to consider is when scrolling to the end and then (while still touching) reversing direction and scrolling back. For sites that have touch handlers and a janky main thread, this proposal would result in jank in such a case. However the user impact won’t be any greater than if they lifted their finger and started a new scroll. Scroll velocity already came to 0, so a delay in starting scrolling isn’t nearly as bad as a jank during scroll. However this is still probably the primary scenario for concern over scroll performance.
One scenario to consider is when vertical scrolling starts to drift sideways on a non-side-scrolling page. Our “scroll rails” will suppress small amounts of movement, but eventually with sufficient sideways motion we will break the rails and start scrolling freely in both axes. At that point, if there is a scroll event that consists entirely of horizontal motion (which is relatively easy when moving slowly), touchmove absorption will cease and the next touchmove event will block on the main thread - potentially introducing a pause into the scroll. In practice it seems hard to break the scroll rails accidentally, but we should watch our new scroll-update latency metrics for any sign of impact here. If we see issues, we can explore tweaking our rail behavior to make accidental rail-breaking harder (or to re-enable rails after they’ve been broken). This issue can also be avoided with “touch-action: pan-y” which prevents any horizontal scroll motion after a scroll has started.
Another scenario to consider is the performance impact on native overscroll effects. The Android glow is subtle enough, and the mitigations above strong enough, that I doubt it’s going to be a problem in practice. However we should keep our eyes out for a loss of smoothness in the overscroll animation. Worst case we could restrict the model to avoid sending repeated unconsumed touchmove events on overscroll (giving the application one chance only to decide to consume events during overscroll).
We previously considered a pure async touchmove model and rejected it due to problems we’d seen using Firefox where you’d get both scrolling and some browser behaviour due to the application consuming touchmove events after it was too late to cancel scrolling. I believe this is most common in practice when moving first in a direction that is not scrollable, such as sliding sideways on an image carousel and then moving vertically. The model described here avoids this problem by sending touchmove events synchronously during overscroll - allowing the page to terminate scrolling and prevent future scrolling.
There is some risk we’ll see double-handling issues in other scenarios. For example if a page is scrollable horizontally and has a component that responds to horizontal touchmove events but doesn’t call preventDefault until after scrolling has started, then we could get some double-handling symptoms. Such a page will be broken on most other browser as well (eg. on safari document scrolling any browser behavior will be invisible until scrolling terminates). However if the application is correctly using the ‘page’ coordinates in the touch event (as opposed to the ‘screen’ co-ordinates), then it won’t see much movement (and so shouldn’t be triggering an effect at all) until the scroll limit is reached.
The biggest compatibility risk is that some website will be surprised by a touchmove event marked with cancelable = false. The TouchEvents spec says that touchmove events are sent synchronously and are cancelable. Many browsers already send touchmove events that are effectively uncancelable during a scroll, but none explicitly mark them that way (I’m attempting to engage the other vendors on this issue here). Code designed explicitly for chromium or Android browser (including Android WebView apps) may reasonably expect that consuming a touchmove event always stops scrolling. We’ll attempt to mitigate this risk through testing, watching closely for feedback in beta, evangelism and tooling. In particular, we’ll explore adding a warning to the devTools console whenever preventDefault is called on an uncancelable event.
There’s also some risk that a site could be confused by the finger appearing to move less smoothly during a scroll. Naive velocity tracking would still work correctly, but won’t have the resolution over short timescales it may expect (eg. in order to detect quick flick gestures while scrolling).
I can’t think of any good way to search for such sites (especially without a single problematic example), but we should keep our eyes open and engage the community in discussion. Even if some sites are affected, it’s likely to be small compared to the sites whose compatibility will be improved.
We’ve done much of the groundwork for this as part of touchmove absorption. Initial results are positive, including:
Next I think we should: