Plumbing mouse as mouse on Android

mustaq@chromium.com, aelias@chromium.org, Oct 12 1016

Contents

1.  Why?

2.  Adding a dedicated mouse event path to Blink

2.1.  MouseEvent.button & buttons

2.2.  Browser behavior plus JS events received for mouse

2.3.  Touch behavior

3.  Conclusion

1.  Why?

Android low-level mouse events in Chrome follows the "default" MotionEvent path which is designed for touches. In particular in Blink, most mouse events appears as PlatformTouchEvents which has some terrible side-effects (crbug.com/468806):

  • TouchEvents are fired for mouse.
  • Mouse events are suppressed through prevent-defaulting touch events.
  • Fired mouse events can only have button=LEFT.

Because touch events don’t inherently support hover, hovering mouse events follow a “special” path: they are plumbed as minimal PlatformMouseEvents. This adds yet another unexpected outcome:

  • Mouse event suppression thru canceled touches apply only to non-hovering mouse.

A dedicated mouse event path from ContentViewCore.java to Blink solves these problems. This solution also solves missing the PointerEvent bug for mouse on Android (crbug.com/587550).

2.  Adding a dedicated mouse event path to Blink

We are proposing to route all Android mouse events through a dedicated WebMouseEvent plus PlatformMouseEvent path to existing event handlers in Blink (crrev.com/2054193002).

2.1.  MouseEvent.button & buttons

MotionEvent.getButtonState() seems to be working perfectly in non-Samsung Android L & M, but returning no data on Samsung Android M. This is true for all mouse buttons.

2.2.  Browser behavior plus JS events received for mouse

The table below lists the observed outcome of this change for various mouse actions on the test page rbyers.github.io/eventTest.html.

On Samsung devices (at least since Android K), mouse events are getting intercepted by OS for middle & right buttons. This affects all apps, and we don’t want to change this only for Chrome/WebView.

Action

Desktop Linux (control)

L 5.1.1 on Asus N7

L 5.1.1 on LG N5

M 6.0.1 on Asus N7

M 6.0.1 on HTC N9

M 6.0.1 on LG N5

M 6.0.1 Galaxy Note5

Expected behavior

Hover

Updates hover state

Events=[mousemove+]

Updates hover state.

Events=[mousemove+]

Updates hover state.

Events=[mousemove+]

As observed.

Left click

Updates active state, follows link.

Events=[mousedown, mouseup, click]

Updates active state, follows link.

Events=[mousedown, mouseup, click, mousemove]

Updates active state, follows link.

Events=[mousedown, mouseup, click, mousemove]

As observed, possibly without the last mousemove.

Right click

Context menu appears.

Events=[mousedown, contextmenu, mousemove]

(On Win, mouseup & auxclick should appear.)

No context menu except for clicks on links.

Events=[mousedown, contextmenu, mouseup, auxclick, mousemove]

Intercepted by OS: “back” button, for all apps.

As observed on non-Samsung devices, possibly without the last mousemove.

Long term: fix context menu? 

Middle click

Opens link in a new tab.

Events=[mousedown, mouseup, auxclick]

Opens link in a new tab.

Events=[mousedown, mouseup, auxclick, mousemove]

Intercepted by OS: “home” button, for all apps.

As observed on non-Samsung devices, possibly without the last mousemove.

Left drag

Events=[mousedown, mousemove+, mouseup]

Events=[mousedown, mousemove+, mouseup, click, mousemove]

Events=[mousedown, mousemove+, mouseup, cilck, mousemove]

Same as left click.

Right drag

No dragging happens because context menu appears.

Events=[mousedown, contextmenu, mousemove]

No context menu.

Events=[mousedown, contextmenu, mousemove+, mouseup, auxclick, mousemove]

Intercepted by OS:

<1sec drag: “back” button.

>1sec drag: browser menu appears, same as long-press of “back”.

Same as right click.

Middle drag

Follows link.

Events=[mousedown, mousemove+, mouseup, click]

If dragged within a link, tries to open window/tab, gets is blocked.

Events=[mousedown, mousemove+, mouseup, auxclick, mousemove]

Intercepted by OS:

<1sec drag: “home” button.

>1sec: Google screen search, same as long-press of “home”.

Same as middle click.

2.3.  Touch behavior

Touch seems unaffected by the mouse plumbing: tap, context menu and text selection through touch can be performed normally. Once a context menu or text selection handles occurs, both touch & mouse can be used to interact with them.

3.  Conclusion

  • The minimal mouse event plumbing (crrev.com/2054193002) correctly exposes mouse events to web. There is a minor bug: we have an extra mousemove after each click. This is minor because Chrome already fires synthetic mousemove in many cases.
  • The only problem we observed in is that neither context menu nor text selection can be triggered through mouse. This doesn’t look like a blocker since touch---the main input mechanism for Android devices---still works as before.
  • Moreover, only the triggering of context menu & text selection through mouse is affected. Mouse can still be used to interact with them once they appear.