1 of 11

WebXR DOM Overlay in 3D

TPAC 2020-11-04

Klaus Weidner / klausw@google.com

2 of 11

Status update

Spec: Editor’s Draft 2020-06-04� https://immersive-web.github.io/dom-overlays/

Explainer: � https://github.com/immersive-web/dom-overlays/blob/master/explainer.md

3 of 11

Implementation

Chrome on Android: since v84+ for handheld AR

Any others?

Used by <model-viewer> and other WebXR AR sites. Use case examples:

Selector / configurator
Exit button
Annotations

4 of 11

WebXR DOM Overlay examples

5 of 11

Use Case: Annotations

Place DOM elements with markers and/or text
Attached to 3D positions in the model, following model and viewer movement
Important use case, i.e. for museum exhibits
Interactivity:

None (display only)
Limited (simple clicks, expand/collapse)
Full DOM functionality (rare?)

DOM update rate != XR update rate

Some lag or jitter due to position mismatches

6 of 11

Overlay types: XRDOMOverlayType

See [specification]:

screen - overlay element covers the entire physical screen for a screen-based device, for example handheld AR. Its visual extent MUST match the visual extent of the XRViewport(s) [...]

monoscopic display: single viewport
stereoscopic display: two viewports, overlay rendered at Z position matching physical screen location.

floating - overlay appears as a floating rectangle in space.
head-locked - overlay follows the user’s head movement consistently, appearing similar to a helmet heads-up display.

7 of 11

Annotations in 3D viewed on a headset

The visual appearance of a floating overlay is similar to a flat glass pane in between the user and model, and annotations would appear as moving stickers on that pane.

If an app such as <model-viewer> places annotations in the DOM overlay assuming a screen type, the position would be inaccurate on a floating overlay.

There’s some degree of graceful degradation. Annotations would still be near the expected location as long as the floating overlay’s size is similar to the rendered viewport.

8 of 11

Potential improvements to a floating overlay

If the applications knows the overlay location in 3D space, it could provide more accurate locations within the single-plane constraint.

Spec NOTE: Future versions of this spec may add additional attributes to the overlay state, for example the current location in world space for a floating overlay.

Optionally, draw 3D connecting lines from the model to the corresponding annotation?

9 of 11

Option: stereoscopic DOM?

If the DOM content could be rendered differently for the left and right eye, the application could use this to place content at 3D positions.

The content would still be constrained to a pyramid/frustum with extent defined by the floating quad’s boundary.

Handwave: CSS :left-eye / :right-eye pseudo-classes?

Per-eye positions causes ambiguity for DOM hit test.

Implementation challenges?

10 of 11

Option: multiple DOM layers in 3D?

Advantages: very flexible.

Downsides: implementation challenges, input / security seems hard to get right.

Feasible for display-only DOM layers with no interactivity? The app would need to do its own hit testing and interactions.