WebVR/XR and A-Frame
Why Web-Based and Markup VR is the way to go in 2018
Dr. Pete Markiewicz - February 2018
About the Author
Dr. Pete Markiewicz worked as a research scientist at UCLA and elsewhere until mid-1993.
Co-founded an early Internet music download & sales service, “Indiespace” in April 1994.
Author of books on the Internet, US Generations, & Tech.
Taught 1000s of students in Web, Graphic Design, Ux and Web Development since 2001.
Focus: WebVR (JavaScript) and Web-VR hybrid experiences.
Scope of the Talk
1. VR’s failed promise.
3. WebVR to the rescue, and WebXR adds AR support.
4. How WebVR helps the “Ux of VR” problem.
5. The Prototyping problem.
5. Should the VR Web be DOM-like?
6. A-Frame Tutorial.
VR’s Failed Promise
Why RP1 is NOT Your Goal...
“I want my VR?”…
VR audience hype does not match consumer wants and expectations…
Well, I AM Stillllll Excited!
There’s No
Mass Market
Headset sales expected to increase from 9.6 million last year to 59.2 million by 2021.
Sony sold about 500,000 headsets per quarter in 2017
344.3 million smartphones were sold worldwide in Q1 2017
But...People LOVE VR!
Only when they experience it at a venue.
Only when someone handles the VR part for them.
Only when the experience is ultra high-end.
The Problem
“...enthusiastic reactions to VR at trade fairs or at industry conferences, based on a few minutes of usage, may not convert into mass market demand...”
What does VR Give That We Aren’t Already Getting?
..from the casual, mobile web?
We’re connected, you just can’t see it
RP1 Will Make Things Worse
Hyped expectations meet lame reality?
Current VR experiences are nothing like the movie.
VR content only appeals to hardcore gamers>
Gamers misunderstand the future as them.
Headset sales will crash following �disappointment.
VR IS Ready...but not in the way many think...
When industry leaders say that “VR is not ready”, they usually mean that you can’t have a painless, ultra-high-end immersive VR experience in your home.
Most consumers won’t have a home experience comparable to a 3D game in cinematic resolution for 10-20 years.
This is the same mistake Print Designers made about the early Web in 1994.
VR is ready...but not in the forms many expect.
Let’s Build Something Real
Ux in VR Points to Web Solution
The VR Market is Splitting 3 Lines
Location/Casual Split..
Location Mixed Experiences�(Cinematic Model)
Casual VR
2000
2010
2020
Dynamic Web
Digital Streams
Mobile Apps
Static Web
2-Dimensional World
2030
Location VR�(Arcade Model)
Designer VR
(Studio Model)
“Home Theater”
VR (limited)
Implications for Casual VR Experiences
WebXR Matches the “Casual” VR market
Runs on smartphone-based browsers with JavaScript and WebGL API.
HTTP connection makes it easy to integrate existing web services.
Can be used in multiple “responsive” modes.
Can run in non-VR “magic window” modes when necessary.
Progressive Enhancement via WebXR
WebVR is the only
framework that works
Predicted Market Share of VR Segments
From WebVirtualR to WebXtendedR
Making VR & AR Practical
What was WebVR?
What is WebXR?
Successor to WebVR 1.1 JavaScript API (no 2.0 version)
Extends WebVR spec, doesn’t replace
Not the same as Khronos OpenXR (WebXR could be implemented using OpenXR in browsers)
Not the same as Jerome Etienne’s AR work via AR.js - https://github.com/jeromeetienne/AR.js
WebVR 1.1 will remain as legacy API
Summer 2018???
Motivations for WebXR
Support AR and VR in the same API
Native AR support (Android ARCore, iOS ARKit)
Built-in support for Progressive Enhancement
Layers (e.g. HUD overlays on 3D scene)
Built-in support for seamless embedding into web pages (like HTML5 Web Audio/Video)
Better support for 360 video
Security Model
WebXR Concept Model
Some aspects of reality are replaced, others are transmitted in all VR and AR
Reality layer defines what will be transmitted from the real world
All VR and AR end up as augmentations, layered over the based Reality in a XRLayer
Note: NOT CLEAR that Google group is doing this!
WebXR Spec
WebXR Key Sites, Ctd.
Discussion Group: public-webvr@w3.org |
Our Challenge
The WebXR specs are unfinished, and are being built almost entirely by engineers
“They” blew it with WebVR, not achieving the “central common road” of HTML that gave the web it’s growth
It’s up to us to make this work
JOIN THOSE GITHUB GROUPS (or be disappointed again)
Using WebXR
Direct Code vs. Markup
Vanilla JS is Hard!
The WebGL API is already extremely complex.
WebVR adds several layers of complexity.
Asynchronous, hardware-dependent nature of interaction calls for advanced programming techniques.
WebXR Coding Strategy
Start Coding – 1 (Direct)
if (navigator.getVRDisplays) {� // Enumerate the VRDisplays� navigator.getVRDisplays().then(function (displays) {� if (!displays.length) {� VRSamplesUtil.addInfo("WebVR supported, but no VRDisplays found.");� return;� }� for (var i = 0; i < displays.length; ++i) {� document.body.appendChild(buildVRElement(displays[i]));� } � }� );� window.addEventListener("vrdisplayconnect", eventFired, false);� window.addEventListener("vrdisplaydisconnect", eventFired, false);� window.addEventListener("vrdisplayactivate", eventFired, false);� window.addEventListener("vrdisplaydeactivate", eventFired, false);� window.addEventListener("vrdisplayblur", eventFired, false);� window.addEventListener("vrdisplayfocus", eventFired, false);�} else {� VRSamplesUtil.addError("Your browser does not support WebVR. See <a href='http://webvr.info'>webvr.info</a> for assistance.");�}
VR/AR/MR JavaScript Frameworks
Frameworks redistribute some complexity…
THREE.JS
THREE.JS a complete �feature set for 3D �application creation.
A framework, rather than a game engine.
Originally a Flash AS3 library, ported to JavaScript in 2009.
A HUGE community of designers and developers supporting the project.
Supports Oculus, Vive hardware, selecting objects in VR.
Babylon.JS
Babylon.js is a game engine written in WebGL.
Provides full support for game development (several titles have been converted into Babylon from native code).
Uses camera (WebVR camera and DeviceOrientation camera) to enable VR scenes.
A large community of designers and developers supporting the project, but few WebVR examples (yet).
VR support currently in state of rapid flux.
ReactVR
Developed by Facebook for their Oculus platform
Exploits existing ReactJS framework
Uses ‘signed distance field’ fonts for greater clarity
class WelcomeToVR extends React.Component { render() { �// Displays "hello" text on top of a loaded 360 panorama image. �// Text is 0.8 meters in size and is centered three meters in front of you. �return ( � <View> � <Pano source={asset('chess-world.jpg')}/>� <Text style={{ fontSize: 0.8, layoutOrigin: [0.5, 0.5], transform: [{translate: [0, 0, -3]}], }}> hello� </Text>� </View> �); } }; AppRegistry.registerComponent('WelcomeToVR', () => WelcomeToVR);
PrimroseVR - http://primrosevr.com
A framework based on THREE.js
Built-in “room” model for creating and navigating between multiple rooms.
Fast, can use directly for App development
Transpiles From Other Dev Environments
Unity 5 – can transpile code from native Unity to JavaScript using ASM.JS. Working examples available.
Unreal4 – support for ASM.JS. No good examples yet.
Note: resulting code files are HUGE (~90 megs) compared to native JavaScript coding (~1 megs).
This is Still HARD CODING!
Frameworks change, but do not remove the complexity of the WebGL + WebXR stack.
Current frameworks do NOT completely support WebXR
VR Markup Languages Offer a Simpler Alternative
WebVR Markup Language (HTML-Like)
X3D (Successor to VRML)
Web 3D Consortium
Formed to handle shortcomings in VRML, the original VR modeling language for the web
In development since 1999(!)
More complex than other 3d markup languages
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE X3D PUBLIC "ISO//Web3D//DTD X3D 3.3//EN" "http://www.web3d.org/specifications/x3d-3.3.dtd"> <X3D profile='Immersive' version='3.3' xmlns:xsd='http://www.w3.org/2001/XMLSchema-instance' xsd:noNamespaceSchemaLocation =' http://www.web3d.org/specifications/x3d-3.3.xsd '> <head>
<meta name='title' content='CircleFishPositionInterpolator.x3d'/>
<meta name='description' content='Run-time modification of a fishes position using PositionInterpolator.'/>
<meta name='creator' content='Ildeniz Duman'/>
<meta name='created' content='1 June 1998'/>
<meta name='modified' content='5 July 2014'/>
<meta name='identifier' content=' http://X3dGraphics.com/examples/X3dForWebAuthors/Chapter07-EventAnimationInterpolation/CircleFishPositionInterpolator.x3d '/>
<meta name='generator' content='X3D-Edit 3.3, https://savage.nps.edu/X3D-Edit'/>
<meta name='license' content=' ../license.html'/>
</head>
A-Frame
A Mozilla project built on THREE.js.
Uses XML style markup to create �3D scenes.
Optimized for WebVR use.
Component model allows behaviors to �be attached to 3D objects.
Good choice for rapid 3D WebVR �prototyping.
<a-scene fog="type: linear; color: #AAB; far: 30; near: 0">
<a-assets>
<img id="highlight1" src="../../assets/img/radial-highlight.png">
<img id="shadow3" src="../../assets/img/radial-shadow-3.png">
</a-assets>
<a-entity position="0 0 -10">
<!-- Ground Highlight -->
<a-image position="0 -.2 5" src="#highlight1" rotation="-90 0 0"
scale="30 30 30"></a-image></a-entity>
A-Frame Tutorial
Install Nodejs (http://nodejs.org) and Git (https://git-scm.com/downloads)
Select a folder, then type in Terminal/Console:
Dev Setup
Add the ‘stats’ component to the <a-scene> markup tag
Save the file, and confirm that browser reloads the page
Reference: https://aframe.io/docs/0.5.0/components/stats.html
The A-Frame Inspector
A a visual tool, similar to browser’s DOM Inspector
Creating Shapes
Add, texture, and transform an object (entity)
<a-scene>� <a-box color="red" position="0 2 -5" rotation="0 45 45" scale="2 2 2"></a-box>� <a-box src="img/aframe.png" position="0 4 -5" rotation="0 90 45"></a-box>�</a-scene>
Sub-entities inherit transformations
<a-scene>� <a-box position="0 2 0" rotation="0 45 45" scale="2 4 2">
<a-sphere position="1 0 3"></a-sphere>
</a-box>
</a-scene>
Reference: https://aframe.io/docs/0.5.0/guides/
Skies and 360 Panorama Images
Add a (static) sky
<a-scene>
<a-box color="red" position="0 2 -5" rotation="0 45 45" scale="2 2 2"></a-box>
<a-sky color="#222" src="img/img360/basement_panorama.jpg" rotation="0 0 0"></a-sky>
</a-scene>
Event-Trapping
var box = document.querySelector('a-box');
box.getAttribute('position');
box.addEventListener('click', function () {
box.setAttribute('color', 'blue');
});
Entities and Components
<script> AFRAME.registerComponent('foo', {
schema: {
bar: {type: 'number'},
baz: {type: 'string'}
},
init: function () {
// Do something when component is plugged in.
},
update: function () {
// Do something when component's data is updated.
}
}); </script> <a-entity foo="bar: 5; baz: qux"></a-entity>�
Component Registry: https://aframe.io/aframe-registry/
Selecting Shapes /w Components
<a-scene>
<a-entity light="color: #ccccff; intensity: 1; type: ambient;" visible=""></a-entity>
<a-sky color="#222" src="img/img360/basement_panorama.jpg" rotation="0 0 0"></a-sky>
<a-camera position="0 2 4">
<!-- setting fuse=true binds camera cursor to hover click event-->
<a-cursor color="#4CC3D9" fuse="true" timeout="10"></a-cursor>
</a-camera>� <a-sphere color=”#F44336" radius=”1" position=”-4 2 0" set-sky="img/img360/harbor_panorama.jpg">
</a-sphere>
</a-scene>
<script> AFRAME.registerComponent('set-sky', {
schema: {default: ''},
init() {
const sky = document.querySelector('a-sky');
this.el.addEventListener('click', () => {
sky.setAttribute('src', this.data);
}); } });
</script>
Reference: https://blog.neondaylight.com/build-a-simple-web-vr-ui-with-a-frame-a17a2d5b484#.eodn483f5
360 Video
<a-scene>
<a-assets>
<video id="video" src="video/video360/sample360_video.mp4" autoplay loop crossorigin></video>
</a-assets>
<!--note how assets are loaded separately for video-->
<a-videosphere src="#video" rotation="0 180 0"></a-videosphere>
</a-scene>
Note: 360 video performance at high resolutions is slow in current browsers.
Reference: https://github.com/aframevr/aframe/blob/v0.5.0/examples/boilerplate/360-video/index.html
References
A-Frame - http://aframe.io
Mozilla A-Frame Presentation - http://delapuente.github.io/presentations/aframe-101/#/
A-Frame, ReactJS, and WebPack Boilerplate - https://github.com/ngokevin/aframe-react-boilerplate
Build a Basic Game Prototype in A-Frame - https://developer.mozilla.org/en-US/docs/Games/Techniques/3D_on_the_web/Building_up_a_basic_demo_with_A-Frame
A-Frame Sample to Android App via Crosswalk - https://crosswalk-project.org/documentation/tutorials/webvr.html
ReactVR (alternative) - https://facebookincubator.github.io/react-vr/index.html