Sustainable Ux in VR
How to use VR to increase structural and behavioral sustainability
Dr. Pete Markiewicz
2017
Scope of the Talk
1. Sustainable Virtual Design Theory.
2. Virtual design megatrends.
3. About VR.
4. About Ux in VR.
5. Creating sustainable “Ux in VR” strategies.
About the Author
Dr. Pete Markiewicz worked as a research scientist at UCLA and elsewhere until mid-1993.
Co-founded an early Internet music download & sales service, “Indiespace” in April 1994.
Author of books on the Internet, US Generations, & Tech.
Taught 1000s of students in Web, Graphic Design, Ux and Web Development since 2001.
Focus: WebVR (JavaScript) and Web-VR hybrid experiences.
Sustainable VR
How green is virtual reality?
The Virtual World is About get 3 New Media…
Virtual and Augmented reality move design from graphic design principles to 3D design principles
Voice-only “Oracle” systems eliminate visual �design and replace it with Ai conversations.
2000
2010
2020
Oracles
VR and AR
Too Often, Virtual Design is Treated as “Weightless”…
I’m green, because I replaced a book with a website!
Carbon Footprint of Virtual Reality?
VR uses 7x as Much Power as the Web!
Jason Paul, the general manager of Nvidia’s gaming, and VR business notes that current consumer PCs often aren’t powerful enough to run VR.
How much more powerful do they need to be? 7 times.
These energy hogs consume the power of 3 refrigerators.
A VR user will have a higher energy budget than real people in “2nd world” countries.
The Grand Collision
Ux has mostly focused on 2d web design.
Game design used 3D, but operated by very different (“pleasures of uninhibited excess”) rules
Creating sustainable behavior can be done in VR (and AR) better than the web, due to its “long-form” nature.
Encouraging sustainable behavior via non-game experiences in 3D will become a routine focus of Ux designers.
Ux can Help us address VR Sustainabily…
Ux designers can’t leave this to “gamers” or game designer…
I thougt it was a game!
Sustainable Virtual Design considers...
Current needs of users, designers, and developers to create sites and apps.
Behavior, so sites, apps and vworlds encourage sustainable user behavior.
Strategy, ensuring projects don’t degrade the Internet’s virtual ecosystem.
Our Sustainable VR Goals
Define strategies in VR where the effects on the user compensate for increased energy consumption.
Efficient use of technology.
Greater focus and attention.
Better task completion.
Transfer of learning inside VR to the outside world.
Virtual Design Megatrends
Isn’t VR just for Gamers?
What Makes VR and AR Different
VR and AR create artificial, immersive 3D worlds.
Behavioral cues, triggers, affordances are different in 3D.
2D “flat” design tools can’t�effectively represent 3D interaction.
Users interact with whole body, not �just their eyes and hands.
Only some stomach VR games, but everyone �enjoys gentle VR experiences.
VR Milestones in 2018
Total Market in 2017 - about 4 billion
12% of consumers will buy VR devices this year.
VR products and demos at Best Buy (but NOT the Apple Store).
Consumer ‘room scale’ experiences possible with HTC Vive.
Android software and hardware gets built-in VR support.
WebVR API (VR created in web pages via HTML, CSS, JavaScript) standard goes live.
Where’s your headset?”…
You’re in Print design, it’s 1994 and somebody is talking about the World Wide Web…
I’m Soooo Excited!
How VR Can Do Better Than the Web
And why you need to expand beyond the Web
VR and AR are Their Own Medium…
Not like the web
Not like a console game
Not like a movie
Most like a…book?
VR is a “Long Form” Medium
Feature | | VR | Web | Mobile | |
1. | Engagement | > | Immersive, user is surrounded by media. | Casual, media only occupies part of the field of view. | Similar to web. |
2. | Distraction | > | Low, self-contained worlds focus attention .locally | High, hyperlinks encourage constant exploring instead of focus. | Medium, small screen limits choice to exit. |
3. | Space and Time | > | Temporal, experience unfolds in space and time. | Spatial, information presented in flat layout. | Similar to web |
4. | Presence | > | High, you feel “you’re there”. | Low, you are reading or looking at an event from outside. | Lower than web or VR. |
5. | Interaction | > | Literal, you reach out to grab something | Abstract, you move a mouse to position a cursor. | Literal, you touch a control to use the control. |
6. | Social | > | Low, current system have limited social networking | High, interface maps to real social networks. | Similar to web. |
VR Experience = “The Empathy Machine”
Extremely strong emotional engagement with content (unlike the web, but like great books).
Users are willing to “keep exploring” or watching (less desire to click somewhere else).
Stronger “sense of having been somewhere” (the 4th wall of movies is in VR than web, TV).
Human empathy seems to require high-rez�depictions of events.
VR has more “emotional bandwidth” than �the web.
VR Enables…
Empathic connection with remote scenes (experience like a knowing ‘ghost’).
Fake body enhancements (fake limbs, body-switching).
Therapy, situation and avoidance.
Acting out real-world behaviors with the �body, instead of just in the head.
Examining a complex systems all at once.
VR Encourages…
Extended interaction with an site, app, or project.
Progressive internal discovery (as opposed to the centrifugal “jumping elsewhere” discovery of web browsing)
Developing a deep connection with what is experienced, and treating it as more than information.
The Dark Side
Is VR more addicting than online games, websites, gambling?
VR is likely to follow a “bell curve” of engagement
Some users will become addicted.
“Third Person” VR creates more empathy �tor the oppressed than “First Person” VR
Addiction may be worse than the web, due �to a nonexistent ability to multitask �while in VR.
Ux in VR
It’s not a web page anymore…
The Goals Ux in VR
Create VR experiences from a Ux, rather than Game Design perspective.
Develop VR interactions that allows users to complete tasks within the medium…
…But even more important, construct VR and AR experiences that lead to more sustainable behavior outside the virtual world.
Sustainable behavior encouraged by VR can compensate for the increased carbon footprint of the medium.
Ux in VR Pioneers
Adrienne Hunter (founder, Tomorrow Today Labs)�https://www.linkedin.com/in/adriennehwong/
Mike Alger (VR designer, Google, Mountain View, CA) �http://mikealger.com/
Timoni West (Ux designer at Unity3d)�http://blog.timoni.org/
Josh Carpenter (Ux in VR for Mozilla/Firefox)�https://www.youtube.com/watch?v=ZOaOYTOpwyM
Jason Jerald (NextGen Interactions, 1st VR design book)�http://www.thevrbook.net/
Arturo Paracuellos (Unboring.net)�https://blog.mozvr.com/puzzle-rain/
Hardware Constraints in 2016
Smartphone VR uses iOS or Android �devices with low-cost headsets (80%).
Headset VR uses a dedicated desktop �or laptop to render higher-quality�gaze-directed (15%).
Roomscale VR provides tethers, or �mounts the VR device on the user �for greater exploration (5%)
No current Macintosh is powerful enough
To do Headset or Roomscale!
Levels of Interaction
Passive – viewing, but only able to move head
Gaze – select objects by looking at them
Haptic – use a hardware sensor represent hands or other parts of the body to select objects
Full-Body – use sensors to determine the position of the entire body in the scene.
Haptics
Typically hand-held.
Have additional controls �(triggers, buttons).
Existing gamepads are often repurposed�into VR.
Gloves NOT in widespread use.
Need virtual display of haptic �inside VR to be effective.
User Environment – View Details
Field of View: 100deg, 60deg up, 40deg down. Experiments with 120deg and 270deg will require very high-end hardware (e.g. StarVR)
Rotation: 55 degrees of head.
Resolution: minimum 960px per eye, 0.1 degrees per pixel. Ideally, 4k or 5k signal split between the eyes for ~1080p.
Apparent Fields of View (FOV)
StarVR 270
User Environment – 3D Zones (Mike Alger)
No-no Zone: at least 0.75 meters away from head
Hands Quadrant: a quarter-sphere below the �user’s arms
Distant: 20 meters current limit for any 3D �effect (~1 pixel stereo diff in Oculus)
Danger zone: anything from above the head!
Real-World Boundaries: device draws a wall when user�approaches bounds of the simulation.
User Behavior in VR Space
Useful Ui Zones in VR (Mike Alger)
Touch Ui Zone
“No-no” Zone
Content Zone
Ui Zones on a 360 image
A Note on Text
Text renders poorly in 3D compared to 2D.
Animated text can be used effectively (banners, or in-place replacement).
Static text should be a last resort.
New Grid Systems will be required.
App Design
Voice-Driven,
Conversational
3D Games
Dashboard
Empty
Roomscale
Armchair
360 Video
3D Web Page
Web-VR Transitions
Current “entry” into a VR scene involves complex and clumsy user actions.
The WebVR API allows VR to be embedded naturally into web pages.
WebXR standard promises a model for transitioning from a web page into VR/AR or vice-versa.
Discussion at: https://www.youtube.com/watch?v=78F_HNWuqbw
Ux Techniques in VR
How are they different from the Web?
Some General Principles
Design for a user embedded in your design.
Anticipate and direct user’s attention
Ensure comfort (good HMDs) and safety (danger room)
Enlarge text to adjust for low resolution
Use audio to improve attention and immersion
Minimize movement in the periphery to avoid motion sickness
Prototyping Techniques
“Spherical Grayboxing” – create grey polygons to substitute for Ui objects on gridded spheres (Alex Chu)
“Cylinder” – Flat 960px Illustrator template, edited, and wrapped around user model at 0.5m (Josh Carpenter)
Remote Testers - companies like �Fishbowl offer remote tests for �task completion within VR.
A 2D VR Template
Cylinder Example (Josh Carpenter)
Low-Resolution
High-Resolution (Cinema 4D)
Sample Cylinder Grid System (Josh Carpenter)
Affordances
Hardware buttons on the device.
Circular flat overlay menus or icons.
Local maps embedded in world.
Text input fields (hard).
Object gestures (the object makes a recognizable motion, like sign language).
Objects which look like the thing they manipulate.
Virtual copies of physical haptic devices in the scene.
More on Affordances - Interactions
User presses button on physical device
Pressing physical buttons causes buttons to appear in virtual space
User uses head to align fixed, floating cursor over scene elements
User uses head to align cursor over scene buttons and controls
Buttons are placed in a fixed location where the user is likely to look
User rotates the viewer to go somewhere (equivalent of a ‘swipe’)
User speaks commands and app responds (preferred?)
Triggers
Prolonged gaze – object can expand or change appearance when stared at for a few seconds.
No Button “clicks” – when “touched”, the experience is more like putting finger into a container of water).
Audio Cues – sound changes on motion, selection.
Feedback
Hardware buttons on headset can provide tactile triggers.
Object moves (instead of scene moving).
New object appears in scene.
Scene dissolves (must be slow or will be nauseating).
Audio changes (intensity or type of sound).
Openings (a path, door, gateway).
Handling Motion
Movement video style (pans, forward movement) = barf-o-matic.
If user is looking in the direction of movement, negative effects are minimized.
Reducing field of view (FOV) during motion highly effective (e.g. GoogleEarth VR app)
Object Roles (Jonathan Ravasz)
The ground/land – allows faster orientation to the scene
The sky – gives a sense of scale
Audio fadeins – introduce a change gently
Context reticle – show user where they gaze is going.
Objects – define paths for exploring
Talking objects – �provide detail usually from text
Inclusive Design in WebVR
A11Y & WebXR - Roland Dubois
https://docs.google.com/presentation/d/1_mpJ1kOHxR1De-0z8G-Qfg8sKNC1DUCTvYw3IkYDyCI/edit?usp=sharing
Anti-Patterns (Timoni West)
Moving the user when they aren’t moving (a common error in 360 video).
Static text (use animated text or audio).
Teleports or sudden scene shifts.
No time for orientation when scene changes.
No tracking feedback (users need to see constant motion related to their own.
Sudden motion of an object towards the user (especially dropping down from the sky to the head).
Design in VR Space
Flat layouts and wireframes are of low value.
Overly-detailed 3D causes premature launch.
Simple scene perspective drawings are effective.
Storyboarding or dialogue may describe what’s going on better than graphics.
.
Designing VR Scenes (LeapMotion)
Safety barriers – objects should be blocked from coming too close to the viewer.
User camera - users need control (otherwise = vomit comet)
Human scale – Don’t surprise the user with “Alice in Wonderland” size changes, unnatural speed of motion.
Responsive VR Breakpoints
Design for low-end haptics, also gaze-only selection, no room-scale.
<= 80deg view,�Head Motions only
90-100deg view, plus hardware �button select on headset
110deg view, plus handheld
haptic devices
> 120deg view, full-body, �room-scale interaction
Sustainable Ux in VR
Structural and Storytelling Features
Measuring structural sustainability using qualitative factors...
Carbon footprints are only part of the sustainable story...
Full Carbon Footprint Calculations are hard...
For an LCA, you need to include:
The Ouroboros Strategy
Create database listing green ingredients used to create project.
Score Presence or Absence of “Green Ingredients” in your project.
Swap “Green Ingredients” for original ones.
Quantify when possible, otherwise list swaps.
+48%
Some “Green Ingredients” in VR
Features reducing motion fatigue, physical �discomfort.
Features minimizing energy consumption.
Features maximizing inclusion of all users.
Features allowing task completion in VR.
Features encouraging empathic replication of VR learning in the real world.
VR Storytelling should encourage sustainable behaviors...
Use the empathy features of the medium to change behavior in the real world.
Experience Metaphors
Curved walls – like a convention installation
Theater in the Round – events happen all around you
The Cave – similar to games, roomscale only
Disabled user (yes) – your limitations match VR limitations
The Workbench – pick and place tools, post-it notes, etc.
The Ghost’s Dream – you’re embedded in a world that can’t see you
‘Gone Fishing – the world flows by in a leisurely way
VR Storytelling methods
A God’s throne in a world of pain (360 video).
A scrolling world with its own intent, moving past the user.
A simulation of a complex simulation �which may be explored.
Allow creation of custom 3d objects to �populate an empty world.
Meditation, contemplateion encouraged �by VR scene.
VR Scenarios for Sustainable Behavior
Show a complex process as a model that users can explore, inspect, manipulate (it can be more complex than a print or web model).
Create your Customer Journey as a VR experience.
Place the user in a scene arousing emotional empathy with people/animals suffering the consequences of bad choices.
Let people create sustainable behaviors �(e.g. sorting trash) in a �VR setting first.
What You Need to Do…
Your Road to VR
Get a headset, of Samsung Gear VR/ Google Daydream or higher.
Explore some commercially-produced worlds (Steam community).
Learn something about audio design (it’s important).
Read “Ux in VR” blogs.
Practice long-form storytelling (again).
Develop your standard deliverable set to �describe VR experiences.
References
The Ourboros Strategy - https://sustainablevirtualdesign.wordpress.com/2012/03/14/the-ouroboros-strategy-for-sustainability-part-1-listing-green-ingredients
Swapping Green Ingredients�https://sustainablevirtualdesign.wordpress.com/2013/02/08/great-example-of-swapping-green-ingredients/
Interaction Design Practices in VR – �https://uxdesign.cc/design-practices-in-virtual-reality-f900f5935826#.n1uahs3if
References
Will Virtual Reality Damage the Earth?�https://sustainablevirtualdesign.wordpress.com/2016/03/26/will-virtual-reality-damage-the-earth/
VR, the Empathy Machine�http://www.opensourcemechanic.com/blog/2015/11/vr-empathy-machine-links-un-dignitaries-with-syrian-refugees/
Ux Design in VR�http://aperturesciencellc.com/vr/VisualDesignMethodsforVR_MikeAlger.pdf
References
Puzzle Rain: a Ux in VR study:�http://unboring.net/cases/puzzlerain.html
Ux in VR Patterns (and anti-patterns)�https://medium.com/@timoni/ux-pointers-for-vr-design-dd52b718e19#.o7srbtvrv
Leap Motion VR Design Blog�http://blog.leapmotion.com/designing-vr-tools-good-bad-ugly/
Ux in VR Blog (Adrienne Hunter)�http://www.tomorrowtodaylabs.com/
References
The VR Book: Human-Centered Design�http://www.thevrbook.net/
Oculus Best Practice�https://developer.oculus.com/documentation/intro-vr/latest/concepts/bp_intro/
Unreal Engine best Practice�https://docs.unrealengine.com/latest/INT/Platforms/VR/ContentSetup/index.html
References
References