360 videos and
environments
Nikolaj Petersen
nsp291@nyu.edu
Workflows, Methods,
Examples and References
Plan/Sketch
Record
Synch and Stitch
Edit / Coding
Publish
Currently most common VR/360 platforms
IN THE BROWSER
HANDHELD 360
HEAD MOUNT DEVICE
Or?
Chapter 1
Recording
> $500,000 RED DRAGON setup
What is really needed to do the job?
Freedom360 RICOH THETA S
~ $3000 excl. software
$370
Essential alignment of settings
Careful Recording routine
Total Record Time: Less than 1.5hr
>8 mins = extra editing & risk of freeze
24, 25, 30, 60, 80, 120 fps
Resolution: UHD 4K (!)
Parallax issues: min. dist to cam = 3 feet
Stabilization is possible. Steady shots are preferred
Manual Synchro and stitching = Hours of work
RICOH THETA S (2 x FishEye cams)
A few settings are adjustable on App (image)
No routine req - Hit Record
Total Record Time: ~ 1hr
25 mins at a time
30fps
Resolution: HD 1080p (!)
parallax issues are not so bad
Stabilization is not possible.
Stitch and synchronize itself.
Freedom360 vs. RICOH
Where do you wanna focus your time?
Recording Flow
Recording flow:
Charge → Empty SD Cards → Align Settings on Go-Pros → Clean Lenses → Hit Record → Double check Record + Video mode → hard twist x2 → CLAP! → “Scene XX Take XX”
GoPro Settings:
2.7k 4:3 30fps or 1440p 60fps, Protune On, Low Light: Off, Spot Meter: Off, White Balance: Native, Colors: Flat, Sharpness: Low, ISO: 400 (bright daylight) or 800 – 1600 (indoor and night).
Recording flow:
Charge → Video Mode → Hit Record.
RICOH Settings:
Some available on App but not much freedom. No settings alignment needed.
Chapter 2
Synchronization and Stitching
(for Freedom360)
Freedom360 stitching w. APV Pro
RICOH THETA stitching
�Ricoh Theta S Formatter + Kolor Eyes
RTS IOS + Android App - has live monitor for images, social media sharing possibilities and remote record button.
Equirectangular Image (F360: 4K, Ricoh: 2K)
Chapter 3
Editing and Publishing
You can give the EQUIRECTANGULAR image to different platforms!
HEAD MOUNT DEVICE (HTC Vive & Oculus R (Mac/PC), Gear VR (Samsung GS5/6), Google Cardboard (iPhone)
Milk VR (android) lets you upload to Gear VR. This page goes through settings. Unity built guide for GearVR/Oculus/Cardboard.
Handheld360 / SMARTPHONE AS WINDOW TO SURROUNDING CANVAS
Supported by Youtube App and Facebook App as well as a variety of other apps (NYTimesVR VRSE,Jaunt VR ). Same procedure as “Browser” Case.
THE BROWSER
Three.js: Build 360 video browser application - basic 360 example here.
Youtube 360 or Facebook: If shooting with a GoPro Rig you need to inject Metadata to the video in order to upload to Youtube or Facebook. Guides to either are linked here.
A storyboarding tool for 360 videos
Point of Interest (POI)
Point of Interest (POI)
Mettle’s Skybox
VR and 360 Video plug-in: Working with After Effects plugin Skybox, let’s you manipulate equirectangular videos with the same freedom you have when manipulate regular videos. You can do masking, rotoscoping and other visual effects.
Tutorials: http://www.mettle.com/skybox-tutorials/
Demo Download: http://www.mettle.com/download-a-demo/
THREE.js
...The Library
Level 1: An easy way to get started: Valiant360 - is built on top THREE.js.
Level 2: http://threejs.org/examples/#webgl_video_panorama_equirectangular - A three.js example with an equirectangular video. Go download the three.js library and inject your own equirectangular video into the script. And mess around in three.js!
Level 3: Great 360 Video browser Examples: MZVR, a mozilla experiment - “OUR GOAL IS TO BRING HIGH-PERFORMANCE VIRTUAL REALITY TO THE OPEN WEB.”
360 Sound in the browser?
360 Sound for Unity?
Three.js example with 360 sound. Look after the script in the source code called AudioObject.js. The code is well commented.
3Dception has a Unity plugin for binaural sound which is a nice extension of Unity’s audio object editing tools.
SUMMARY: Work backwards!
Plan/Sketch
Record
Synch and Stitch
Edit / Coding
Publish
Publish?
Edit / Coding?
Synch and Stitch?
Record?
Plan/Sketch
How will I..
Think big in terms of ideas, think small in terms of production.
What is the end result? - How can you accomplish that?
Platform?: Three.js // 360 room projection // Head Mount Device // Something 3rd?
Ricoh Theta S vs. GoPro Rig?
Useful Links
“Making 360” - The most comprehensive guide and toolset online so far. Great encyclopedia.
360 Videologist from NASA making experiments and researching the camera market.
Elevr.com - a very cool open source minded blog around VR. Great and widely shared post about 360 sound.
Directing VR (ongoing class at ITP) blog site.
In Their Shoes Class (ITP Class Fall 2015) syllabus
THANKS