High performance stereo rendering for VR
Timothy Wilson
San Diego Virtual Reality Meetup
January 20, 2015
Stereo performance concerns
Myth: Everything has to be drawn twice - not exactly...
Some tricks to save vertex performance on monitors (normal mapping, parallax mapping) no longer fools the eye at close distances, so we need real geometry.
Conclusion: Vertex processing is inescapably doubled, focus on reducing vertex cost.
One camera per eye
Left and right eye have different positions and viewports.
IPD = (Interpupillary distance), on average 66mm between each eye, results in two view transforms.
Oculus Rift SDK provides these two matrices.
Stereo Viewports
Viewports used to divide up single display into left and right views. A viewport is a position, width and height on the physical framebuffer, in pixels. Rasterizers only generate pixels inside of the current viewport, even if it only covers a portion of the display
Position(0,0)
Width: 960
Height: 1080
Position(960,0)
Width: 960
Height: 1080
Example DK2 Viewports
Approaches to stereo rendering
Submit entire scene twice
Pros:
Easy to implement.
Cons:
Draw calls and state changes are doubled. Not friendly to CPU or GPU caches.
Naive implementations may re-calculate shadows or other non-stereo resources per eye.
Popular game engines utilize this method to your detriment.
D3D11CommandLists might reduce overhead for the 2nd submit, but the author has not benchmarked this.
Submit entire scene twice
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Update view to other eye
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
All API calls duplicated
Submit each object twice
Pros:
Somewhat easy to implement.
API calls reduced significantly. Only view state, transform and viewport, is doubled.
Cons:
Draw calls are still doubled.
Submit each object twice
Set Textures
Set Transforms
Set geometry state
Draw
Set View State
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set View State
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set View State
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set View State
Draw
Added view state, and second draw, bulk of state is ‘reused’
Hardware to draw stereo: Geometry Shader
Pros:
Hardware does heavy lifting.
Multiple D3D11 viewports handled automagically using SV_ViewportIndex.
Pixel shaders left untouched.
CPU side has minor changes to set up additional left/right eye data and bind GS.
No state duplication, no additional draw calls. Can handle texture-per-eye rendering with SV_RenderTargetArrayIndex, without additional cost.
Cons:
Platform may not have concept of a GS, (OpenGL ES) or even expose the functionality in the game engine.
Can be invasive to vertex shaders.
Measured 3x or more slower in geometry throughput in testing. (587 µsec versus 150 µsec for a sample mesh on an nVidia 660 GTX)
Hardware draws both eyes at once
In both GS and instanced case, API usage looks like a typical desktop application.
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Set Textures
Set Transforms
Set geometry state
Draw
Hardware to draw stereo: Instancing
Pros:
Fast - doubles hardware geometry processing only. 3 times faster on the GPU than using the GS to amplify geometry.
No additional state changes.
Minor change to vertex shaders.
Cons:
Won’t support per-eye render target, currently not an issue.
Makes regular instancing a tiny bit more complicated.
Graphics API may not support dynamic clipping. (OpenGL ES)
Using instancing to draw stereo: the magic
In clip space, X ranges from -W to +W
1. Scale X by 0.5
2. Shift X coordinate by half of W, left or right depending on the eye.
3. Use hardware triangle clipping to prevent spill over into the opposite eye.
-W
+W
0
Using instancing to draw stereo
For the left eye, clip against the right edge.
For right eye, clip against the left edge.
example: In clip space, the right frustum plane coefficients are (-1,0,0,1)
Left Eye
Right Edge Clip Plane
Render stereo with hardware: Details
Use the Instanced version of the draw API
Render stereo with hardware: Code
Pseudocode follows:
Matrix WorldToEyeClipMatrix[2] // computed from SDK
Vector4 EyeClipEdge[2]={(-1,0,0,1), (1,0,0,1)}
float EyeOffsetScale[2]={0.5,-0.5}
uint eyeIndex = instanceID & 1 // use low bit as eye index.
Vector4 clipPos = worldPos * WorldToEyeClipMatrix[eyeIndex]
cullDistanceOut.x = clipDistanceOut.x = clipPos · EyeClipEdge[eyeIndex]
clipPos.x *= 0.5; // shrink to half of the screen
clipPos.x += EyeOffsetScale[eyeIndex] * clipPos.w; // scoot left or right.
clipPositionOut = clipPos
Contact the author