1
Kernel Foveated Rendering
Xiaoxu Meng, Ruofei Du, Matthias Zwicker and Amitabh Varshney
Augmentarium | UMIACS
University of Maryland, College Park
ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2018
2
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Application | Resolution | Frame rate | MPixels / sec |
Desktop game | 1920 x 1080 x 1 | 60 | 124 |
3
Application | Resolution | Frame rate | MPixels / sec |
Desktop game | 1920 x 1080 x 1 | 60 | 124 |
2018 VR (HTC Vive PRO) | 1440 x 1600 x 2 | 90 | 414 |
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
4
* Data from Siggraph Asia 2016, Prediction by Michael Abrash, October 2016
Application | Resolution | Frame rate | MPixels / sec |
Desktop game | 1920 x 1080 x 1 | 60 | 124 |
2018 VR (HTC Vive PRO) | 1440 x 1600 x 2 | 90 | 414 |
2020 VR * | 4000 x 4000 x 2 | 90 | 2,880 |
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
5
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
6
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
fovea:
the center of the retina
corresponds to the center of the vision field
7
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
foveal region:
the human eye detects significant detail
peripheral region:
the human eye detects little high fidelity detail
8
foveal
region
foveal region
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
foveal region:
the human eye detects significant detail
peripheral region:
the human eye detects little high fidelity detail
9
96 %
27 %
Percentage of the foveal pixels
4 %
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
* Data from Siggraph 2017, by Anjul Patney, August 2017
10
11
Foveated Rendering
12
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
13
Related Work
14
Full Resolution
Multi-Pass Foveated Rendering [Guenter et al. 2012]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
15
Rasterizer
Early Z
Generate Coarse Quad
Shade
Evaluate Coarse Pixel Size
Input primitives
Coarse Pixel Shading (CPS) [Vaidyanathan et al. 2014]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
16
CPS with TAA & Contrast Preservation [Patney et al. 2016]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
17
Can we change the resolution gradually?
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
18
Perceptual Foveated Rendering [Stengel et al. 2016]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
19
Is there a foveated rendering approach
without
the expensive pixel interpolation?
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
20
Log-polar mapping [Araujo and Dias 1996]
Log-polar Mapping
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
21
Log-polar mapping [Araujo and Dias 1996]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
22
Log-polar mapping [Araujo and Dias 1996]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
23
Log-polar mapping [Araujo and Dias 1996]
Log-polar Mapping
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
24
Log-polar mapping [Araujo and Dias 1996]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
25
Log-polar mapping [Araujo and Dias 1996]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
26
Log-polar mapping [Araujo and Dias 1996]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
27
Log-polar Mapping for 2D Image [Antonelli et al. 2015]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
28
Log-polar Mapping for 2D Image
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
29
Our Approach
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
30
Kernel Log-polar Mapping
range: [0,1]
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
31
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Log-polar Mapping
32
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Kernel Log-polar Mapping
Kernel Foveated Rendering
33
34
Kernel log-polar Mapping
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
35
Kernel log-polar Mapping
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
36
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
37
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
38
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
39
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
40
Fovea
Fovea
Fovea
41
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
42
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
43
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
44
Original Frame
Buffer
Screen
Sample Map
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
45
Fovea
Fovea
Fovea
46
User Study
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
47
accept
reject
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
48
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
49
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
50
Kernel log-polar transformation
G-buffer
Inverse kernel
log-polar transformation
& post anti-aliasing
Shading &
internal anti-aliasing
World position
Bit tangent
Normal
Texture coordinates
Albedo map
Roughness, ambient, and refraction maps
Screen
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
51
original ray-marching scene
10 FPS
foveated ray-marching scene (σ = 1.8, α = 4)
30 FPS
fovea
* Scene created by Íñigo Quílez.
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
52
original 3D geometries
31 FPS
foveated 3D geometries (σ = 1.8, α = 4)
67 FPS
fovea
fovea
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
53
Scene | 3D Textured Meshes | Ray Casting | ||||
Resolution | Ground Truth | Foveated | Speed up | Ground Truth | Foveated | Speed up |
| 55 FPS | 110 FPS | 2.0X | 20 FPS | 57 FPS | 2.9X |
| 31 FPS | 67 FPS | 2.2X | 10 FPS | 30 FPS | 3.0X |
| 8 FPS | 23 FPS | 2.8X | 5 FPS | 16 FPS | 3.2X |
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
Summary
54
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
55
Ground Truth
Kernel Foveated Rendering
56
Thanks!
Introduction
Related Work
Our Approach
User Study
Experiments
Conclusion
57
Kernel Foveated Rendering
Xiaoxu Meng, Ruofei Du, Matthias Zwicker and Amitabh Varshney
Augmentarium | UMIACS
University of Maryland, College Park
ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2018
video
paper
FOVE Headset
58
User Study: Significance
59
| 1.2 | 1.4 | 1.6 | 1.8 | 2.0 | 2.2 | 2.4 |
Cochran’s Q value | 1.72 | 5.76 | 8.20 | 8.25 | 7.49 | 14.27 | 5.48 |
p-value | 0.631 | 0.122 | 0.042 | 0.041 | 0.058 | 0.002 | 0.139 |
Two-level Anti-aliasing
60
Kernel log-polar transformation
G-buffer
Inverse kernel
log-polar transformation
& post anti-aliasing
Shading &
internal anti-aliasing
World position
Bit tangent
Normal
Texture coordinates
Albedo map
Roughness, ambient, and refraction maps
Screen
Two-level Anti-aliasing
61
Inverse kernel
log-polar transformation
& post anti-aliasing
Shading &
internal anti-aliasing
Non-uniform Gaussian Blur
Kernel size increase from left (fovea) to right (periphery)
Non-uniform Gaussian Blur
Kernel size increase from fovea to periphery
Video & Paper
62
video
paper