NVIDIA RTX Ray Tracing
And Unreal Engine
Questions
How Did They Get This Working In Real Time?
Something Something Denoising?
How Did They Get This Working In Real Time?
4 Nvidia Volta GPUs ($60-100k machine)
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
The issue: current state of the art ray-tracers only have time to trace a few rays per pixel at 1080p/30hz. ~Doubles every few years, but counteracted by rising display resolutions. Current budget (in games, not the short) is ~1 short path per pixel.
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:� - Uses “next event event estimation”
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:� - Uses “next event event estimation”� - Rasterize first hits
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:� - Uses “next event event estimation”� - Rasterize first hits� - Use low-discrepancy halton sequence instead of uniform sampling to reduce variance
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:� - Uses “next event event estimation”� - Rasterize first hits� - Use low-discrepancy halton sequence instead of uniform sampling to reduce variance� - Maximum depth of 1, so (camera-surface-light) and (camera-surface-surface-light) paths only.
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Motivated by Deep Image Completion, but with additional features designed specifically for Monte Carlo Rendering.
Primary contributions above and beyond prior image completion work:� - RNN for temporal stability� - End-to-end training with additional information available in the rendering setting� (depth, normals, material roughness)
Their Path Tracer:� - Uses “next event event estimation”� - Rasterize first hits� - Use low-discrepancy halton sequence instead of uniform sampling to reduce variance� - Maximum depth of 1, so (camera-surface-light) and (camera-surface-surface-light) paths only.� - In order to aid training, they remove the texture from the first object hit, and then re-apply the� the texture after lighting is calculated.
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Noisy input plus normals, depth and material properties.
Reconstruction loss against smooth image rendered non-interactively.
Fully convolutional, so this can be trained on small crops (128x128)
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf
Interactive Reconstruction of Monte Carlo Image Sequences�using a Recurrent Denoising Autoencoder [1]
Training:� - Generate a camera fly-through of an environment.� - For each frame, generate 10 different noisy estimates as a way of collecting more data� - Render full image, but cut out 128x128 crops over seven frames � - Also can use reverse playback of the camera motion for more data augmentation� - Use L1 loss on both pixel values and image gradients� - Temporal loss to penalize flickering between frames
[1] Chaitanya et al. 2017 http://research.nvidia.com/sites/default/files/publications/dnn_denoise_author.pdf