1 of 58

Computer Graphics�Behind the Smoke and Mirrors

ARMAN PAPIKAN

Technical Lead

2 of 58

Introduction

  • Born and raised in Gyumri,
  • Currently Technical Lead at Ubisoft,
  • Around 8 years in the industry,
  • 4 years specializing in graphics

​

Worked On:

  • Various hypercasual mobile games – generalist programmer,
  • AR / VR – generalist programmer,
  • Architectural software – generalist programmer,
  • Steam game (Sim Casino) – render programmer,
  • Rainbow Six: Mobile – render programmer,
  • Anvil Pipeline – render programmer

3 of 58

credit: Branch Education youtube channel

4 of 58

5 of 58

6 of 58

7 of 58

Rasterization

8 of 58

Rasterization

9 of 58

10 of 58

11 of 58

Rendering order

  • Need to sort before / during rendering to achieve expected order
  • The hardware exposes depth buffer to solve this
  • Each pixel writes color into color buffer and depth into depth buffer
  • We compare if the new color is closer to camera or further before rewriting existing color
  • Also helps with performance (reduces overshading),�But requires object sorting Front-To-Back for Depth-Tested geometry

12 of 58

credit: Adrian Courrèges

https://www.adriancourreges.com/blog/2015/11/02/gta-v-graphics-study

13 of 58

14 of 58

15 of 58

16 of 58

17 of 58

18 of 58

Base

19 of 58

+ Water

20 of 58

+ Fog

21 of 58

Fog + Sky

22 of 58

Fog + Sky + Clouds

23 of 58

Deferred rendering

  • Rasterize the virtual scene,
  • Store all the relevant information for future use,
  • Post processing (SSAO, etc),
  • Pre-baked reflection maps

​

24 of 58

Forward vs Deferred

Prepass

Material + Lighting

Final

Prepass

Final

Material

Lighting

25 of 58

Forward vs Deferred vs VisBuffer

Prepass

Material + Lighting

Final

Prepass

Final

Material

Lighting

Prepass

Final

Visibility

Vis Mat +

Lighting

Instance ID,

Primitive ID,

Barycentric Coord,

Depth

Material ID

26 of 58

VisBuffer

27 of 58

G-Buffer vs VisBuffer

28 of 58

Forward vs Deferred vs VisBuffer

Prepass

Material + Lighting

Final

Prepass

Final

Material

Lighting

Prepass

Final

Visibility

Vis Mat +

Lighting

Instance ID,

Primitive ID,

Barycentric Coord,

Depth

Material ID

6 – 10 bytes pp

16 – 32 bytes pp

0 bytes pp

29 of 58

Trade offs / Comparison

  • Forward has lowest memory bandwidth 👍, but highest overshading 👎. Has high algorithmic complexity 👎 for lights O(h * w * objN * lightN).

​

  • Deferred is convenient, powerful 👍 and has lower 👍 algorithmic complexity for lights O(h * w * lightN). Requires high bandwidth 👎.

​

  • Visibility buffer has all the advantages of classic deferred 👍, but lower bandwidth 👍.

​

�Where h = screen height, w = screen width,�objN = number of objects affected by lights, �lightN = number of lights.

​

​

30 of 58

31 of 58

Red Line – signal to be captured

​

Blue Line – sampling

Blue Dots – sampled points

​

The signal frequency is higher than the sampling frequency, so we lose information because we are lacking enough resolution to capture the full signal.

32 of 58

33 of 58

Anti-Aliasing Algorithm (MSAA / FXAA)

​

  • Sample neighboring pixels (4x, 9x, 16x),
  • Take the average of the group of pixels

​

MSAA – hardware accelerated, works on edges after raster step

FXAA – post processing shader

​

MSAA + Cheaper, works well with Forward

MSAA – doesn’t work with Deferred

​

FXAA works on the final frame

34 of 58

35 of 58

Temporal Anti-Aliasing (TAA)

Stepping back in time

​

Want to correlate current frame pixel with pixel from previous frame(s)

​

Need some spatial translation to work

Static: introduce jitter,

36 of 58

Temporal Anti-Aliasing (TAA)

Stepping back in time

​

Want to correlate current frame pixel with pixel from previous frame(s)

​

Need some spatial translation to work

Static: introduce jitter,

37 of 58

Temporal Anti-Aliasing (TAA)

Stepping back in time

​

Want to correlate current frame pixel with pixel from previous frame(s)

​

Need some spatial translation to work

Static: introduce jitter,

Dynamic: calculate motion vectors and reproject

38 of 58

Temporal Anti-Aliasing (TAA)

Stepping back in time

​

Blend:

Ex: 0.9 * cur – 0.1 * prev

39 of 58

​

+TAA: Quality, Works well with deferred rendering.

-TAA: Requires higher memory bandwidth and computations.

�Has bad edge cases.

40 of 58

Temporal Anti-Aliasing (TAA)

Dynamic objects

​

Result

41 of 58

TAA in Doom 2016

42 of 58

TAA in Doom 2016

43 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

�vkCmdBindDescriptorSets(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);

�vkCmdBindVertexBuffers(commandBuffer, 0, 1, &_vertexBuffer, _offsets);

vkCmdBindIndexBuffer(commandBuffer, _indexBuffer, 0, VK_INDEX_TYPE_UINT32);

�vkCmdDrawIndexed(commandBuffer, _index_count, 1, 0, 0, 1);

�vkCmdEndRenderPass(commandBuffer);

VkResult result = vkEndCommandBuffer(commandBuffer);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

�glActiveTexture(0);

glBindTexture(GL_TEXTURE_2D, tex);

glUniform1i(glGetUniformLocation(shaderProgram, "tex"), 0);

�glBindVertexArray(VAO);

�glDrawElements(GL_TRIANGLES, _index_count, GL_UNSIGNED_INT, 0);

Graphics API (OpenGL / Vulkan)

44 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

Set up the frame

Graphics API (OpenGL / Vulkan)

45 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

Use shader

Graphics API (OpenGL / Vulkan)

46 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

�vkCmdBindDescriptorSets(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

�glActiveTexture(0);

glBindTexture(GL_TEXTURE_2D, tex);

glUniform1i(glGetUniformLocation(shaderProgram, "tex"), 0);

Use Texture

Graphics API (OpenGL / Vulkan)

47 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

�vkCmdBindDescriptorSets(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);

�vkCmdBindVertexBuffers(commandBuffer, 0, 1, &_vertexBuffer, _offsets);

vkCmdBindIndexBuffer(commandBuffer, _indexBuffer, 0, VK_INDEX_TYPE_UINT32);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

�glActiveTexture(0);

glBindTexture(GL_TEXTURE_2D, tex);

glUniform1i(glGetUniformLocation(shaderProgram, "tex"), 0);

�glBindVertexArray(VAO);

Use vertex & index buffers

Graphics API (OpenGL / Vulkan)

48 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

�vkCmdBindDescriptorSets(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);

�vkCmdBindVertexBuffers(commandBuffer, 0, 1, &_vertexBuffer, _offsets);

vkCmdBindIndexBuffer(commandBuffer, _indexBuffer, 0, VK_INDEX_TYPE_UINT32);

�vkCmdDrawIndexed(commandBuffer, _index_count, 1, 0, 0, 1);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

�glActiveTexture(0);

glBindTexture(GL_TEXTURE_2D, tex);

glUniform1i(glGetUniformLocation(shaderProgram, "tex"), 0);

�glBindVertexArray(VAO);

�glDrawElements(GL_TRIANGLES, _index_count, GL_UNSIGNED_INT, 0);

Draw call

Graphics API (OpenGL / Vulkan)

49 of 58

vkBeginCommandBuffer(commandBuffer, &cmdBufInfo)

vkCmdBeginRenderPass(commandBuffer, &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);

�vkCmdBindPipeline(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline);

�vkCmdBindDescriptorSets(commandBuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayout, 0, 1, &descriptorSet, 0, nullptr);

�vkCmdBindVertexBuffers(commandBuffer, 0, 1, &_vertexBuffer, _offsets);

vkCmdBindIndexBuffer(commandBuffer, _indexBuffer, 0, VK_INDEX_TYPE_UINT32);

�vkCmdDrawIndexed(commandBuffer, _index_count, 1, 0, 0, 1);

�vkCmdEndRenderPass(commandBuffer);

VkResult result = vkEndCommandBuffer(commandBuffer);

glClearColor(0.f, 0.f, 0.f, 1.f);

glClear(GL_COLOR_BUFFER_BIT);

�glUseProgram(shaderProgram);

�glActiveTexture(0);

glBindTexture(GL_TEXTURE_2D, tex);

glUniform1i(glGetUniformLocation(shaderProgram, "tex"), 0);

�glBindVertexArray(VAO);

�glDrawElements(GL_TRIANGLES, _index_count, GL_UNSIGNED_INT, 0);

Finish

Graphics API (OpenGL / Vulkan)

50 of 58

Each GPU comes with a driver, that exposes the function pointers for supported Graphics APIs

​

Graphics API helps standardize access to GPUs

​

Centralized way of sending commands to GPU from CPU

​

Engine / Graphics programmers use the Graphics API to implement the desired features and techniques

Graphics API (OpenGL / Vulkan)

51 of 58

So many interesting things to cover

  • How does lighting (Global Illumination) work in games?
  • How do shadows work in games?
  • How is intense light (high LUX) mapped to monitor display range?
  • How are different material interactions with light simulated?

​

  • Various optimization techniques (Culling, Batching, etc)
  • GPU-Driven rendering.

​

52 of 58

General areas in Graphics Programming�and required skills

  • Hardware (Graphics API, Graphics Pipeline, …),
  • Techniques (Deferred Pipeline / VisiBuffer / SSAO / TAA, …),
  • Computer Science (Algorithms, Math, …),
  • Tooling (Renderdoc, PIX, Nsight, ...),
  • Optimization (Measurement, Analysis, …),
  • Other (Color theory, Physics and optics, Statistics, …)

53 of 58

References

  • Branch Education on youtube�https://youtu.be/C8YtdC8mxTU?si=Itxxogi3lf7fqjV2
  • Graphic Studies (GTA V, Doom2016 and other) https://www.adriancourreges.com/blog/2015/11/02/gta-v-graphics-study
  • Visibility buffer�http://filmicworlds.com/blog/visibility-buffer-rendering-with-material-graphs
  • Temporal Anti-Aliasing (TAA)�https://sugulee.wordpress.com/2021/06/21/temporal-anti-aliasingtaa-tutorial/
  • Temporal Reprojection in INSIDE�https://youtu.be/2XXS5UyNjjU?si=_2f9b0fpUXLpFLun
  • Real-Time Rendering, 4th Edition

54 of 58

THANK YOU FOR THE ATTENTION!

55 of 58

Tonemap Doom 2016

56 of 58

Tonemap Doom 2016

57 of 58

Color Grading Metal Gear Solid V

58 of 58

Color Grading Metal Gear Solid V