Demystifying Graphics in Video Games

by Michael Sukkarieh

Who am I?

• Michael Sukkarieh
• U2 Software engineering student
• Experience:
• Worked at Ubisoft Montreal as a graphics programmer summer 2018
• Dabbled with a couple graphics API’s and engine development on the side
• Most of my experience related to computer graphics is low-level system implementation and less so fancy lighting methods

​

Goals (What I hope you’ll know more about after listening to me talk for an hour)

• How the visual component of a game object is represented
• How a frame in a video game is created (rendered)
• Some common computer graphics practices and ideas
• A glimpse at some more advanced stuff?

​

“I just want to make games, why should I care about this?”

• Will help you understand a little bit more of the bigger picture
• If you’re using a known game engine (Unity, Unreal), knowing the basics of graphics programming can help understand what is going on in your scene

​

Visual representation of a game object

• The visual component of a game object is primarily composed of:
• Triangle Mesh
• Material

Triangle Mesh

• Every real-world 3d object can be approximated using triangles
• Triangles are made up of 3 vertices, which can contain several vertex attributes
• Basic vertex attributes include:
• Position vector
• Normal vector
• UV coordinate

Vertex Attribute Interpolation�

• Note that we described a triangle with 3 vertices which have attributes (position, normal, uv)
• But what about the points in the triangle?

Material

• The material associated with an object describes what an object should look like and how it should interact with light
• Should the object be matte or shiny? Blue or red?
• The material also contains the textures the object uses
• The diffuse texture describes the base color of a point on the object
• The material may also have a normal texture

Shininess:

Glossiness:

Normal Texture:

Diffuse texture:

Lighting

• Lighting is a huge factor in how realistic and aesthetically pleasing a game is
• There are two major lighting “models”
• Local illumination
• Global illumination

The image on the left is what GTA 5 looks like if we only had diffuse colors. The image on the right is after the black magic is done (note: it involved more than just lighting)

Source: http://www.adriancourreges.com/blog/

Diffuse and Specular reflection

• Diffuse reflection: light rays scatter in all directions when they hit a surface because no surface is perfectly flat
• Specular reflection: if a surface is sufficiently smooth, light rays that hit the surface may be able to reflect sharply through a cone of reflectance

Local (direct) Illumination

• Only light hitting an object directly from the light source is taken into account
• Intensity of the light is computed using lambert’s cosine law with the vector from the point to the light source and the normal vector at that point

Global (indirect) illumination

• Light coming from the light source and the rest of the objects in the world are taken into account
• Required for photorealistic rendering
• Complex and expensive

The Graphics Pipeline

Render Target

• The graphics pipeline renders to the currently bounded render target
• In the most basic case, this is just the final frame that the player sees (i.e. the color buffer)
• All the buffers used to render the final frame are included in the frame buffer
• It’s possible to render to multiple render targets at the same time (MRT)

�https://gyazo.com/9438bed2c2177329bceb7398998d9231

Source: http://www.adriancourreges.com/blog/

“All these buffers were generated in about 1900 draw calls.”

Depth/Stencil Test

• Sometimes we don’t want to replace the value of a pixel on the render target
• Ex: if the color buffer contains the color of an object which is in front of the object we’re currently drawing, we don’t want to replace the pixel value
• This can be done using the depth and stencil buffers

Shaders

• Put simply, a shader is a program that runs on the GPU
• We need a vertex and pixel shader in order to draw something
• Vertex shaders accept a vertex and return a vertex (used for transforming the vertex)
• Pixel shaders return a color (final color of the pixel)

Vertex Shader Stage

• The main purpose of the VS stage is to transform the vertex from model space into homogenous clip space

Pixel Shader Stage

• This is where the magic happens (for the most part). Stuff that is often done here includes:
• Sample textures
• Calculate lighting
• etc
• The pixel shader returns the final color of a pixel

​

Note: several steps that happen between the VS and PS are were skipped (like figuring out which pixels are covered by a given primitive)

More Advanced Topics

Forward vs Deferred Rendering

• Forward rendering computes lighting for every pixel that is drawn to the color buffer
• Pros: simple, low memory cost, supported by old GPU’s
• Cons: extremely slow if you have a lot of lights in your scene, does not scale well
• Deferred rendering calculates the diffuse opaque colors first, and then computes lighting and resolves the two (i.e. lighting/shading is deferred)
• Pros: can support a large number of dynamic lights, faster most of the time
• Cons: complex (needs MRT), high memory cost (multiple buffers)

How Deferred Rendering Works

• Each light source has it’s own geometric shape, and is rendered into a light accumulation buffer
• Pixel must pass depth and stencil test to be rendered onto the buffer (i.e. depth and stencil buffers are used to insure that the pixel currently on the diffuse buffer is contained within the geometry of the light)

Point light geometry:

Post Processing FX

• Post processing effects can make a game look much better, examples:
• DOF (depth of field)
• Color correction
• Bloom
• Etc
• Post processing effects work with the final frame after everything is rendered onto it (i.e. after the scene is done rendering)

​

​

Post Processing FX Framework

• The final frame (which was previously a render target) is now used a shader resource
• A square, representing the frame, is constructed and passed back into the graphics pipeline with the frame as a resource
• Post processing shader then works over the entire frame

Square made up of 4 vertices gets mapped to an image of the final frame

Questions ?

Thank you!