Computer Graphics

LAB 4: Introduction to the GPU

LAB3: Render 3D meshes

LAB5: 3D Shader Lighting

Objective of this lab

• Simplify rasterization using the GPU
• OpenGL
• Introduction to shaders
• Render a screen quad using the GPU
• Basics of GLSL
• Render 3D meshes

Index

• Introduction to OpenGL
• GPU Rendering Pipeline
• Using Shaders
• Shader Programming
• FAQS
• Useful resources

Introduction to OpenGL

Recap: Graphics Processing Unit (GPU)º

The GPU is the responsible drawing to the framebuffer and submitting it to the screen.

• Focus on accelerate rasterization → architecture designed for that!
• CPUs could focus in other tasks meanwhile GPUs can work in parallel

Recap: What is OpenGL?

OpenGL (Open Graphics Library) is a standard specification that defines an API multi-language and multi-platform, to write applications that render 2D and 3D graphics.

• This specification is a document that describes a set of functions and the exact behaviour of those functions.

From this specification (or API), manufacturers implemented every function of this API transforming the calls to their internal GPU calls, and published this with their drivers.

There are several versions of OpenGL ( OpenGL 1.1, OpenGL 3, OpenGL 4.4 and OpenGL ES).

Graphics APIs

We are using OpenGL in this lab for simplicity, but it’s important to know that there’s more graphics APIs besides it, for example:

• Direct3D (a subset of DirectX)
• Metal developed by Apple
• Vulkan
• WebGPU (new API for the web)

​

​

Benefits of using OpenGL

• Its hardware and language independent (no matter the GPU or language used) and cross platform, there are versions of OpenGL for Windows, Linux, Mac, and even mobile devices.
• You can render much faster due to dedicated hardware acceleration, allowing to raster millions of polygons per second in the latest GPUs.
• You do not need to know the rasterization algorithms, because that is handled by the GPU. That implies the CPU is free to do other tasks
• Because the calls are very low-level you can build any kind of application on top of OpenGL (games, 3D editors, visualizations, engines, etc).

​

What is NOT OpenGL?

• It is not a programming language. You can call OpenGL from any programming language.
• It is not a paint application. OpenGL only allows to raster primitives.
• It is not a graphics engine. It doesn’t have basic functions as loading images, 3D meshes, etc.
• It is not a 3D scene manager. You cannot define and manage the scene objects.
• It is not a videogames engine. It doesn’t know anything about user-input, scene, collisions, physics, etc.

​

OpenGL is just used as a way to raster triangles very fast.

​

Common OpenGL actions

• Raster basic geometric primitives (draw points, lines and triangles)
• Apply transformations to vertices (project, translate, rotate and scale)
• Avoid occlusions between triangles at different depth (using a Z-Buffer)
• Blend between different pixels (support transparences)
• Texturing, projecting images onto the primitives to add detail

Architecture of a CPU+GPU app

When you code an application, it must communicate with the GPU via OpenGL functions.

​

These functions will be translated from the driver to the actions that are sent to the GPU and the GPU will execute them changing the pixels on the framebuffer.

​

​

OpenGL primitives

When you want to raster you must specify the primitive type and according to the vertex array it will raster one shape or another.

They can be divided in three groups:

• Points
• Lines
• Triangles

Depending on the number of vertex provided it will draw more or less primitives.

​

GL_TRIANGLES is used by default in your framework

Framebuffer

The GPU has its own framebuffer, stored in the VRAM (GPU’s RAM) so we do not have direct access from the CPU.

​

The amount of memory used by the framebuffer comes from this formula (the same as our previous framebuffer):

Screen Width  x  Screen Height  x  number of channels  x  bytes per channel = image size in bytes

Double Buffer

One common problem when rendering the scene is that you do not want the user to see the image until all objects have been rendered, but if we render to the framebuffer the user would see how the image gets build constantly.

To solve this problem we use double buffering, where we use two buffers, the one being displayed and the one where we render the image.

Once we finish rendering the image we swap both buffers so the user can see always the final image and not the process.

​

​

How to render in OpenGL (step by step)

1. Clear the screen (set all pixels to one specific color) and clear the Z-buffer (set the Z to a very far value).
2. For every object: Send to the GPU all the geometry information (vertices, etc).
3. Execute draw call specifying the primitive type.

The GPU will read all the vertices and according to the primitive type it will render on the screen the appropriate shapes.

• Once finished, swap buffers to display the final image into the screen.

Basic render function

void render()

{�     // Clear the framebuffer and the depth buffer�    glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );��    // Draw scene

    // ...��    // Swap between front and back buffer� // This method name changes depending on the platform�    glSwapBuffers();

}

OpenGL States

Sometimes you want to disable some parts of the rendering pipeline.

OpenGL stores lots of flags to control which steps are performed during the rasterization.

To change those flags you can call the glEnable and glDisable, and other functions of openGL.

Once we change a state it will be applied to all the draw calls that are sent after that.

​

void drawTriangles() {�

// Disable Z-buffer testing

glDisable( GL_DEPTH_TEST );

​

// This triangle wont take into account occlusions

renderTriangle();

​

// Enable Z-buffer testing

glEnable( GL_DEPTH_TEST );

​

// This triangle will take into account occlusions

renderTriangle();

}

Z-Buffer and Occlusions

The GPU is in charge of processing occlusions using its own depth buffer (Z-buffer). The only thing we have to do is clearing the depth buffer at the beginning of each frame:

​

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);�

and enable it before rendering an object:

​

glEnable( GL_DEPTH_TEST );

​

GPU Rendering Pipeline

CPU vs GPU rasterization

• Recap: GPUs are hardware specifically designed to rasterize primitives
• Parallelization: Raster several pixels simultaneously (results from one doesn’t affect others)
• Lots of cores (shader units): Execute lots of GPU programs in parallel
• VRAM: Fast memory access to primitive information

​

​

CPU

Optimized for serial tasks

GPU

Optimized for many parallel tasks

Coding inside the GPU: Shaders

A shader is a piece of code meant to be executed inside the GPU, designed to overwrite the rendering pipeline used when rasterizing primitives.

• Render very complex surfaces that mimics real objects
• Freedom to create more interesting and realistic scenes

Shaders have limitations (mostly due to memory constraints):

• No access to the variables in your CPU code or the memory of your application (because of shaders being executed inside the GPU)

​

Shaders: Coding language

There are several languages for coding shaders, depending on the API used:

• GLSL: OpenGL Shading Language; part of the OpenGL standard
• HLSL: High Level Shader Language, DirectX; developed by Microsoft
• WGSL: WebGPU Shading Language, WebGPU standard
• CG: C for Graphics; developed by NVidia

There is not much difference between them in terms of syntax or performance, as they are all compiled to the assembly code of the GPU.

​

​

Shaders: Coding language

From now on, we will use GLSL to explain shaders as it is the one used by OpenGL:

• High-level shading language with a syntax based on the C programming language
• It has some differences mostly due to the memory constraints
• Same operators as in C and C++, with the exception of pointers
• Support for loops and branching
• Recursion is forbidden!

​

​

Shader types

The mandatory shaders that every rendering pipeline must have are:

• Vertex shader: Executed for every vertex. It is used mostly to project from local coordinates to clip coordinates (use mvp)
• Output: Vertex in clip-space [-1.0, 1.0]
• Fragment (or Pixel) shader: After the rasterization of the triangles; executed for every pixel
• Output: Final color to be draw into the framebuffer

​

GLSL variable types

Some of the basic types and classes in GLSL:

• bool: conditional types (true or false)
• int: integer numbers
• uint: unsigned integer numbers
• float: floating point numbers
• double: double-precision floating point numbers (double size!)
• vec2: vector of 2 components
• vec3: vector of 3 components
• vec4: vector of 4 components
• mat3: matrix 3x3
• mat4: matrix 4x4
• sampler2D: 2D textures

Data Type (GLSL) - OpenGL Wiki (khronos.org)

GLSL variable types

Access to the components of a vector (swizzling). E.g. A Vector4 (vec4):

​

• v.x, v.y, v.z, v.w
• v.xy, v.xyz to get several components
• v.yyz, v.zyx is also valid!

GLSL variable types

• You can build a vec3 from a float like this: � vec3( 1.0 ); // This will create a vec3(1.0,1.0,1.0)

​

• We can convert a vec3 to vec4 by calling the constructor and passing the fourth component� vec4 v = vec4( position, 1.0 );

​

• GLSL types have operators!

vec4 a = v1 + v2;

vec4 b = v1 * v2;

GLSL functions

Here is a list of common GLSL functions that you can use:

• pow( a, b ): Returns a raised to the power of b
• normalize( a ): Returns the vector normalized
• dot( a, b ): Returns the dot product between two vectors
• cross( a, b ): Returns the cross product between two vectors
• length( a ): Returns the length of a vector (its module)
• distance( a, b ): Returns the distance of 2 vectors
• reflect( v, n ): Returns the reflected incident vector based on that normal
• mix( a, b, f ): Linear interpolation between a and b using f as factor
• max( a, b ): Returns the max value
• min( a, b ): Returns the min value
• step( edge, a ): Step function that returns either 0 if a is less than edge or 1 if not
• clamp( a, minv, maxv ): Returns a clamped between minv and maxv

​

GLSL reserved keywords

From the Vertex Shader:

• gl_Vertex (vec4): input - Contains the vertex coordinate*
• gl_Normal (vec3): input - Contains the normal coordinate*
• gl_MultiTexCoord0 (vec2): input - Contains the texture coordinates (UVs)*
• gl_Position (vec4): output - Store the projected coordinate in clip-space

From the Fragment Shader:

• gl_FragColor (vec4): output - Store the final color (from 0 to 1) to draw into the framebuffer
• discard: Stops processing the pixel and ignores it

​

*: as it was stored in the vertices array

Global variables: attribute

Shaders have 3 types of global variables, depending on where the variables comes from:

• attribute: Comes from the mesh itself, is the information that is passed to the GPU. Variables associated to every vertex that is read from the vertex shader
• Position, normals, texture coordinate, color
• Using gl_Vertex, gl_Normal and gl_MultiTexCoord0 you can avoid defining them

​

Global variables: Uniform and Varying

• uniform: Variables passed from our application to the GPU
• View projection matrix, camera/light position, material information, time, etc.
• varying: Variables that we want to pass from the VS to the FS. Using this keyword the variable will arrive interpolated to the pixel according to the distance to every vertex.
• Position, normals

Is it possible to have a regular global variable? Yes, without specifying a type, but it will be available only in the current shader stage!

​

​

​

Vertex Shader

​

​

Fragment Shader

Application

varying

(interpolated)

Mesh Data

attribute

gl_Vertex�gl_Normal�gl_TexCoord

uniform

GPU

CPU

gl_Position

gl_FragColor

Executed per vertex

Executed per pixel

Framebuffer

V0,V1,V2,... N0,N1,N2,...

uniform

Using Shaders

Using shaders

The most common case would be to replicate how the light behaves in the real world by coding an algorithm to paint every pixel of a primitive in 3D.

​

Same mesh rendered with different shaders

Using shaders

First step: 2D shaders; e.g. create images or apply effects to already existing images

• Instead of using a complex mesh, we will upload a quad mesh (only 2 triangles)
• Adjust clip-space position of each vertex
• We can render it covering the screen and use a shader to choose the color for each screen pixel (simulating a classic framebuffer)

​

www.shadertoy.com/

Using shaders: Draw 2D formulas

The result of the equation defines the color of the pixel:

• For every pixel of the image we evaluate a formula to compute the R, G, B values and display them:

R = f(x,y) G = f(x,y) B = f(x,y)

• The more complex the formula the more interesting the image will be

Examples: https://tamats.com/apps/mathnimatics/

​

​

Using shaders

To render meshes (from quads to other 3D meshes) we have get some things clear first:

• The most important thing is that we don’t have a custom framebuffer
• Render calls must use OpenGL to communicate with the GPU
• The Mesh class has a method Render() that already communicates with the GPU via OpenGL.

// Enable shader and upload needed variables from CPU

// ...

mesh->Render();

​

Using shaders

In case of rendering 3D meshes, we need some variables coming from the CPU: the model and the view-projection matrices.

• We no longer project vertices on the CPU, but that doesn't mean we don't have to.
• Since it’s an action that affect each vertex, we must do it in the vertex shader
• Remember the vertex shader has as input the vertices in local space and output the vertices in clip-space, so we need to convert them using the CPU matrices.

​

Using shaders

The framework encapsulates the lower level OpenGL calls, so your code for creating a shader in your application will be VERY simplified:

Shader * shader = Shader::Get("shader.vs","shader.fs");

When rendering, there are some steps that must be followed once the shader is created:

• Clear buffers and set OpenGL flags
• Enable shader, upload data and render the mesh
• Disable shader

shader->Enable();

shader->SetMatrix44(...);

mesh->Render();

shader->Disable();

Clearing buffers

Since OpenGL controls now the framebuffer, we must call some of its functions before starting to draw anything (already being called in the framework).

// Set the background color of the framebuffer (0..1 range!)

glClearColor(r, g, b, alpha);

​

// Clear the window and the depth buffer (occlusions)!

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

On window creation

Main loop, before Application::Render

Depth Occlusions

Since OpenGL controls now the framebuffer, we must call some of its functions before starting to draw anything.

// Enable depth testing for occlusions

glEnable( GL_DEPTH_TEST );

​

// Z will pass if the Z is LESS or EQUAL to the Z of the pixel

glDepthFunc(GL_LEQUAL);

Uploading data to the GPU

Again, the framework encapsulates the lower level OpenGL calls to upload data to the GPU (that will be used in your shader):

// Call variable specific method to upload data

shader->SetFloat("u_value", 1.0);

shader->SetMatrix44("u_viewprojection", viewprojection);

shader->SetTexture("u_texture", texture);

​

Once the data is at the right place, you can render the mesh (Mesh class has a method to render itself using the GPU, check the framework!).

Full Screen Quad Shader

Rendering a full screen quad is very simple since we can avoid the projection of its vertices.

The 6 vertices composing both triangles are in the [-1..1] range, so we already have it in clip-space → No projection in the vertex shader!

-1, 1

1, 1

1, -1

-1, -1

-1, 1

1, -1

Full Screen Quad Shader

VS:

// Store the UVs to use later interpolated in the fragment shader�varying vec2 v_uv;

void main()�{

// Set vertex uvs� v_uv = ...;

// Output clip-space� gl_Position = ...;�}

​

​

FS:

​

varying vec2 v_uv;

​

void main()�{� gl_FragColor = ...;�}

Reading textures from shaders

You can read pixels from an image passed to the shader (texture) using the function texture2D in GLSL:

vec4 color = texture2D( u_texture, uv );

• texture2D returns a vec4 containing the color in RGBA of the pixel in the range [0..1]
• UVs are normalized (0,0 is bottom-left and 1,1 is top-right)

Quad texture shader

FS:

​

// Receive the uvs interpolated from the vertex

shader

varying vec2 v_uvs;

​

// Receive the texture as a sampler2D from our application

uniform sampler2D u_texture;

​

void main()�{

// Fetch sampler

vec4 texture_color = texture2D( u_texture, v_uvs );

​

// Assign the color to the pixel� gl_FragColor = texture_color;�}

Applying effects

You can add visual effects before sending the final color to the framebuffer:

• Use modified texture coordinates
• Apply math functions
• Use 2D transformations
• Use time to get dynamic values!
• Use noise values (for random values)
• Apply changes to the final color using�image effects (pixelize, change saturation/hue/contrast, vignetting, chromatic aberration...)
• Research about more of them!

Chromatic aberration

Applying effects and transformations

FAQ

1. The shader doesn’t work but Visual Studio compiles perfectly. Remember that Visual Studio doesn't compile your shaders, that part is done during execution so if there are errors in the shader you will see them in the console of your application.
2. Shader compiler gives an error of types not compatible. Shaders do not allow to assign a vec3 to a vec4, or the opposite. You must pay attention to the types of every data. gl_FragColor and gl_Position must be vec4.
3. I get errors when operating with numbers. Be careful, older versions of GLSL do not allow to operate between floats and ints, you must cast them.
4. I get an error when I try to modify a varying var from the Fragment Shader. In some GPUs the varying vars in the fragment shader are constant and cannot be modified, so copy them to another variable.
5. I cannot multiply a vec3 by a mat4. You must convert the vec3 to vec4 in order to multiply by a mat4.
6. I get an error when I read gl_Vertex, gl_Normal or gl_Color from my fragment shader. Those variable are restricted to the Vertex Shader, if you want them in the Fragment Shader you must pass them using varyings.

Useful resources

Shadertoy

The book of shaders