2 of 58

Introduction

Until now we have simulated the light reflected by a surface by taking just into account the NdotL (Lambertian reflectance), which approximates quite well the diffuse component of a very rough material.

This may work in many situations but we know light doesn't reflect the same in metal, plastics, skin, rubber, and other materials.

We will need a way to improve our material and light reflection model.

3 of 58

Light equation vectors

Because we will talk about light equations it is important to refresh in our mind the vectors that we will require for our computations:

N: normal vector at the point
L: vector toward the light
V: vector towards the eye
R: reflected L vector
H: half vector between V and L

All this vectors must have module one.

4 of 58

Phong

Phong is a nice simplification, it is not perfect but it is a cheap starting point to model materials, and has been in use for decades.

It separates the amount of light reflected by a surface between Ambient, Diffuse and Specular, each one controlled by independent factors.

This gives freedom to artist to create impossible materials but makes it hard to create realistic ones.

5 of 58

Specular factor

The problem with our current diffuse model based only on NdotL is that objects should have bright spots (also called hotspots) to make us distinguish between more polished materials and more rough materials, what we call specularity.

And these bright spots should be affected by the eye position as they are reflections.

So we must introduce two new factors: the specular factor (how much specular) and the glossiness factor (the size of the hot spot).

//compute reflected eye vector

vec3 R = reflect(N,V);��//compute how aligned is the eye with R

float RdotV = max( dot(R,V), 0.0 );

//make it less linear using a glossiness factor

float specular = pow( RdotV, glosiness_factor );

//add it to the final light contribution

light += light_color * specular * spec_factor;

6 of 58

Specularity

Now thanks to the specular factor and the specular glossiness we can control the hotspots in our render, which helps to convey information about the material properties (how polished it is)

7 of 58

Cubemap Reflections

We know that polished materials tend to reflect light almost perfectly, but our model until now doesn't take into account reflections of the environment, only direct light reflected.

There will be a chapter focused on reflections later, but for now what about using a cubemap that contains the environment information so we can fetch the color given a direction?

It won't be true reflections (as you can see in the example, the ball is not reflected as it is not stored in the texture) but the effect works if the user is not paying much attention.

//compute reflected eye vector

vec3 R = reflect(N,V);

//fetch the color from the texture

color = textureCube( u_env_texture, R );

8 of 58

Reflection + Phong

So now we can compute a fake reflection, but how do we blend this with our existing phong model?

Well, we can try to have a reflection factor in our material properties and use some linear interpolation between our phong result and the reflection computed from the cubemap.

Keep in mind that if you are using multipass blending this won't work, in that case consider using premultiplied alpha.

//linear interpolation

final_color = mix( final_color,

environment_color,

reflection_factor );

9 of 58

Fresnel

In the real world materials have a different reflection component depending on the angle at which we see them. This is called fresnel component.

This can be approximated using the dot product between the V vector and the normal.

To have more control over the function we can use an exponential.

//compute fresnel factor

float fresnel = 1.0 - max( dot(N,V), 0.0 );

//make it less linear...

fresnel = pow( fresnel, 2.0 );

10 of 58

Refractions

Following the trick we used for reflections, we can do it also to simulate refractions (glass materials) by computing the refracted vector and sampling from the environment cubemap.

And as with the reflections, we won't really see what's behind, but it is a simple approach that conveys to the user that this object is made of a refractive material.

We could also apply fresnel effect here.

//compute refracted vector with IOR 0.8

vec3 R = refract(V,N,0.8);

//fetch the color from the texture

ref_color = textureCube( u_env_texture, R );

11 of 58

Beyond Phong

And we can keep improving our model just by looking at reality and making small changes to our shader to mimic every effect we detect in the real world.

But this approach doesn't helps us get closer to a realistic rendering as we are doing a perceptual approach, not a physical approach, and perception can be deceiving.

If we really want to have a realistic rendering we need to start considering how light really behaves.

12 of 58

Physically Based Rendering

13 of 58

Physically Based Rendering

The idea behind Physically Based Rendering (PBR) is to try to replicate how light behaves in the real world, using formulas that replicate real physics instead of visual approximations.

For instance, taking into account the energy conservation theorem, that is, a surface cannot reflect more energy than the energy received (assuming a surface is not emissive)

This will make the light equation a little bit more complex but gives more accurate results.

You can download for free the most famous book about PBR from here.

14 of 58

The Cornell Box

The idea behind Physically Based Rendering was introduced by the Cornell University Program of Computer Graphics around 1997.

Their idea was simple, just take a picture of a very simple environment, take precise measures of every element (not only its shape but also its material properties and light energy) and try to recreate it digitally until both images match perfectly.

They called it the Cornell Box, and you can explore it here.

Can you guess which one is the photo and which one the render?

15 of 58

Diffuse + Specular

The Phong formula wasn't so bad after all, it's main problem was that diffuse and specular are not related in any way inside the formula, which ends up breaking the energy conservation principle (more light can be reflected than light is received).

At the same time the diffuse component defined as NdotL makes sense as there is always going to be a percentage of the light that gets scattered in all directions.

The main problem was the specular component that was defined without a physical principle.

So let's keep the NdotL for diffuse but try to find a better model for speculars.

16 of 58

Microfacets

In the next slides we will be talking about how is the surface at the microscopic level, and using the concept of microfacet.

A microfacet is a portion of the surface so small that we cannot see it, but that affect how light behaves. Every microfacet could have a different normal and what we see is the average of the contribution of every microfacet.

So when you see an diagram like the one in the right, remember: the surface seen on the diagram is smaller than a single pixel of our screen.

less than one pixel

microfacet

17 of 58

Better specular

The Beckmann Distribution is a formula to compute the amount of specular more accurately.

It has been created based on measures of how materials behave, and trying to find the equation that fits better the physical world.

It tells us the amount of Specular in a pixel based on some properties of the surface.

But there are other models, here is a list of famous Specular models.

Here is a comparison between Beckmann Distribution and other similar models.

Beckmann distribution

18 of 58

Beckmann Distribution

The Beckmann distribution uses the coefficient m in the equation. The coefficient m models the roughness of the point, a factor from 0 to 1 that defines what is the average distribution of the normals of the microfacets.

In a perfectly flat surface the normals of all microfacets will be the same, so the distribution will be zero. In a totally irregular surface, every normal will point into a different direction, so the distribution will be one.

This function may seem very complex (as it has lots of operations) but because many of them depend on constants it is not so expensive to compute.

19 of 58

Roughness

Here we can see how if we increment the roughness at the microscopic level it will scatter more the light in all directions, producing a more blurry reflection.

20 of 58

Energy Conservation

But we still have the diffuse and specular independent, �let's try to find a model that connects them together, so we can ensure that the Energy reflected is the same as received.

That's what the Cook-Torrance model for specular reflections does. It uses the Beckmann distribution (term D in the formula) but adds other factors (F is the Schlick's approximation for fresnel).

If you check the formula you will see that the specular factor in divided by the NdotL, that makes sense if we want to make the energy constant. The more light is reflected as Diffuse, the less should be reflected as Specular:

Total = (Diffuse * NdotL) + (Specular / NdotL)

Cook-Torrance model

21 of 58

Fresnel

The problem with our previous reflection coefficient based on view angle (fresnel as NdotV) is that it is not very accurate.

Different materials have different fresnel functions so it is no longer valid to assume all materials have the same.

We need a better model that give us more control and that it is more physically accurate.

Measures taken to real materials to see its Reflective factor based in the incidence angle of the Eye and the Normal

22 of 58

Schlick's approximation

The Schlick's approximation tries to find an accurate representation of the reflection component, based on the view angle and the normal.

Given an angle theta (θ) it computes the reflection coefficient R(θ).

Pay attention that the cos(θ) is the same as VdotH.

n1 and n2 are the refraction factors of the two media (usually air and the surface), and R0 is the reflection coefficient of the material for a perpendicular ray.

23 of 58

Light absorption

The model also introduces the concept of light absorption. Which is the amount of light that never escapes the object due to its irregularity and self-shadowing.

This light is energy that will be transformed into heat over the surface based on its roughness.

The developers of the Filament engine propose a different approach to reduce the energy loss in some situations.

Geometric attenuation term (self-shadowing)

24 of 58

Cook-Torrance model

Beckmann Distribution of specular

Geometric attenuation term (self-shadowing)

Schlick's Fresnel approximation

Normalization term

25 of 58

Metalness

When we talked about microfacets we assumed that their distribution was constant, but in reality materials could have a percentage of the microfacets aligned with the global normal.

There could be flat areas mixed with rough areas (at the microscopic level). So the light we will see would be a sum of two groups, the specular group (light that hit the flat areas) plus the diffuse group (light that hit the rough areas).

And because that's a probability, we can model this factor as a simple coefficient from 0 to 1, that tell us what is the probability that a microfacet is aligned.

We call this coefficient metalness.

26 of 58

Metalness specular

Here we can see how the amount of specular reflection varies depending on the metal factor.

27 of 58

Metalness + Roughness

Using this two values we can approximate how light behaves in most common materials (but not all).

And because it is just two factors from 0 to 1 it is very easy to store them inside a texture (using two color channels) so objects can have different metalness and roughness along its surface.

If you think about it, this two coefficients where somehow already present in Phong as specular factor and glossiness, but because they were not related and no properly approximated, the results weren't realistic.

28 of 58

Color + Roughness + Metalness

29 of 58

Clear coat

We can go even further and model materials that have layers. Imagine a very rough surface that has a thin semi-transparent layer covering it (like a coating). Light will interact with both layers.

This will invalidate our current model as a point cannot be rough and not rough at the same time.

This case seems rare but it is quite common with metallic paint (cars).

Gladly is quite easy to model, just compute both cases and interpolate between them based on some fresnel coefficient.

Here is a tutorial about clear coat shading.�Here is the specification on GLTF

30 of 58

Anisotropy

Until now we used the angle between the V and N for our computations, but we didn't care about the orientation relative to the tangent.

In some materials (hair, and some metals) the orientation is important, due to the microsurface shape and orientation.

In these cases we need a way to define different values depending on the tangent of the point.

31 of 58

32 of 58

Material Definition Language

The people at NVidia have been pushing a standard definition for PBR materials called MDL to make easy to port materials from one platform to another.

For more info check their website.

33 of 58

Material Scanning

There are solutions to capture material properties from real objects, which could improve the accuracy of materials.

This solutions are based on taking photographs of the materials under controlled lighting conditions to recompute their color, normal, metalness and roughness.

In this presentation from Ready At Dawn (creators of the game The Order 1886) they explain the pipeline to capture real material properties they used for their game. Slides here.

There are already commercial softwares that helps you with this process, like Substance Alchemist.

34 of 58

PBR Materials database

Because PBR has become a standard model between all renderers, now it is easy to find repositories that contain lots of materials with their color + roughness + metalness textures.

Webs like texturehaven.com or c00textures.com offer licensing free resources for PBR materials.

Others like textures.com offer some of them for free and others commercially.

And for professionals there is Quixel Megascans database that contains scanned assets.

35 of 58

PBR + Environments

Until know all light in our equation came from infinite small light points, which is not very realistic.

We can improve our result if besides lights, we also add light from the skybox.

This is called Image-Based Lighting (IBL). The idea is that we also sample from the skybox based on the reflected specular vector or the normal.

But for this to work we need to use the mipmaps to approximate roughness levels.

36 of 58

Subsurface Scattering

And what about transmissive materials?

Those materials where the light can travel inside (like skin, wax, eyes, or dense liquids).

This is important mainly for characters as our eyes are very used to the subtleties of how the skin reflects light.

A wrong model could give us the impression of fake skin.

No SSS (left) and with SSS (Right)

37 of 58

Translucency

In the same line as subsurface scattering, we also must consider translucency (when an object is illuminated from behind and light travels through it).

This is important not only for humans but also for fruits, wax and liquids.

One approximation is to compute the depth of every point as seen from the light, using a two channel shadowmap.

38 of 58

Other rare properties

And we could keep extending our formula to support more strange material properties, like iridescence (materials that change the reflection color based on the angle of view), velvet (diffuse color change based on view angle), metals that colorize its specular, etc.

But that would depend on the domain of our application, and the more properties we take into account, the more complex the shader will be and more time it will take to compute.

39 of 58

Blender and PBR

Blender has a great PBR implementation, for both renderers (Eevee in real-time and Cycles using ray-tracing).

You can even choose which formula you want to use for specular and diffuse!

I recommend you to play with it a little to gain some intuition of how light behaves and what must you see.

It even has some extra parameters that we didn't discuss.

40 of 58

Implementing PBR

41 of 58

Implementing PBR

Now let's add an implementation of PBR to our shader.

Remember that to implement PBR you just need to improve your specular function (and the way we combine diffuse and specular).

Also you will need to pass more parameters to the shader (like the metalness, roughness, …)

42 of 58

BRDF for Specular

This is the function in charge of computing the amount of specular given the pixel roughness, the fresnel coefficient, and the dot product of some of the vectors.

It returns the amount of specular light.

Also the norm factor uses a 4 instead of Pi, I guess being a constant this is a correction over the formula.

We pass the dot products precomputed, that makes it easier.

//this is the cook torrance specular reflection model

vec3 specularBRDF( float roughness, vec3 f0,

float NoH, float NoV, float NoL, float LoH )

{

float a = roughness * roughness;

// Normal Distribution Function

float D = D_GGX( NoH, a );

// Fresnel Function

vec3 F = F_Schlick( LoH, f0 );

// Visibility Function (shadowing/masking)

float G = G_Smith( NoV, NoL, roughness );

// Norm factor

vec3 spec = D * G * F;

spec /= (4.0 * NoL * NoV + 1e-6);

return spec;

}

43 of 58

Distribution GGX

For the Distribution function we will use the Walter GGX instead of the Beckmann Distribution.

// Normal Distribution Function using GGX Distribution

float D_GGX ( const in float NoH,

const in float linearRoughness )

{

float a2 = linearRoughness * linearRoughness;

float f = (NoH * NoH) * (a2 - 1.0) + 1.0;

return a2 / (PI * f * f);

}

44 of 58

Schlick Fresnel

This is the implementation of the Schlick Approximation for the fresnel.

The f0 could be a float if we assume all color components have the same fresnel, or could be a vec3 to define different fresnel factors per color component.

Depending on your requirements.

// Fresnel term with scalar optimization(f90=1)

float F_Schlick( const in float VoH,

const in float f0)

{

float f = pow(1.0 - VoH, 5.0);

return f0 + (1.0 - f0) * f;

}

// Fresnel term with colorized fresnel

vec3 F_Schlick( const in float VoH,

const in vec3 f0)

{

float f = pow(1.0 - VoH, 5.0);

return f0 + (vec3(1.0) - f0) * f;

}

45 of 58

Geometry Visibility

This factor represents the self-shadowing.

It is based in the Smith geometric shadowing function.

// Geometry Term: Geometry masking/shadowing due to microfacets

float GGX(float NdotV, float k){

return NdotV / (NdotV * (1.0 - k) + k);

}

float G_Smith( float NdotV, float NdotL, float roughness)

{

float k = pow(roughness + 1.0, 2.0) / 8.0;

return GGX(NdotL, k) * GGX(NdotV, k);

}

46 of 58

Burley Diffuse

The people at Disney propose a better Diffuse function than the Lambertian, called the Burley.

The problem with this method is that it has higher cost and the difference if very small from the Lambertian.

The team at Filament comment that it is not worth the cost for a real-time rendering but I leave it here in case.

#define RECIPROCAL_PI 0.3183098861837697

// Diffuse Reflections: Disney BRDF using retro-reflections using F term, this is much more complex!!

float Fd_Burley ( const in float NoV, const in float NoL,

const in float LoH,

const in float linearRoughness)

{

float f90 = 0.5 + 2.0 * linearRoughness * LoH * LoH;

float lightScatter = F_Schlick(NoL, 1.0, f90);

float viewScatter = F_Schlick(NoV, 1.0, f90);

return lightScatter * viewScatter * RECIPROCAL_PI;

}

47 of 58

Diffuse + Specular

And finally add together the diffuse component with the specular.

Here you can see that the metalness will modulate the diffuse and it is used to compute the fresnel so it will also affect the specular amount.

Remember that until now we assumed the light was white and had intensity 1.0, but now we can correct that by multiplying the reflected light by the amount of light received.

//we compute the reflection in base to the color and the metalness

vec3 f0 = mix( vec3(0.5), baseColor.xyz, metalness );

//metallic materials do not have diffuse

vec3 diffuseColor = (1.0 - metalness) * baseColor.xyz;

//compute the specular

vec3 Fr_d = specularBRDF( roughness, f0, NoH, NoV, NoL, LoH);

// Here we use the Burley, but you can replace it by the Lambert.

float linearRoughness = squared roughness

vec3 Fd_d = diffuseColor * Fd_Burley(NoV,NoL,LoH,linearRoughness);

//add diffuse and specular reflection

vec3 direct = Fd_d + Fr_d;

//compute how much light received the pixel

vec3 lightParams = u_light_color * u_light_intensity * att *

shadow_factor;

//modulate direct light by light received

light += direct * lightParams;

48 of 58

Results

We can apply this new illumination model to our previous renderer and we should see a slight improvement in the quality, specially in the metallic materials.

We are still far from a realistic renderer, but we are getting closer.

49 of 58

Testing

You can test your results by using an assets created for testing PBR materials.

Here you have a GLTF that contains a matrix with all combinations of metalness and roughness.

Take into account that we haven't coded reflections yet, so test it only for direct lights.

50 of 58

White furnace

Because it is easy to have an equation that doesnt conserve energy, it is useful to test your scene with a totally gray sphere in a totally gray environment.

This is called a white furnace.

In that case the object should be indistinguishable from the background, if not, that means that your equation is wrong.

Here is a better explanation.

51 of 58

PBR with IBL

Until know the implementation only takes into account lights in our scene.

If you want to use the environment texture and sample light from it then you will need to add some extra steps.

Here is a nice tutorial explaining it.

float lod = roughness * 5.0; //our HDRE format stores 5 mipmaps to pack blurred

vec3 prefilteredColor = textureCubeLod( u_skybox_cubemap, refVec, lod);

vec2 envBRDF = texture2D( BRDFIntegrationMap, vec2(NdotV, roughness)).xy;

vec3 indirectSpecular = prefilteredColor * (F * envBRDF.x + envBRDF.y)

52 of 58

Other material properties

53 of 58

Normalmap edge cases

We saw normalmaps previously, the problem with normalmaps is that they work as long as the details are small (like scratches, pores, wrinkles, etc).

If the normalmap has big changes over the surface, and the user gets close enough he will be able to see that the surface is flat, breaking the illusion.

It would be great if there was a solution that help us make surfaces less flat even when we look at them for close distance.

54 of 58

Displacement Texture

Some engines allow materials to include a displacement texture to give more detail to the mesh without the cost of heavier meshes in VRAM.

This texture contains the height of every pixel mapped in grayscale, like in a heightmap.

Now from the GPU we must use this information to make the surface look less flat.

There are two options, both have a high cost but give great results:

Tessellating shaders.
Parallax mapping.

55 of 58

Parallax Mapping

Parallax mapping is a technique that aims to give a sense of depth over a flat surface just by changing the UVs we use to fetch from the textures.

There are two ways: by offsetting the UV based on the height (fast by gives ugly artifacts in steep angles), or by doing raymarching over the heightmap to approximate the true UV based on the view vector.

Because raymarching means having to iterate many times per pixel it is not a fast solution, but it is better than adding more geometry.

The edge cases is when the mesh is seen from a steep angle (or the number of steps is not big enough), artifacts appear. Here is a great tutorial or this paper.

56 of 58

Tessellation Shader

Modern GPUs support another type of shader called the Tessellation Shader.

This shader allows to increase the number of triangles of a mesh from the GPU during rendering.

Because the computation is done per triangle, it increases the GPU work, and because the resulting mesh will have more triangles it will take more time to render.

But mixing this with the displacement texture we can render meshes that look more detailed. Here is a tutorial.