���Chapter XIV�Normal Mapping

All class materials including this PowerPoint file are available at

https://github.com/medialab-ku/openGLESbook

Introduction to Computer Graphics with OpenGL ES (J. Han)

Bumpy Surfaces

14-2

• Rendering a high-resolution high-frequency polygon mesh

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

• Unfortunately, it is not cheap to process the high-resolution mesh.

n∙l

Introduction to Computer Graphics with OpenGL ES (J. Han)

Bumpy Surfaces (cont’d)

14-3

• Rendering a low-resolution mesh

​

​

​

​

​

​

​

​

​

​

​

• It is cheap to process such a low-resolution mesh. However, the quad does not properly expose the bumpy features. The quad has a uniform normal across its surface, and there is smooth change in shading across the surface. If the viewpoint gets closer to the quad, the lack of geometric detail will be more clearly noticeable.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Bumpy Surfaces (cont’d)

14-4

• A clever way out of this dilemma
• Pre-compute the normals of the high-frequency surface and store them into a special texture named normal map.
• At run time, use a lower-resolution mesh, which we call base surface, and fetch the normals from the normal map for lighting.

​

normal-mapped

base surface

normal map

+

=

Introduction to Computer Graphics with OpenGL ES (J. Han)

Height Map

14-5

• Normal maps can be generated from height maps.
• A popular method to represent a high-frequency surface is to use a height field. It is a continuous function h(x,y) that returns a height or z value given (x,y) coordinates.
• The height field is sampled with a 2D array of regularly spaced (x,y) coordinates, and the height values are stored in a texture named height map.
• The height map can be drawn in gray scales. If the height is in the range of [0,255], the lowest height 0 is colored in black, and the highest 255 is colored in white.

height map

(gray-scale image)

Introduction to Computer Graphics with OpenGL ES (J. Han)

Height Map from Image Texture

14-6

• The height map can be generated from an image texture. Using 2D graphics packages, the RGB colors of the image texture are converted into gray scale, e.g., (R+G+B)/3.
• Then, the gray-scale image is often edited by hand to produce the height map.
• The height map has the same resolution as the image texture.

​

​

​

​

​

​

​

​

​

​

• The next step “from height map to normal map” is done automatically.

image texture height map normal map

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Map from Height Map

14-7

• Take (2,0,δh_x) and (0,2,δh_y) as the tangent vectors at (x,y,h(x,y)) and compute their cross product. It’s normal, which reflects the slopes of the nearby area.

​

​

​

​

​

​

​

• In order to store the normal in an OpenGL texture, where each RGB component is in the range of [0, 1], we need range conversion.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Mapping

14-8

• During rendering, the polygon mesh is rasterized and texture coordinates (s,t) are used to access the normal map.
• The normal at (s,t) is obtained by filtering the normal map.
• Consider the diffuse reflection term, max(n•l,0)s_d⊗m_d.
• As usual, m_d is fetched from the image texture.
• Now, the normal n is fetched from the normal map.

​

normal map

+

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Mapping (cont’d)

14-9

• Vertex shader modified from Chapter 9

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Mapping (cont’d)

14-10

• Fragment shader

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Mapping (cont’d)

14-11

• Fragment shader

(0,0)

(1,1)

(0,0)

(1,1)

Introduction to Computer Graphics with OpenGL ES (J. Han)

Normal Mapping (cont’d)

14-12

• Normal mapping gives the illusion of high-frequency surface detail in that it is achieved without adding or processing more geometry.
• Unlike the high-frequency mesh at p.2, the normal-mapped quad shown here exposes its linear edges. It is unavoidable since normal mapping does not alter the geometry of the base surface at all but simply perturbs its normals during lighting.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent Space

14-13

• Recall that texturing is described as wrapping a texture onto an object surface. We should be able to paste it to various surfaces.
• Planar surfaces with arbitrary orientations.
• Curved surfaces.
• For this, we need an additional process, which is not performed for image texturing.

​

​

​

​

​

​

​

​

​

​

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent Space (cont’d)

14-14

• For a surface point, consider a tangent space that is defined by the point and three orthonormal basis vectors:
• T (for tangent)
• B (for bitangent)
• N (for normal)

​

​

​

• Consider p. Without normal mapping, N_p would be used for lighting. In normal mapping, however, the normal map is filtered using p’s texture coordinates, (s_p,t_p), and the normal, n(s_p,t_p), replaces N_p, which is (0,0,1) in the tangent space of p. Similarly, n(s_q,t_q) replaces N_q, which is (0,0,1) in the tangent space of q. These imply that, as a ‘perturbed’ instance of (0,0,1), n(s_p,t_p) is considered to be defined in the tangent space of p, and n(s_q,t_q) is in the tangent space of q.
• Whatever surface point is normal-mapped, the normal fetched from the normal map is considered to be defined in the tangent space of that point.

• The tangent spaces vary across the object surface.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent Space (cont’d)

14-15

• Consider dot(normal, light)in p.11.
• In general, it does not work because normal is a tangent-space vector but light is a world-space vector. Two vectors defined in different spaces may not be combined through dot product.
• In the example, dot(normal, light)luckily worked because the basis of the world space happened to be identical to that of the tangent space for every point of the base surface, the quad.

​

​

​

​

​

​

​

​

• If the quad is tilted, for example, the tangent-space bases are no longer identical to the world-space.

​

​

​

​

​

​

​

​

​

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent Space (cont’d)

14-16

• Above all, the basis of the tangent space {T,B,N} is defined for each vertex of the polygon mesh.
• Vertex normal N – defined per vertex at the modeling stage.
• Tangent T – needs to be computed
• Bitangent B – needs to be computed

​

​

​

​

​

​

​

​

• There are many utility functions that compute T and B on each vertex of a polygon mesh. If interested, read Section 14.4.3.
• Typically, the per-vertex TBN-basis is pre-computed, is stored in the vertex array and is passed to the vertex shader.

​

​

​

​

​

​

​

​

​

​

​

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping

14-17

• Return to the diffuse term of the Phong lighting model.

​

• Fetched from the normal map, n is defined in the tangent space.
• In contrast, l is defined in the world space.
• To resolve this inconsistency,
• n has to be transformed into the world space, or
• l has to be transformed into the tangent space.
• Let’s take the second option.
• As T, B and N are object space vectors, they should be transformed into the world space.
• Then, a matrix is constructed with the “world-space T, B and N.” It rotates the world-space light vector into the per-vertex tangent space.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping (cont’d)

14-18

• B can be computed on the fly.
• In GL, a matrix constructor such as mat3 fills the matrix column by column. So, we need transpose.
• Both v_lightTS and v_viewTS are defined in the tangent space.
• Observe that normal is used just for defining the transform to the tangent space and is not output.

​

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping (cont’d)

14-19

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping (cont’d)

14-20

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps

14-21

• Normal maps for planar or low-curvature surfaces can be created using 2D graphics packages. However, the 2D packages are not appropriate for creating a normal map for a complex surface.
• Digital sculpting tools, such as ZBrush developed by Pixologic, allow digital sculptors to make global or local changes to a polygon mesh.

​

​

​

​

​

​

​

​

​

• Then, the original low-resolution surface in (a) is taken as the base surface. Then, using it and the high-frequency mesh in (e), we can generate a normal map.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps (cont’d)

14-22

• In the example of planar surface normal mapping, the normal map can be considered as being obtained by firing a vertical ray from the base surface toward the high-frequency mesh and extracting the normals of the mesh.

​

​

​

​

​

​

​

• Similarly, let’s superimpose the high-frequency mesh onto the base surface and then fire a vertical ray from the base surface toward the high-frequency mesh to extract the normal from it.

base surface

high-frequency

mesh

high-frequency

mesh

base surface

normal map

high-frequency

mesh

base surface

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps (cont’d)

14-23

• For image texturing, a polygon mesh is unfolded so as to give the normalized texture coordinates, the parameter space is scaled up by r_x and r_y, and the artist to draw an image on the scaled-up space, i.e., the scaled-up space is filled with colored pixels.

​

​

​

​

​

​

​

​

​

• For normal mapping, let’s also unfold the base surface, scale it up by r_x and r_y, and fill the scaled-up space with normals.
• Then, both the normal map and image texture will be accessed via the same normalized texture coordinates, (s, t).

​

(0,0)

(r_x,r_y)

parameter space

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps (cont’d)

14-24

• The base surface is unfolded and parameterized such that each vertex is assigned normalized texture coordinates, (s, t). For the image texture and normal map of resolution r_x × r_y, the coordinates of each vertex are scaled to become (r_xs, r_yt).
• For the image texture, the graphic artists fill the texels of the unfolded base surface with RGB colors. In contrast, normals will be automatically computed and assigned to the texels to create the normal map. For this, every triangle of the unfolded base surface is rasterized.
• Consider a texel, T. Being a 3D polygon mesh, each vertex of the base surface is associated with a normal. Using the barycentric coordinates of T, the vertex normals, n₀, n₁ and n₂, are interpolated at T. Let n_T denote the interpolated normal.

unfolded scaled-up base surface

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps (cont’d)

14-25

• In the figure, <v₀, v₁, v₂> = bold line segment, and T = p. The 3D coordinates of p are computed using the barycentric coordinates of T with respect to <v₀, v₁, v₂>. Then, p and n_T define a ray.
• The intersection between the ray and a triangle of the high-frequency mesh is computed. Each vertex of the high-frequency mesh is associated with a normal. Using the barycentric coordinates of the intersection point with respect to the triangle, the vertex normals of the triangle are interpolated and stored at T in the normal map. During rendering, the normal map is filtered using (s, t).

high-frequency

mesh

• Note that the normal map’s resolution doesn’t have to be identical to the image texture’s.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Authoring Normal Maps (cont’d)

14-26

• The normals we computed are object-space vectors. The texture that stores such normals is called an object-space normal map.
• Consider the base surface’s triangle where p lies. For each vertex of the triangle, a tangent space can be computed, and the per-vertex TBN-bases are interpolated at p using its barycentric coordinates.
• Then, the TBN vectors at p are used to define the rotation matrix, which converts the object-space normal, n, into the tangent space.

object-space normal map

tangent-space normal map

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping - Examples

14-27

Introduction to Computer Graphics with OpenGL ES (J. Han)

Tangent-space Normal Mapping - Examples

14-28

Introduction to Computer Graphics with OpenGL ES (J. Han)

Supplement – 3D Scanning

14-29

• This lecture presented just a ‘typical’ computer-graphics production. There are many more modeling and animation methods. Let’s see a modeling example.
• Real-world objects can be captured via 3D scanning, which samples their surfaces with a non-contact technology and creates the so-called point clouds.
• The points are connected to make a polygon mesh. See (a), which has about 100K polygons. Such a high-resolution mesh is not good for real-time graphics.
• The resolution can be reduced. See (b), which is produced by ZBrush, and (c), where the eyes are created with 3ds Max.
• Finally, (d) shows the textured model.

Introduction to Computer Graphics with OpenGL ES (J. Han)

Supplement – Height Map for Terrain Rendering

14-30

Introduction to Computer Graphics with OpenGL ES (J. Han)