1 of 1

Synthesizing a 3D Human Face Database

Amy Liu

Vision and Graphics Lab, USC Institute for Creative Technologies

INTRODUCTION

Past face-related computer vision research has shown that synthetic data can greatly improve a machine learning model’s ability to interpret and understand real face data.

However, the domain gap between real and synthetic faces has remained the largest challenge for researchers to overcome.

My challenge is to create individuals who model the diversity and complexity of our real-world population, effectively bridging the gap between digital and ‘in-the-wild’ humans.

3. METHODS

2. PURPOSE

5. FUTURE SIGNIFICANCE

6. ACKNOWLEDGMENTS

4. RESULTS

Yunxuan Cai: Internship Mentor

Rafael Duffie: 3d Modeling Artist

Yajie Zhao: VGL Lab Director

Kathleen Haase: VGL Project Manager

Using analytical and formulative research methods, I demonstrate the ability to automatically generate a large training dataset of hyper-realistic human faces.

The film and gaming industry has proven that rendering high-fidelity digital humans is possible, yet each individual requires significant time and manual effort.

Given a volumetrically-generated 3d base mesh, I present an algorithm that procedurally constructs photo-realistic hairstyles and applies deformation techniques to a variety of headwear and garments based on the mesh’s specific dimensions. The final output is fully renderable and achieves level of realism comparable to hours of human manipulation.

Input:

Deforming rigid bodies with rigging and inverse kinematics (IK)

To deform rigid asset (e.g. glasses), a joint-based system that relied on specific anchor points on the face mesh was developed. Then, the algorithm relied on IK to calculate and pose the object without unwanted collision with the face geometry.

Apply photo-realistic shaders and personalized textures

Create blendshape deformer with target mesh

Manipulate hair attributes using Autodesk Maya XGen Python API

XGen modifiers are used in Maya to change the appearance and behavior of Spline primitives.

Careful interaction with the codebase of these modifiers allowed for automated modification of hair characteristics such as color, length, clumping, and curl.

Fit headwear with Proximity Wrap Deformer

Use Delta Mush Deformer to maintain structure of face mask

In summary, my contributions include an algorithm that automatically generates hyper-realistic facial components and builds the foundation for synthetic data to achieve levels of realism and diversity never before anticipated.

Internally, this work will be utilized in the future by the USC Vision and Graphics Lab for all potential human face-related research.

Some applications include:

Multimodal Avatar Creation and Reenactment:

Combining CG, computer vision, and natural language processing and building off of the render-ready 3d models generated from my pipeline , we predict the ability to create virtual avatars capable of lifelike speaking and expressional characteristics

Facial Parsing and Landmark Localization:

With the right synthetic data, you can guarantee perfect facial analysis without the regular annotation noise.

We hope to train computers to recognize differences between real and virtual data, allowing us significantly more freedom with tasks such as assigning class labels to each individual pixel in an image or calculating the exact position of specific points of interest.

RAW TARGET MESH

ORIGINAL TEMPLATE MESH WITH DESIRED ASSETS

PROCESSED TARGET MESH WITH DESIRED ASSETS