Maya to Unity Vrchat Avatar Creation
This tutorial was made using Autodesk Maya LT 2017 and Unity Version 5.3.4p1
Part 0: Summary
Preparing your scene and avatar for rigging
Rigging your mesh
Visemes, Jaw, And Other Blend Shapes
Choose a method for moving your avatar’s mouth. There are 3 methods to choose from. Jaw Flap, Jaw Open blend shape, or Visemes. (This tutorial will mainly focus on Visemes)
Method 1: Jaw Flap
This is the easiest method. In many cases, fully rigged avatars you wish to bring in will already have a skinned jaw bone. If this is the case, you are already done. If not, you will need to skin your avatar to a rig that includes a jaw bone. (This part of the tutorial is marked Optional)
Method 2: Jaw Blend Shape
Similar to the jaw flap, this method will only allow your jaw to open and close while you speak, only in this case, you will be using a blend shape/shape key. (This part of the tutorial is marked Optional)
Method 3: Visemes
This method harnesses the full power of Vrchat’s avatar creation pipeline. This method will give your avatar the ability to mimic each sound that comes from your mouth by blending between various shapes to mimic your actual voice… And it looks really cool.
Exporting Your Avatar From Maya
Getting your avatar working in Unity
Part 1: Preparing your scene and avatar for rigging
Open Maya or Maya LT. Go to Preferences, then Settings. Then change Working Units from Centimeters to Foot for Americans. Meters or Centimeters for those who prefer metric.
Import your 3D Model into Maya. For this tutorial I will be using the vrchat sample avatar model. Ideally, you will want spheres for eyes and you will want to ensure the balls of your avatar’s feet are on the grid.
Go to Create > Measure Tools > Distance Tool. In an orthographic front view of your avatar, click on 2 points to create a measure tool to show the distance between points.
Make sure your avatar is scaled to an appropriate size. 6 feet or 183cm is a realistic height for this guy, so I’ll be keeping him as is.
Part 2: Rigging your mesh
Download the Vrchat Sample Avatar from https://drive.google.com/file/d/0B_1UcaOQwKrnQWhnSTBzZDdYMGc/view?usp=sharing
and import it into your scene. Delete the sample avatar mesh if you do not intend to use it. The Armature bone should remain at 0,0,0. Scale the bones until the rig matches the size of your mesh. Move the bones to appropriately fit the avatar’s body. Eye bones should be located in the center of the eye spheres. Then with all bones selected, go to Modify > Freeze Transformations
Make sure the Armature joint stays at 0,0,0
Make sure your tongue, teeth and face are all combined into one mesh. You will need them to be combined for Visemes to work later. Name this mesh “Body”
Also, don’t forget to delete all history on your mesh and freeze it’s transforms if you haven’t already by selecting all of your geometry and going to Edit > Delete All By Type > History and Modify > Freeze Transformations. This should be your final step after physically editing your mesh before skinning.
Next up is Skinning.
NOTE: Skinning a mesh can be a long process and I will not cover the finer details of skinning a mesh to bones in this tutorial. For details on how to Properly skin a mesh to bones, seek another tutorial.
Firstly, select your geometry and all of the bones you want to influence your avatar. Go to Skin > Bind Skin > Options
Make sure to bind to “Selected Joints” and make the bind method “Closest In Hierarchy.” Weight distribution should be “Neighbors”.
After you fix the skinning to your liking, we will go on to Viseme and blend shape creation.
The easiest way to edit skin weights is to go to the rigging tab and go to Skin > Paint Skin Weights > Options. You will see your influences as a black/white gradient.
-Always have Paint Operation set to “Add.” and edit your Opacity to manually smooth between influences.
-Save your file before smoothing if you decide to attempt it. Smoothing skin weights doesn’t always have the result you want and undoing isn’t always an option.
Part 3: Visemes And Other Blend Shapes
Next up, the various Blend shapes we will be using for our avatar. This will make him “talk” in Vrchat.
Firstly, duplicate your face mesh 19 times and name the new meshes as follows:
The 19 shapes will have to be manually created by moving the vertices on each mesh to match the appropriate sound your mouth would make. The following is an example of each shape as they were made for the sample avatar:
The v_aa shape. An open mouth mimicking the sound you would make if you were to open your mouth and say “aa”
The v_ch shape: Opened lips and clenched teeth. A good starting point for v_rr as well
The v_dd shape: I made this as a variation of v_nn. Keep the tongue raised but have the jaw more closed than v_nn
The v_ee shape: The most iconic sound. Very wide mouth. (this is the shape people will likely try to make when they say “cheese” for the camera. It is important to exaggerate this shape a bit ESPECIALLY on cartoony characters. You can duplicate the v_ee shape to act as a starting point for vrc.v_ss
The v_ff shape: Raise the upper lip a bit and drag the lower lip under the front teeth.
v_ih: Open the mouth and jaw slightly. (Just enough to fit the tongue between your teeth) This shape can act as a starting point for many other shapes.
The v_kk shape:. Start with vrc.v_ih and raise the sides of the top lip. You can also move the tongue up to the roof of the mouth in the back if you are feeling fancy.
The v_nn shape: Start with a duplicate of v_ih and raise the tongue to the roof of your mouth.
The v_oh shape: a modified version of v_aa. Just make the lips narrower and more O shaped.
The v_ou shape: a much smaller exaggerated version of the v_oh shape. This shape makes the difference between you saying “oh” and “oooooo”
v_pp: tilt the upper and bottom lip into the teeth a bit and make sure the mouth is closed. This shape will be used for sounds like M, B and P.
v_rr: like v_ch, except lower the teeth/jaw and raise the upper lip a bit more for an exaggerated RR.
v_sil: the Silent shape. Most of the time your base mesh should start with a closed mouth in a rested pose. If your avatar comes with a resting face by default, then this will be the easiest shape to make (exact duplicate of the Body mesh)
The v_ss shape: a variation of v_ee with closed (or nearly closed) teeth and lips that are barely open.
The v_th shape I made from v_ih, but just stuck the tongue out.
Adding Eyes And Eyelids:
Here’s the 4 eye shapes: A left and right blink and a slightly raised left and right lower lid. So long as the eye shapes are present and named correctly, Name them blink_left, blink_right, lowerlid_left and lowerlid_right. they should work automatically assuming everything else is set up properly. Setting these up along with spherical eyes, causes eyes to track close objects and players in VrChat which is cool.
When you have finished modifying your blend shapes, select each of the 19 blend shape meshes in alphabetical order (Starting with blink_left) and then select your Body mesh. In the Rigging tab go to Deform > Blend Shape > Optionbox and name your blend shape node “vrc” Click apply.
Congrats! You now have a fully rigged avatar ready for Unity.
Part 4: Exporting Your Avatar From Maya
Select your bones and geometry and go to File > Export Selection > Option.
Choose FBX as the file type and then go to Edit Preset.
In the Advanced Options tab under FBX File Format, choose Binary under the dropdown menu for Type, and choose FBX 2013 for the Version.
Continue to export your FBX.
Part 5: Getting your avatar working in Unity
Download the VRSDK unity package from vrchat.net and import it to Unity by going to Assets > Import Package > Custom Package and importing the package you downloaded from vrchat.net
Import your FBX with it’s Textures to your Assets Folder.
Change the FBX’s Animation Type to Humanoid and click Configure
Ensure your hand bones look like they are in a flat “T-pose” with palms facing the ground. If they aren’t already in this position, rotate the wrists so the fingers bend directly downward and under Pose, choose Enforce T-Pose
Drag your avatar into your Heirarchy.
In the inspector, go to Add Component > VRC_Avatar Descriptor.
Adjust the Y and Z coordinates of your View Position until the little grey ball is between your eyes.
(Optional)If you chose the Jaw Flap method for your mouth, select Jaw Flap Bone from the drop down menu under Lip Sync and drag your jaw bone into the “Jaw Bone slot. This method may be easier for you than the Viseme system, but requires a rigged model with a Jaw bone. Using this method, your jaw will flap when you speak. It’s less detailed than the Viseme system, but it is a valid option for simpler avatars. Your eyes will not work if you choose this method.
(Optional) If you chose the Jaw Flap Blend Shape method, choose the “Jaw Flap Blend Shape” option from the drop down and drag your Body mesh into the Face Mesh slot. Using this method, you only need to create a single Viseme. vrc.v_aa would be a good shape for this. This option is similar to the jaw flap option, but uses an “open mouth” blend shape instead of a jaw bone. Your eyes will not work with this method.
For the Visemes Method, the Set Lip Sync to Viseme Blend Shape and drag your Body mesh from the Hierarchy to the empty Face Mesh slot in your Inspector.
15 new drop down menus should now show up. You will need to assign the proper blend shapes to the proper slots.
NOTE: Eye shapes do not need to be assigned. They should work on their own if set up correctly.
Part 6: Uploading Your Avatar To Vrchat.
Go to Vrchat SDK > Show Build Control Panel and select Build and Publish on your avatar. You will need to log into your vrchat account using your Vrchat Account Credentials.
Save your scene, name your avatar, give it a description and then go over to the scene tab to adjust your camera. Back in the Game Tab, agree to the above info and click Upload.
Congrats! You have just uploaded a fully functional avatar with Blend Shapes and tracking eyes.
Common Issues and Solutions:
-If your mesh fails to import parts of geometry into Unity, Open Maya, re-import your FBX (not your scene file) and export the FBX again.
-If you find your avatars fingers are crooked in Vrchat, you will need to go into Configure in your Unity inspector on your avatar and rotate the hands to be in a perfect T-pose with palms flat and facing the ground. Then click “Enforce T-pose
- If you modify the rig or change bone names the eyes may not work. Also re-ordering blend shapes in an order that isn’t listed above the eyes won’t work. If you decide to add extra blend shapes for whatever reason, make sure they are listed AFTER the original 19.
-If your eyes come out backwards or at an angle that isn’t what you started with, one possible solution is to reset your pose and then enforce T-pose when you go to configure your humanoid skeleton