Lina. This character maps face tracking expressions to Facial Acting Coding System (FACS)-compatible blendshapes for a humanoid character. The FaceDriver component animates a high-quality face representation, proportionate to human faces.

Blendshape Mapping Example. This character maps face tracking expressions to ARKit blendshapes for more cartoonish characters. The FaceDriver component animates a high-quality face representation, proportionate to human faces.

After reviewing this sample scene, the developer should:

Understand how to set up a complex character with face tracking and correctives that work with visual or audio-based face tracking.

Understand how to set up a character model with ARKit blendshapes to support visual or audio-based face tracking.

Face and eye tracking

You can find the Movement Face sample under the Samples/FaceTrackingSamples/Scenes folder in the Unity Movement GitHub repo⁠. Please reference the import samples section of the Getting Started guide.

This scene has a first-person character standing in front of a mirrored version, so that the user can see their facial emotions in real time. Note that the scene allows switching the currently active character, whether that is Lina or the ARKit counterpart, using the CharacterSwapMenu on the left.

The first person character tracks a person’s face, body, and eye movements, while the third person character simply copies those values for mirroring purposes. The first person character’s head is also disabled, so there is no need for normal recalculation of its head mesh. Only the head and hands are visible on all characters; no body meshes exist for them.

Use the SceneSelectMenu to switch between scenes on the left. Since this scene uses prefabs for the first and third-person characters, you can navigate to the original prefabs by clicking on the “Select” button after selecting the prefab that exists in the scene. The following images show the Lina and Blendshape Mapping Example characters used in this scene.

Lina Character

Blendshape Mapping Example Character

Lina character details

The scripts section describes what each component does. The following lists the basic information about the components of the Lina character:

OVRBody: Necessary for body tracking.

OVRCustomSkeleton: Necessary for body tracking and is used to map the BoneId enum to their corresponding bone transforms. This can be done manually or via the “Auto Map Bones” button.

FaceDriver: Drives blend shapes on the character using a retargeter component, which sources the scene’s OVRExpressionsProvider.

RecalculateNormals: Recalculates normals when the mesh animates.

Blendshape mapping example character details

The scripts section describes what each component does. The following list provides basic information about each component that lives on the Blendshape Mapping Example character, which are as follows:

OVRBody: Necessary for body tracking.

OVRCustomSkeleton: Necessary for body tracking and is used to map the BoneId enum to their corresponding bone transforms. This can be done manually or via the “Auto Map Bones” button.

RecalculateNormals: Recalculates normals when the mesh animates.

FaceDriver: Drives blend shapes on the character using a retargeter component, which sources the scene’s OVRExpressionsProvider.

Did you find this page helpful?