Skip to main content

Face

Regular • File: FaceTracking.csv • Writer: FaceTrackingDataWriter.cs

Media

At a glance​

Face tracking captures the facial skeletal / blend-shape-driven pose — one set of position + rotation columns per tracked face bone, produced by FaceUtility from the active SilicoSkeletonFace. On Meta Quest Pro / supported HMDs, this is driven by the ~63 Meta face-tracking blendshapes; on other devices, fewer bones may be tracked.

Use Face for:

  • Expression time series (jaw open over time, brow raise, smile intensity proxies).
  • Blendshape-driven analyses (compare against Meta's blendshape conventions).

For discrete expression detections above a threshold (e.g. "smile detected"), use the irregular Expressions stream instead — it emits one row per detection event rather than per frame.

When it writes​

One row per LateUpdate, provided:

  • ExperienceUtilities.CaptureFaceData is true, AND
  • a SilicoSkeletonFace is assigned.

File & location​

  • Default: FaceTracking.csv in the run directory.
  • Writer: FaceTrackingDataWriter.cs.
  • Columns come from FaceUtility.ColumnNamesSkeleton(skeleton) in Runtime/ExperienceObjects/Skeletons/Face/FaceUtility.cs.

Configuration​

Gated by ExperienceUtilities.CaptureFaceData and the face-skeleton assignment — no dedicated ScriptableObject. The active SilicoSkeletonFace determines which bones are in the output.

Columns​

Shared prefix​

See Column conventions — The shared prefix.

Per-bone blocks​

One block per face bone in the active skeleton, in skeleton-defined order. A typical block:

ColumnTypeUnitsDescription
<Bone>_PosX, <Bone>_PosY, <Bone>_PosZfloatmetresWorld-space position of the bone.
<Bone>_RotX, <Bone>_RotY, <Bone>_RotZ, <Bone>_RotWfloatunit quaternionWorld-space rotation of the bone.

Typical face-bone names map loosely to Meta blendshapes: BrowL, BrowR, EyeL, EyeR, JawOpen, MouthSmile, MouthFrown, CheekPuff, NoseSneerL, NoseSneerR, etc. Exact set depends on the skeleton.

For the authoritative blendshape list and semantics, see Meta's face-tracking reference.

Sample rows​

...shared...,BrowL_PosX,BrowL_PosY,BrowL_PosZ,BrowL_RotX,BrowL_RotY,BrowL_RotZ,BrowL_RotW,JawOpen_PosX,...
...,0.02,1.72,-2.5,0.0,0.0,0.0,1.0,0.0,...
...,0.02,1.72,-2.5,0.0,0.01,0.0,0.9999,0.005,...

Join with other streams​

Standard FrameNumber left-join to ExperienceState:

import pandas as pd
exp = pd.read_csv("ExperienceState.csv")
face = pd.read_csv("FaceTracking.csv")
joined = exp.merge(face, on="FrameNumber", how="left", suffixes=("_exp", "_face"))

Gotchas​

  • Device support varies. Face tracking requires hardware support (Meta Quest Pro and compatible devices). On devices without face tracking, the skeleton may not produce meaningful bone data — columns will exist but values will be static or zero.
  • Blendshape ↔ bone mapping is not 1:1. LABO stores bone pose, not raw blendshape weights. Some analyses care about the blendshape weight directly (0 to 1); you'll infer that from bone displacement. Link back to Meta's reference to interpret.
  • Confidence is not per-column. Unlike some face-tracking APIs, Face CSV does not include a per-bone confidence score. Treat low-signal frames (head off-camera, eyes closed) as potentially invalid based on context from other streams.
  • File only exists if the gate passes at run-start. Same as Body — no skeleton, no file.
  • Quaternion sign flips. See Participant gotchas.

Analysis recipes​

For discrete expression detections above threshold, use the irregular Expressions stream — it's a much better match for "did the participant smile during stimulus X?" questions.