Skip to main content

Join keys

Every analysis that combines two LABO streams needs a join key — a column that means the same thing in both files so you can line up rows. This page is the short answer to "which column do I join on?" plus worked examples in pandas and R.

The short answer​

Joining two...Use this key
Regular streams from the same sessionFrameNumber (exact match)
Irregular stream to a regular streamFrameNumber (exact match)
Two irregular streamsFrameNumber first; fall back to MonotonicExecutionTime within a tolerance if they fire in different frames but refer to the same moment
LABO data to an external device (EEG, biopac, standalone tracker)MonotonicExecutionTime (drift-free, seconds from session start)

FrameNumber is the preferred key within a session. MonotonicExecutionTime is the preferred key across sessions or across devices. See Timing for why.

What to avoid​

❌ Don't join onWhy
ObservationTimeUnity-side clock; may drift between writers in the same frame.
FrameTimeSame value for every row in a frame — lossy within a frame.
FixedIntervalNumberOnly meaningful for streams writing in FixedUpdate. Mismatched across streams on different sampling loops.
Wall-clock timestampsSubject to NTP drift and system clock adjustments.

Pandas — join ExperienceState to EyeTracking​

import pandas as pd

exp = pd.read_csv("ExperienceState.csv")
eye = pd.read_csv("EyeTracking.csv")

# Exact inner join on FrameNumber — both streams write on LateUpdate,
# so every eye-tracking row has a matching ExperienceState row.
joined = exp.merge(eye, on="FrameNumber", how="inner", suffixes=("_exp", "_eye"))

# Now any ExperienceState column (EpochName, EventNumber) is attached
# to every eye-tracking sample. Group gaze by epoch:
joined.groupby("EpochName")["Eye_Center_DirZ"].mean()

Pandas — join irregular Events to ExperienceState context​

import pandas as pd

exp = pd.read_csv("ExperienceState.csv")
events = pd.read_csv("EventPoints.csv")

# Each event row already carries FrameNumber from Stamp() at trigger time.
# Attach the full per-frame state (fps, deltaTime, etc.) via left join.
enriched = events.merge(exp, on="FrameNumber", how="left", suffixes=("_event", "_frame"))

# Now every event has the frame's fps and deltaTime next to it —
# useful for flagging events that fired on slow frames.
slow_events = enriched[enriched["deltaTime"] > 0.025]

Pandas — align to an external device via MonotonicExecutionTime​

import pandas as pd

labo = pd.read_csv("ExperienceState.csv")
eeg = pd.read_csv("external_eeg.csv")  # has its own 'timestamp_s' column, seconds from session start

# Sort both by the join column; merge_asof does a nearest-match join
# within a tolerance (5 ms here).
labo = labo.sort_values("MonotonicExecutionTime")
eeg = eeg.sort_values("timestamp_s")

aligned = pd.merge_asof(
    labo,
    eeg,
    left_on="MonotonicExecutionTime",
    right_on="timestamp_s",
    direction="nearest",
    tolerance=0.005,  # 5 ms
)

merge_asof is the right tool whenever the two streams are sampled at different rates or slightly offset — it does a windowed nearest-match rather than requiring exact equality.

R — same joins​

library(dplyr)

exp <- read.csv("ExperienceState.csv")
eye <- read.csv("EyeTracking.csv")

joined <- exp %>%
  inner_join(eye, by = "FrameNumber", suffix = c("_exp", "_eye"))

joined %>%
  group_by(EpochName) %>%
  summarise(mean_gaze_z = mean(Eye_Center_DirZ, na.rm = TRUE))

For merge_asof-style nearest-match, see data.table::nafill / roll parameter in data.table::merge or the fuzzyjoin package.

Gotchas​

  • Regular streams in "Both" sampling mode write two files (e.g. ExperienceState_Update.csv and ExperienceState_FixedUpdate.csv). Pick the right one: join LateUpdate-sampled streams to _Update, FixedUpdate-sampled streams to _FixedUpdate.
  • Missing frames are normal. A stream that's gated (e.g. body tracking without a skeleton) won't have rows for every frame. Use left joins when FrameNumber coverage is not guaranteed.
  • Hand tracking is two files, not one. LeftHandTracking.csv and RightHandTracking.csv each have their own FrameNumber column. To get both hands in one table, join them to ExperienceState independently, then join the two results.
  • Irregular rows can fire between regular samples. An event-point at FrameNumber = 745 aligns to the ExperienceState row with FrameNumber = 745, but the times within that frame differ by a millisecond or two. This is real, not noise; it reflects the gap between the trigger firing and the regular sample being written at LateUpdate. See Timing for the full story.

Further reading​