Type a sentence about how a car moved. The trained sentence encoder turns it into a vector,
and every held-out test clip is ranked by how close its motion vector is. Open any clip to go the other way:
its motion vector is matched against 5,670 descriptions.
Try an example
Reverse direction
Loading the clip index…
Text queryyour sentence
→
Sentence encoderfp16, runs in this browser
→
Motion embeddingsIMU + CAN clips, motion-encoded
↓
Cosine similarityagainst every test clip
·
→
Ranked clipstop ten, unfiltered
Ranked clips and their descriptions
Text-to-Motion: top ten of …, unfiltered
No search yet.
Select a retrieved clip to read its signals and labels.
How to read this
Candidates. All 3,623 held-out test clips, unfiltered: a clip that contradicts the sentence can still
rank high.
Clip view. The 2-second window is drawn in colour with 3 seconds of grey context on each side.
Signals are shown at 10 Hz; a channel the recording lacks is left blank.
GT label. The six ontology labels are ground truth. An unknown category is never counted as a match.
Requested. An example sentence carries declared states, so each clip gets a verdict. A sentence you
type shows the clip labels only.
Motion-to-Text. The clip's stored embedding is ranked against 5,670 complete descriptions; the top
one's six states are the prediction. Green agrees with the label, red differs, dashed has no valid label.
No encoder runs, so it works before the download.
Sentence encoder, motion embeddings and description embeddings from one trained
checkpoint. No ranking is precomputed or replayed.