LEROBOT / SO101 DATASET

A useful robot-learning dataset is more than a folder of episodes.

LeRobotDataset stores state/action time series, multimodal video and episode metadata. OpenRGD adds a provenance question: can each row be traced to the physical robot, calibration, camera and task protocol that produced it?

MATURITYDATASET REFERENCE · SMOKE RECORD OPERATOR-CONFIRMED
LAST VERIFIED16 Oct 2026
SOURCE VERSIONLeRobotDataset v3 documentation + OpenRGD operator work record · 16 Oct 2026
EVIDENCE BOUNDARYOne local no-video episode was reported; a public v3 dataset, pinned revision, video-synchronized dataset and training-ready evaluation corpus are not yet available as OpenRGD evidence.

RECORDED BASELINE

What the first local smoke dataset proves—and what it does not.

A work session on 30 August 2026 reported a local dataset identified as openrgd/so101-smoke: episode 0, 448 samples during a nominal 15-second capture without video. The operator reported a clean stop and successful replay. The dataset is not presented as a public Hugging Face repository, and no source checksum is available in the public evidence record.

That result supports the narrower claim that the local capture/replay path worked in the reported session. It does not establish dataset quality, fixed actual FPS, successful autonomous manipulation, generalization, or a SmolVLA/ACT trained policy.

LEROBOTDATASET V3

Separate low-dimensional signals, video and episode metadata.

The official v3 layout stores low-dimensional state/action/time-series signals in Parquet shards and camera frames in MP4 shards. The storage layout can concatenate multiple episodes per file, so episodes are recovered from metadata rather than assuming one file equals one trial.

The v3 metadata describes feature types/shapes and FPS, tasks, episode boundaries and global statistics. This allows training systems to load synchronized observation/action windows while the full physical provenance still needs to be documented separately.

meta/info.json

Feature and FPS schema

Feature names, data types, dimensions, codebase version and storage templates.

meta/tasks.jsonl

Task language

The task descriptions and task IDs associated with episodes.

meta/episodes

Episode indexing

Lengths, offsets and episode-to-shard relationships.

data + videos

Sensorimotor and image data

Parquet time series and video shards; not automatically a synchronized, validated lab instrument.

CAMERA PROVENANCE

Camera availability is not proof that an earlier episode contains video.

A laboratory may have usable USB cameras and still record an episode with no video. That is the case for the original reported SO-101 smoke run. A future dataset must freeze camera placement, lens, exposure behavior, image resolution, FPS, stream names and timestamp strategy before training.

If the learner uses a wrist view and an overhead view, the data card needs to map those stream names into model input features. A camera name alone does not prove calibration or synchronization.

DATA CARD GATE

The data card should describe how the dataset was produced.

Record exact LeRobot revision, robot identity, joint order, actuator units, calibration SHA-256, dataset revision, camera configuration, task setup, trial selection and operator resets. Preserve failure episodes rather than deleting them silently.

Published benchmark claims require independent held-out evaluation episodes and explicitly separated training data. Model evaluation cannot simply replay demonstrations and call that task success.

01Task definition→
02Preflight + calibration→
03Synchronized observation/action→
04Episode metadata→
05Checksums + data card→
06Train/validation split→
07Held-out evaluation

NEXT COLLECTION

The next milestone is a pinned, video-aware dataset—not a larger unverified smoke file.

The canonical next step is to repeat the baseline under a predeclared task protocol, capture raw logs and optional camera streams with tracked frame timing, archive the dataset and attach a machine-readable evidence record. The public lab protocol provides the fields needed to make that recording auditable.

PRIMARY SOURCES