Menu

1 / 19

You Can’t Watch 10,000 Episodes. Measure These Instead: Curation Metrics for LeRobot Datasets

DEV Community: computervision·Harpreet Sahota·about 2 hours ago
#zLFdXxDl
Reading 0:00
15s threshold

A field guide to the metrics worth computing on a LeRobot dataset before you train a policy or fine-tune a vision-language-action (VLA) model : what each one measures, when to trust it, where it misleads you, and how to run the whole workflow in FiftyOne.

You recorded 400 teleop episodes last week. You pulled another 6,000 from the Hugging Face Hub because the task looked close to yours. Now you want to fine-tune a VLA.

You can’t watch all of it. At 30 seconds per episode, 10,000 episodes is about 83 hours of video, and that’s before you open a second camera. So you scrub through a handful, they look fine, and you hit train.

Then the policy hesitates at the first grasp. Somewhere in there is an episode with a frozen wrist camera. Another has 11 seconds of dead time before the arm moves. A third has the task string "Hold". A fourth has a video track that runs 0.4 seconds behind the actions. None of these show up when you watch one episode at a time because none of them is visible in a single episode.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.