A field guide to 5 manipulation datasets in LeRobot v3.0 , 2 real and 3 synthetic, with their exact fields and three strategies for generating robot data when real teleoperation can't scale.

Large language models train on trillions of tokens. Vision-language models train on billions of images. The largest dataset in this post has 637,498 trajectories, all synthetic. The two real, teleoperated datasets top out at 4,258 and 1,804 episodes. Every real demonstration requires a human at a controller, a physical robot, and a prepared scene. That collection process doesn't scale like web data.

We pulled 5 manipulation datasets into LeRobot v3.0 and FiftyOne: two real, teleoperated single-arm datasets and three simulated datasets spanning single-arm and bimanual setups.…