As augmentation turns a single human demonstration into thousands of robot episodes, the advantage shifts to how carefully that one capture was made.

Teaching a robot a new task has mostly meant paying someone to drive the robot through that task, over and over, until there are enough recordings to train on. Human video looks like the obvious shortcut, since people already do this work and filming them needs no robot hardware at all. The hard part has always been getting from a flat video of a hand to something a machine with a gripper can actually learn from.

A recent paper called Real2Sem2Real takes that problem head on, describing a pipeline that turns one ordinary video of a person doing a task into robot demonstrations spread across many different instances of the same object category.…