If you send a video to a vision LLM, the usual first step is to sample a frame every few seconds. On a 4:44 conference talk, one frame every 10 seconds gives you 28 images. The talk has 7 slides. You pay for 21 repeats, and some of the 28 catch a slide halfway through a transition or before its last bullet appears.
What you actually want is one sharp frame per scene: per shot in a film, per slide in a talk. Here is how I built that, with numbers from real videos.
1. One cheap scan
Decode the video once, at 320 px wide in grayscale, and keep 0.5 to 4 samples per second depending on length. For each sample, compute three numbers:
Sharpness : variance of the Laplacian.
Change : mean absolute difference between this sample and the previous one, on a 64×36 thumbnail.
Flatness : whether the frame is nearly uniform (fades, black screens).
This pass is the only full decode of the video, so it sets the cost.…