Menu

Turning a 5-minute talk into 7 images for an LLM: picking one sharp frame per scene
📰

Turning a 5-minute talk into 7 images for an LLM: picking one sharp frame per scene

DEV Community: computervision·Nookforge Studio·14 days ago
#6AwTemsH
Reading 0:00
15s threshold

If you send a video to a vision LLM, the usual first step is to sample a frame every few seconds. On a 4:44 conference talk, one frame every 10 seconds gives you 28 images. The talk has 7 slides. You pay for 21 repeats, and some of the 28 catch a slide halfway through a transition or before its last bullet appears.

What you actually want is one sharp frame per scene: per shot in a film, per slide in a talk. Here is how I built that, with numbers from real videos.

1. One cheap scan

Decode the video once, at 320 px wide in grayscale, and keep 0.5 to 4 samples per second depending on length. For each sample, compute three numbers:

Sharpness : variance of the Laplacian.

Change : mean absolute difference between this sample and the previous one, on a 64×36 thumbnail.

Flatness : whether the frame is nearly uniform (fades, black screens).

This pass is the only full decode of the video, so it sets the cost.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.