Menu

Post image 1
Post image 2
Post image 3
1 / 3
36

Data Filtering for Generative Video Pre-training | Field Notes by Linum

#linum#model#training#article#video#ama
Reading 0:00
15s threshold

Image and video models have gotten a lot better over the last few years, even though the internals of these models haven't changed much since Stable Diffusion 3. Of course, there have been small variants like the auto-regressive diffusion that GPT-Image popularized. But at a high level, it's pretty much all flow matching with a transformer backbone and a v-prediction objective. In our experience, most of the gains are directly attributable to 3 flavors of data improvements: RL has driven some improvement, but it's only started truly working for image and video in the past few months. Data Filtering & Rebalancing : Remove noisy data and resample your data strategically so your model learns more effectively Data Annotation : Gather better annotations like richer captions, bounding boxes, and font details so that it's easier for your model to disambiguate visual concepts LLMs have gotten infinitely better at image captioning in the last 12 months. Less so for video, but that's for another time.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More