Lately I've been using video diffusion models and learning more about them. As I was digging around the various codebases I noticed that there were a couple of common approaches to actually encoding the frames that are generated by the diffusion models: mainly use PyAV or shell out to FFmpeg. There's nothing especially wrong with either of these approaches but I think it is possible for us to do a little better. So I (along with my robot friends) built Blazar. You can find the code repo for Blazar here .

Today with PyAV the flow is roughly:

Convert tensor from BF16 or F32 to u8 and reshape to (H, W, C)

D2H transfer

RGB to YUV conversion

(if NVENC) upload to the GPU

Encode

So, I think, this is roughly what we want to do:

Be able to take the native output tensor shape of the model as an input.…