I generate 512x768 images on a 4GB GTX 1650. About 150 seconds per image in fp32 with model offload. That is the number I plan around, and it does not leave room for rerunning a batch because one file was quietly broken.

How fp16 on this card produced NaN without raising

I switched the pipeline to fp16 to see if I could buy time back. Nothing crashed. No exception, no warning, no non-zero exit code. The script finished, PIL wrote the files, and the batch looked like a success. Some of the PNGs were pure noise, or flat grey, or black.

I did not trust my eyes, so I diffed the two runs. The only change was the precision flag:

# before pipe = StableDiffusionPipeline . from_pretrained ( model_id , torch_dtype = torch . float32 ) # after pipe = StableDiffusionPipeline . from_pretrained ( model_id , torch_dtype = torch . float16 )

Enter fullscreen mode

Exit fullscreen mode

Same seed, same prompt, same scheduler. fp32 gave me an image every time.…