Somewhat related is stavros's method to compress 500KB to something like 50 bytes https://www.stavros.io/posts/compressing-images-with-stable-... main drawback is that it's not lossless ;-) but this is great. I hope this actually becomes a format that wraps the weights and transformer module (maybe this can also be NAS-optimized too?). Maybe it would even work for video? It's like calling gzip but instead of compression level you choose kolmogorov complexity level What do those compress to with conventional approaches? For comparison. I am curious. A classic machine learning ensemble approach is to overfit a collection of small models then bag them (e.g. voting) allowing the models to generalize. I'm sure someone's tried to overfit a bunch of transformers for compression like this, then bag them to see how well it does? Ensembling is not compute or parameter-efficient, so compression per se is a terrible application.…