Menu

1 / 5

How a 2021 Quantization Algorithm Quietly Outperforms Its 2026 Successor | Towards Data Science

Towards Data Science·Amit Portnoy·6 months ago
#saNCaN81
#ref5#ref1#editorspicks#deepdives#newsletter
Reading 0:00
15s threshold

[3] , an online vector quantization method, drew wide public attention at ICLR 2026. For me, it looked very familiar: it overlaps heavily with EDEN , a quantization method first introduced as the 1-bit method DRIVE at NeurIPS 2021 [1] and generalized to arbitrary bit-widths at ICML 2022 [2] . Co-authored by myself, with Ran Ben-Basat, Yaniv Ben-Itzhak, Gal Mendelson, Michael Mitzenmacher, and Shay Vargaftik.

The TurboQuant paper presents two variants: TurboQuant-mse and TurboQuant-prod . In a detailed new comparison [5] we show that TurboQuant-mse is a degenerate case of EDEN, and that the EDEN variants consistently outperform their counterparts.

How EDEN quantizes a vector

Suppose you need to compress a d d -dimensional vector x x (a gradient update, an embedding, a KV-cache entry) down to a few bits per coordinate. EDEN proceeds in four steps:

Random rotation — Multiply by a random orthogonal matrix Π \Pi .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.