mycute
Posted on Nov 9
Edited on Nov 9
4-bit Naive Bayes: symmetric fails on AG News, zero-point rescues it
Minimum bits is not one knob. A small model has two budgets: data bytes and weight
bits . I ran two experiments on the same project to see which one matters where.
1. 4-bit Naive Bayes
The SMS model is a 160-word Complement NB with int8 weights. I quantized the same weights to
int4 with a per-class scale and kept the raw integer dot product (the rule the fixed SMS
containers deploy). Five 80/20 seeds:
precision weight bytes mean accuracy
float 1,280 B 91.6%
int8 320 B 94.2%
int4 160 B 93.2%
On SMS, int4 keeps 93.2% at half the weight bytes. Seed 0 is 93.6% int4 vs 93.5% int8.
The full JSON still holds the 160-word vocabulary (about 1.2 KB), so the artifact shrinks by
roughly 10%, not 50%. The weight matrix itself is halved.
Then the same recipe on the 500-example, 4-class AG News coreset:
precision weight bytes accuracy
int8 20,596 B 78.3%
int4 10,298 B 66.5%
It collapses.…