mycute
Posted on Nov 9
Edited on Nov 9
Model symbiogenesis: the 200-row illusion and the 4-bit survivor
Two LoRA adapters share one 4-bit Qwen3B host. One is a sentiment adapter (SST-2), the
other a topic adapter (AG News). This is not a container experiment; it asks whether two
learned behaviors coexist when their weights are merged.
The first 200-row probe looked dramatic. The full 400-row test is much more boring, which
is the useful result.
Reproduce:
python research/symbiogenesis.py # -> research/results/symbiogenesis.json
Enter fullscreen mode
Exit fullscreen mode
Full 400-row logit-scored accuracy
state SST-2 AG News
base 60.8% 59.2%
SST-2 adapter 94.5% 65.2%
AG News adapter 50.0% 45.5%
merge 0.25 (favor AG) 51.0% 52.0%
merge 0.50 90.0% 62.7%
merge 0.75 (favor SST-2) 94.5% 67.0%
merge 0.75, q8 94.5% 67.0%
merge 0.75, q4 94.5% 67.0%
The 0.75 merge matches the SST-2 adapter on SST-2 (94.5%) and improves AG News by 1.8 pp (67.0 vs 65.2).…