Menu

Model symbiogenesis: the 200-row illusion and the 4-bit survivor
📰

Model symbiogenesis: the 200-row illusion and the 4-bit survivor

DEV Community: machinelearning·mycute·7 days ago
#2wk4WUhL
Reading 0:00
15s threshold

mycute

Posted on Nov 9

Edited on Nov 9

Model symbiogenesis: the 200-row illusion and the 4-bit survivor

Two LoRA adapters share one 4-bit Qwen3B host. One is a sentiment adapter (SST-2), the

other a topic adapter (AG News). This is not a container experiment; it asks whether two

learned behaviors coexist when their weights are merged.

The first 200-row probe looked dramatic. The full 400-row test is much more boring, which

is the useful result.

Reproduce:

python research/symbiogenesis.py # -> research/results/symbiogenesis.json

Enter fullscreen mode

Exit fullscreen mode

Full 400-row logit-scored accuracy

state SST-2 AG News

base 60.8% 59.2%

SST-2 adapter 94.5% 65.2%

AG News adapter 50.0% 45.5%

merge 0.25 (favor AG) 51.0% 52.0%

merge 0.50 90.0% 62.7%

merge 0.75 (favor SST-2) 94.5% 67.0%

merge 0.75, q8 94.5% 67.0%

merge 0.75, q4 94.5% 67.0%

The 0.75 merge matches the SST-2 adapter on SST-2 (94.5%) and improves AG News by 1.8 pp (67.0 vs 65.2).…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.