Menu

Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure
📰

Serving Sage-A1.5 on an RTX 4090: one broken flag, one queue, and a self-check mode to measure

#devcommunity#dev#model#card#llama
Reading 0:00
15s threshold

Run Sage-A1.5 as a research agent on your own card and the surprises are not in the model. They hide in a reasoning flag, a request limit, a token budget and one system prompt string.

Sage-A1.5 comes from Shanghai AI Laboratory (the InternScience organisation on Hugging Face), with a paper by Wenjie Lou and 98 co-authors. It is retrained from the Qwen team's open 35B model and the lab's earlier Agents-A1. The authors publish an official Q4_K_M GGUF , a compressed single-file version of about 22 GB, under Apache-2.0. Stock llama.cpp, the open server that runs models on one machine, loads it without changes. We ran it on one RTX 4090 on our cloud; every measurement is in the Sage-A1.5 single-GPU research write-up .

Why it fits on 24 GB, and what that means for sizing

It is a mixture-of-experts model: the weights are split into 256 small specialist blocks, and each token (roughly a word piece) goes through 8 of them plus one shared block.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.