EladSer
Posted on Nov 17
AI-assisted
mtop is a terminal view for local model servers. One window shows the models you have loaded and the VRAM they hold, the GPU, and the requests going through with their tok/s. It works with Ollama, llama.cpp (including router mode and llama-swap), LM Studio, vLLM, Lemonade and SGLang.
Requests while they stream
Until now a request only showed up once it finished. In 1.6 it appears at the top of the requests pane as soon as it starts, with how long it's been going, tokens so far and a live tok/s. At 100 columns you also get time to first token and decode time, which is usually where the answer to "why did that feel slow" is.
For streamed OpenAI-style requests the final tok/s is now decode only. If llama.cpp sends its own predicted_per_second , mtop uses that.…