Menu

Antonello Fratepietro

Antonello Fratepietro·3 months ago
#XqbMNFIk
#aaffee#patterns#actually#production#building
Reading 0:00
15s threshold

Oct 21, 2026 llama.cpp, vLLM, and SGLang: Three Approaches to LLM Inference

llama.cpp, vLLM, and SGLang all run transformer weights and stream tokens, yet each is built around a different systems bottleneck. Here is how they differ across model loading, KV cache memory, request scheduling, and execution, and how those choices set your latency, resource footprint, and serving cost.

Oct 20, 2026 How to Build a Decision Model Like Jev

Autoregressive decoding is the wrong tool when software needs a fast, calibrated probability over a fixed set of outcomes. Here is how System-One decision models work from the inside: Hugging Face's new /v1/systemone endpoint for turning 250+ LLMs and VLMs into zero-shot decision engines, reserved option tokens and readout heads, calibration with soft labels, tree-batched serving, and what black-box probing reveals about TypeSafe's Jev.

Oct 8, 2026 Google Cloud's Agentic AI Blueprint: From Design Patterns to Private Inference Networking

Google's Architecture Center now has a…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.