Menu

1 / 7

GLM-5.3 on Workers AI Returned Other Sessions' KV Cache Pages

DEV Community: cloud·Taha Echakiri·17 days ago
#yE7wtfpC
#devtocloud#devto#dev#request#pool
Reading 0:00
15s threshold

TL;DR

Workers AI is Cloudflare's hosted inference service, and GLM-5.3 is the Z.ai text-generation model it served to customers. A KV cache is the working memory an inference engine keeps for one request: the keys and values already computed for the prompt, stored in fixed-size GPU memory pages that later requests reuse.

Cloudflare's incident report, published on 26 October 2026, records 155 responses between 27 August and 9 September 2026 that carried data from other customers' sessions, every one of them to a prompt longer than 681,280 tokens.

Two numbers came from different places: the configured context window was 1,048,576 tokens, and the prefill KV pool on the serving GPUs held 681,280, because SGLang sizes that pool at startup from whatever GPU memory is left after the weights load.

In disaggregated prefill/decode serving, a request the scheduler had marked as rejected still reached the decode node, which read the stale pages left by earlier requests and answered with HTTP 200.

Cloudflare set the…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.