The Problem We Were Actually Solving

At 12 k concurrent users, the Golang-based treasure-hunt engine began allocating 2 GB of heap in <250 ms.

A 512-byte event message ballooned into 4 MB after being processed through a nested map[string]interface{} structure.

Allocations triggered three full GC cycles before the response escaped the handler, violating our SLA of <50 ms p99.

Running go tool pprof revealed 68 % of CPU time inside yaml.Unmarshal, but we could not patch the allocator without risking a rewrite of the entire event pipeline.

What We Tried First (And Why It Failed)

We swapped syaml.v3 for go-yaml.v3, expecting a 2× speed-up.

Metrics showed p99 down to 187 ms, still unacceptable.

Digging into the heap profile, v3 still materialized the entire document before streaming could begin—our real cost was the intermediate in-memory tree.

We next rewrote the schema layer in protobuf and generated Go with gogoproto.

Latency fell to 23 ms, but CPU usage spiked 28 % because protobufs reflection can…