An Alloy user had one header configured on pyroscope.scrape , an X-Api-Key . About two hours in, profiling stopped. Their packet capture showed a single GET carrying roughly 1,000 copies of that header , and the target answering 431 Request Header Fields Too Large .

The bug was in prometheus/common , the library behind http_config in Prometheus, Alertmanager and the exporters. The reporter had already traced it to cloneRequest . I wrote the upstream fix, #982 , which shipped in v0.72.1 .

Reproducing it

One header in the config, the same request sent four times:

$ go run . # prometheus/common v0.71.0 server saw X-Api-Key x1 server saw X-Api-Key x2 server saw X-Api-Key x3 server saw X-Api-Key x4

$ go run . # prometheus/common v0.72.1 server saw X-Api-Key x1 server saw X-Api-Key x1 server saw X-Api-Key x1 server saw X-Api-Key x1

Enter fullscreen mode

Exit fullscreen mode

The copy that copied nothing

r2 := new ( http . Request ) * r2 = * r // Deep copy of the Header. maps . Copy ( r . Header , r2 .…