Menu

1 / 3

πŸ“¦ My Redis Payload Was Too Large, So I Just Gzipped It

DEV Community: redisΒ·Ferry Ananda FebianΒ·5 days ago
#CWlUrkR0
#devtoredis#devto#dev#json#redis
Reading 0:00
15s threshold

A while ago I started getting this error in production: "payload too large" . The culprit was a Redis value that kept growing, a big JSON of vehicle data that we cache and read all the time.

My first thought was to bump the limit. My second thought was: why am I storing this much text in the first place? JSON is super repetitive. Same keys, over and over, for thousands of rows. That's basically a gift for compression.

So I tried gzip. It turned out to be one of the simplest fixes I've done, and it gave me more benefits than I expected.

🧩 The Idea

Nothing fancy:

If the JSON is small, store it as usual.

If it's bigger than a threshold, gzip it, encode it as base64, and add a gz: prefix.

When reading, check the prefix. Has gz: ? Decompress. No prefix? Parse as usual.

That prefix is the trick that made this safe to ship. A valid JSON string never starts with gz: , so old entries already sitting in Redis still work.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.