Menu

Need advice on choosing DB and caching architecture for a read-heavy Go service

Reading 0:00
15s threshold

We are planning to rewrite an existing Java service in Go. We're also free to redesign the architecture completely, as long as the end result and functionality remain the same.

I'd appreciate suggestions on database selection, caching strategy, and some additional requirements we're considering for the next version of the APIs.

Data model

Each entry contains the following fields:

- "id"

- "url"

- "name"

- "tags" (array of strings)

- "appGroupId"

- "rules" (JSON object)

We currently have around 50,000 entries.

Note: "appId" is provided by the caller and is not stored as part of an entry.

Workload

- Approximately 99% reads and 1% writes.

- Around 4-5 million reads per day (~46-60 reads/second on average).

- Multiple application pods running simultaneously in multiple clusters.

Existing architecture

The existing service uses S3 for persistence and maintains an in-memory cache in every application pod.

On startup, each pod loads all entries from S3 into memory.

Two in-memory maps are…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.