We are planning to rewrite an existing Java service in Go. We're also free to redesign the architecture completely, as long as the end result and functionality remain the same.
I'd appreciate suggestions on database selection, caching strategy, and some additional requirements we're considering for the next version of the APIs.
Data model
Each entry contains the following fields:
- "id"
- "url"
- "name"
- "tags" (array of strings)
- "appGroupId"
- "rules" (JSON object)
We currently have around 50,000 entries.
Note: "appId" is provided by the caller and is not stored as part of an entry.
Workload
- Approximately 99% reads and 1% writes.
- Around 4-5 million reads per day (~46-60 reads/second on average).
- Multiple application pods running simultaneously in multiple clusters.
Existing architecture
The existing service uses S3 for persistence and maintains an in-memory cache in every application pod.
On startup, each pod loads all entries from S3 into memory.
Two in-memory maps are…