Four Elasticsearch SLM policies (15-min, hourly, monthly, yearly) write to four repositories in one S3 bucket.
Nodes authenticate with an EC2 instance profile. Zero access keys in the keystore.
SLM handles retention, from 7 days up to ~7 years. S3 lifecycle rules stay off, on purpose.
The whole thing is about 30 lines of JSON and a handful of AWS CLI commands.
Three questions every backup should answer
Elasticsearch cluster runs on EC2 in a regulated environment. I wanted backups that answer three questions in one sentence each: how much data can we lose, how far back can we go, and what credentials does the job use?
Elasticsearch already ships a scheduler for this: Snapshot Lifecycle Management (SLM). So I moved everything into SLM and let S3 be dumb, durable storage.
The architecture
Read it left to right. Nodes get temporary credentials from an IAM role. SLM fires snapshots on cron schedules. The S3 repository writes objects into an SSE-S3 encrypted bucket, one folder per tier.…