Most data engineers assume that a single "ETL-Role" shared across an entire orchestration platform is sufficient because the services are internal. This assumption is a common precursor to lateral movement if a single job is compromised, as that job inherits the broad permissions of every other pipeline in your environment. When a job that only needs to read from a specific S3 prefix suddenly has the ability to delete tables in your data warehouse or read sensitive secrets from Parameter Store, your blast radius is effectively the entire platform.
This article shows how to break the "shared service role" pattern by implementing job-specific IAM roles. We will focus on defining discrete execution roles for individual tasks, ensuring that a failure in one pipeline component cannot be used to escalate privileges or access data outside of that component’s narrow scope.…