S3 Bucket Keys With SSE-KMS
s3-bucket-keys-with-sse-kms · always_apply: yes
Content
SSE-KMS on lake buckets (raw, curated, Athena results) must set bucket_key_enabled=True (S3 Bucket Keys). Without it, every PutObject is a KMS GenerateDataKey and every GetObject is a Decrypt. One-file-per-event Parquet plus Glue/Athena reads burned the 20k always-free KMS requests in a single backfill morning (opsdl, 2026-09-15). Template: infra/ops_lake/ingestion_stack.py _make_bucket / _make_results_bucket. Bucket Keys apply to new objects; existing objects keep per-object keys until overwritten. Do not attach a customer-managed kms.Key to buckets that also have S3→SQS notifications — CloudFormation hits LakeKey -> RawBucket -> LakeKey. Use BucketEncryption.KMS_MANAGED + Bucket Keys.
Details
- Slug:
s3-bucket-keys-with-sse-kms- Always apply:
- Yes
- Created:
- Sep 15, 2026
- Updated:
- Sep 15, 2026
Playbook
Activities 2
- Store Compressed and Partitioned in S3 Build the Operational Data Lake Pipeline
- Model the Data Lake as CDK Stacks Model and Deploy Infrastructure as Code