Partition by Source and Date at Minimum
partition-by-source-and-date-at-minimum · always_apply: yes
Content
Every curated table is Hive-partitioned as {table}/source={source}/dt={yyyy-mm-dd}/ so Athena and Glue crawlers prune scans. {table} is a Lake Scope Contract view name. Raw JSON uses prefix {source}/{yyyy}/{mm}/{dd}/ in a separate bucket. Add finer partitions only if query patterns justify them.
Details
- Slug:
partition-by-source-and-date-at-minimum- Always apply:
- Yes
- Created:
- Aug 31, 2026
- Updated:
- Sep 14, 2026
Playbook
Activities 2
- Store Compressed and Partitioned in S3 Build the Operational Data Lake Pipeline
- Catalog with AWS Glue Build the Operational Data Lake Pipeline