Skill: AWS CDK Infrastructure as Code

Capability Domain: Infrastructure as Code
Technology Stack: AWS CDK (Python default), CloudFormation
Linked Activities: Model the Data Lake as CDK Stacks

Language

Greenfield lake repo: Python (aws-cdk-lib, pytest). TypeScript only if the host repo is already TypeScript. Do not mix.

Recipe

1. Read docs/lake-scope-contract.yaml. Stacks instantiate only sources and views listed there.
2. Layout in the starter kit (https://github.com/FeatureFactory-io/ops-lake-starter): infra/app.py, infra/ops_lake/deploy_stack.py, ingestion_stack.py, catalog_stack.py, eda_stack.py, scope.py. Customize from the filled contract; do not scaffold empty stacks.
3. Four stacks:
   - DeployStack (OpsLakeDeploy): GitHub (or GitLab) OIDC provider + deploy IAM role. Trust both sub patterns (name-only and ID-qualified). Output DeployRoleArn.
   - IngestionStack: scoped ingestion Lambdas (GithubWebhookFn, GithubBackfillFn, GithubProjectsPollFn if github scoped), HTTP API, DynamoDB cursor table, raw/curated/athena-results buckets, SQS buffer, ParquetTransformFn. GithubBackfillFn timeout 10 min / 1024 MB.
   - CatalogStack: Glue database, one crawler per views entry with daily EventBridge schedule cron(0 6 * * ? *), Athena workgroup (5GB scan cap), named preview queries.
   - EdaStack: Streamlit mart on ECS Fargate + ALB (not App Runner — WebSockets unsupported). Image linux/amd64. Task role: Athena + Glue GetTables + curated_bucket.grant_read + Athena results read/write.
4. Bucket encryption: s3.BucketEncryption.KMS_MANAGED (SSE-KMS) and bucket_key_enabled=True (S3 Bucket Keys). Do not attach a customer-managed kms.Key to buckets that also have S3→SQS notifications — CloudFormation hits LakeKey -> RawBucket -> LakeKey. Without Bucket Keys, every PutObject/GetObject is a KMS request; one-file-per-event Parquet plus Glue/Athena reads burns the 20k always-free KMS requests in a single backfill (recapture X-15, 2026-09-15). Bucket Keys apply to new objects only.
5. pytest: tests/unit/test_scope_contract.py, test_deploy_stack.py, test_ingestion_stack.py, test_catalog_stack.py, test_eda_stack.py. Use CDK context skip_bundling for synth tests without Docker. Must be green before CI deploy. Assert BucketKeyEnabled is true on all three lake buckets. Copy-this tests stay in the starter kit.
6. cdk deploy from a laptop is bootstrap + one sandbox mint of OpsLakeDeploy only. Shared env is GHA/cdk deploy --all.

Copy-this tests

Template.from_stack on IngestionStack: 3 buckets, each SSEAlgorithm: aws:kms with BucketKeyEnabled: true, GithubWebhook present, Gitlab/Jira absent. CatalogStack: Glue database name from contract, crawler count == len(views), Athena workgroup name from contract. DeployStack: OIDC provider + IAM role. EdaStack: Fargate service + ALB, no App Runner.

Pitfalls

• Modeling GitlabWebhookFn / JiraPollFn when sources: [github].
• Customer-managed KMS key shared by bucket + SQS notification (CFN cycle).
• SSE-KMS without S3 Bucket Keys (bucket_key_enabled=False) — KMS Free Tier gone in one backfill morning.
• cdk deploy from a laptop to a shared account after the OIDC role exists.
• Putting Streamlit on App Runner (blank skeleton; /_stcore/stream 403).
• Scaffolding empty infra/ instead of customizing the starter kit clone.