Deploy via GitLab CI

MCE-3 Order: #3 Datalake Formation Has Dependencies

Updated 1 month, 2 weeks ago

Purpose

Deploy the CDK app through GitLab CI with OIDC. Alternative to GitHub Actions.

Lake Scope Contract (ci must be gitlab_ci).
CDK Application.
Starter .gitlab-ci.yml.example — copy to .gitlab-ci.yml. Skill GitLab CI/CD → Recipe.

ci is github_actions. Running both against one AWS account causes split-brain deploys.

Copy .gitlab-ci.yml.example to .gitlab-ci.yml. Do not also keep a live GitHub deploy.yml against this account.
Chicken-and-egg: laptop cdk bootstrap then one cdk deploy OpsLakeDeploy to mint the GitLab OIDC IAM role. Store the role ARN as CI/CD variable AWS_DEPLOY_ROLE_ARN (not AWS_ACCESS_KEY_ID). Trust aud = https://gitlab.com (or this instance URL) and the project sub.
id_tokens block mints the OIDC token. pytest → cdk synth → cdk diff → cdk deploy with when: manual for production. Register GitLab Environment production.
After first deploy, record URLs/ARNs in docs/deploy-manifest.yaml.

Pipeline (populate equivalent on GitLab if you add it; otherwise GitHub populate-lake is the GitHub-ci path).

Lock the lake's scope and stand the AWS estate up as code so every later step lands on the same …

Agent: Data Platform Engineer Role Owns Datalake Formation: Scope Contract → Python CDK (DeployStack + …

CDK Application (Infrastructure Stacks) Document Required
Produced by: Model the Data Lake as CDK Stacks
Lake Scope Contract Document Required
Produced by: Scope Sources, Org, and Environment