Skill: Scope the Operations Lake

Capability Domain: Discovery
Technology Stack: YAML, GitHub CLI, AWS STS
Linked Activities: Scope Sources, Org, and Environment

When to use

First activity of STEP 1. Do not model CDK stacks until docs/lake-scope-contract.yaml exists.

Recipe

1. Clone https://github.com/FeatureFactory-io/ops-lake-starter (GitHub → Use this template). Do not invent infra/ and do not copy the live opsdl lake. The clone already has docs/lake-scope-contract.yaml plus the four CDK stacks and GHA workflows.
2. Identify the host of the lake repo and the source systems that actually exist (github, gitlab, jira). Never add a source "just in case".
3. For a GitHub org, default entities are: commits, pull_requests, issues, workflow_runs, project_items. Drop any the org does not use. List org repos (gh api orgs/{org}/repos) and Projects v2 (gh api graphql organization.projectsV2).
4. Set ci to exactly one of github_actions | gitlab_ci matching host. XOR.
5. Record aws_account from aws sts get-caller-identity, aws_region, env (sandbox|dev|staging|prod), lake_name.
6. Record lake_repo (short name of the lake git repo; rename from ops-lake-starter if the clone is named differently) and oidc_sub. GitHub OIDC tokens use two sub shapes; DeployStack must trust both (see GitHub Actions CI/CD skill).
7. Derive Athena view names as {source}_{entity}. That list is the catalog contract later activities must implement. There is no fixed set of four GitLab/Jira views.
8. Fill the file below in the clone. Stop. The next activity is Model the Data Lake as CDK Stacks.
9. If jira is in sources, add the Jira flow keys below and follow skill Jira Flow Extract. Omit the whole block when jira is not scoped.

Copy-this: docs/lake-scope-contract.yaml

Template lives in the starter kit. Fill, do not invent from scratch:

org: YOUR_ORG                   # GitHub org, GitLab group, or company
host: github                    # github | gitlab
sources: [github]               # subset of: github, gitlab, jira
github_entities:                # omit this key if github not in sources
  - commits
  - pull_requests
  - issues
  - workflow_runs
  - project_items
github_projects: []             # fill from Projects v2; empty if none
repos: []                       # org repo names in scope
ci: github_actions              # github_actions XOR gitlab_ci
lake_repo: ops-lake-starter     # git repo that hosts the CDK app
# Trust both name-only and ID-qualified GitHub OIDC subs (see DeployStack).
oidc_sub: repo:YOUR_ORG/ops-lake-starter:*
aws_account: "000000000000"
aws_region: us-east-1
env: sandbox                    # sandbox | dev | staging | prod
lake_name: ops-lake
athena_database: ops_catalog
athena_workgroup: ops-lake-analysts
views:                          # derived; STEP 2 must create exactly these
  - github_commits
  - github_pull_requests
  - github_issues
  - github_workflow_runs
  - github_project_items
buckets:
  raw: ops-lake-raw-{account_suffix}
  curated: ops-lake-curated-{account_suffix}
  athena_results: ops-lake-athena-results-{account_suffix}
paths:
  raw: "s3://{raw}/{source}/{yyyy}/{mm}/{dd}/{event-id}.json"
  curated: "s3://{curated}/{table}/source={source}/dt={yyyy-mm-dd}/part-{uuid}.parquet"
secrets:
  github_webhook: ops-lake/github-webhook-secret
  github_api_token: ops-lake/github-api-token

Jira flow keys (only when jira is in sources)

Add these keys; extend views and secrets. Do not add them for a GitHub-only lake.

jira_project_keys: []           # parameterized JQL project keys; never unbounded
jira_entities:
  - issues
  - status_segments
started_statuses:               # cycle-time start set (names or IDs)
  - In Progress                 # example convention only — replace with this org's Started statuses
jira_edition: cloud             # cloud | server
secrets:
  jira_api_token: ops-lake/jira-api-token
# Also add to views:
#   - jira_issues
#   - jira_status_segments

Lead time = created → resolutiondate (business days). Cycle time = first started_statuses entry → resolutiondate. See Jira Flow Extract.

Done when

• Starter kit cloned (not opsdl).
• File exists and views is non-empty.
• ci is exactly one value.
• sources lists only systems this org has.
• lake_repo, oidc_sub, and secret names are filled.
• If jira is scoped: jira_project_keys, started_statuses, and jira views are filled.

Pitfalls

• Hard-coding gitlab_commits / jira_issue_transitions for a GitHub-only org.
• Standing up both GitHub Actions and GitLab CI against the same account.
• Naming a GitHub Actions repo secret GITHUB_* — GitHub rejects that prefix. Use OPSLAKE_GITHUB_PAT and AWS_DEPLOY_ROLE_ARN.
• Publishing or cloning the live opsdl lake as the template (real org, account, PAT, findings).
• Scoping jira without a Started status set or without status_segments — cycle time will be empty or invented.