Backfill Historical Jira Issues and Status History

Purpose

When jira is in the Scope Contract, pull historical issues plus a status timeline into the raw bucket so Constraint and Flow Analysis can compute lead time and cycle time. A poller that only sees current status cannot reconstruct cycle start.

Skip when

Scope Contract sources does not include jira. Do not stand up Jira Lambdas or views "just in case". Do not copy jira_flow.py.example onto a GitHub-only lake.

Inputs

• Lake Scope Contract (sources, project keys from skill Scope the Operations Lake → Jira flow keys, views, Started status set, watermark / secret names).
• Starter stub infra/lambdas/jira_flow.py.example — copy to infra/lambdas/jira_flow.py only if jira is scoped.
• CDK Application (JiraPollFn / Jira backfill function already declared when jira is scoped).
• Skill Jira Flow Extract → Extraction recipe.
• Skill Serverless Data Ingestion (handler wiring only).

Steps

1. If jira is scoped: copy infra/lambdas/jira_flow.py.example → jira_flow.py. Open skill Jira Flow Extract → Extraction recipe. Implement that contract — do not invent a different clock. Secret ops-lake/jira-api-token; base URL from contract Jira flow keys.
2. Persist both issue snapshots (closed_at = resolutiondate) and status segments (status, entered_at, exited_at).
3. JQL is scoped and bounded (updatedDate). Cloud: cursor pagination. Server/DC: startAt. Page size about 100.
4. Build the timeline from changelog field == status. Paginate per-issue changelog when search expand truncates.
5. Record the Started status set on the contract. Compute lead and cycle from the landed columns (business days). Do not re-query Jira in EDA for those clocks.
6. Incremental cursor in DynamoDB keyed by source. Periodic bounded backfill for tickets older than the watermark. Populate-lake today is GitHub-shaped — if you add a Jira invoke job, put it in the same workflow; do not laptop-invoke shared env.
7. Write unmodified JSON to s3://{raw}/jira/{yyyy}/{mm}/{dd}/{event-id}.json. Parquet is the next activity.

Done when

• Glue (or the contract views) contain resolved rows (closed_at not null), not only open issues.
• jira_status_segments (or the named contract view) has at least one closed segment per resolved issue that has changelog.
• Athena proof LIMIT 10 succeeds for every jira view that has Parquet.
• Lead/cycle can be computed from curated columns without another Jira call.

Feeds

Store Compressed and Partitioned in S3, then Glue/Athena. Constraint analysis may use these clocks as measured touch/wait inputs; it does not replace impact scoring.

Honesty

Sprint/velocity board reports are out of scope here. Do not treat them as lead or cycle.