Skill: EDA with Streamlit Data Mart

Capability Domain: Business Analytics
Technology Stack: Streamlit, ECS Fargate, ALB, Athena
Linked Activities: EDA with Streamlit Data Mart (ECS Fargate + ALB)

Skip when (QuickSight)

No QuickSight subscription. Do not lead STEP 4 with QuickSight. Optional later: Athena data source, SPICE refresh ≤ crawler cadence, anomaly detection only on validated metrics.

Recipe (Streamlit — the proven path)

1. Code lives in streamlit/ in the lake repo. CDK EdaStack builds streamlit/Dockerfile as linux/amd64 and runs ECS Fargate behind an ALB. Never App Runner (no WebSockets; blank skeleton).
2. Dockerfile must COPY *.py . and COPY pages pages/. Missing glue_catalog.py crashed production (ModuleNotFoundError).
3. List Glue tables with boto3 paginated GetTables (streamlit/glue_catalog.py). Do not use wr.catalog.tables (incomplete list / default limit). Catalog cache TTL ≤ 60s.
4. Task role: Athena start/get, Glue GetTables, curated_bucket.grant_read (ListBucket deny → Athena PERMISSION_DENIED), Athena-results read/write. Prove with IAM simulate in deploy.yml.
5. .streamlit/config.toml: disable CORS/XSRF as needed behind ALB; websocket compression off if the ALB flakes.
6. Baseline #1 — Feature Cycle Time (pages/1_Feature_Cycle_Time.py + cycle_time.py):
   - Big picture first: median total hours, WoW % last 12 weeks, features completed.
   - Line chart of median total by completion week; stacked bars Plan/Build/Test/Finalize; WoW table; project/repo filter.
   - Query $.payload.*; join issues to PRs on Closes #N / #N in body, never timestamps alone.
   - This page is a look-until-trusted check, not the Tell the Data Story opening visual (that remains the reconstructed cycle swimlane).
7. Lake Explorer page: sample contract views for debugging.
8. Additional look-until-trusted pages: follow skill Metric Render Catalog. Only render families whose contract views exist. KPI strips use inverse delta when lower is better. Prefer per-item median + n over a mean of daily averages.

Pitfalls

• App Runner for Streamlit.
• Docker COPY only app.py.
• wr.catalog.tables hiding views that exist in Glue.
• Task role without curated-bucket ListBucket.
• Feature filter that only accepts GitHub label Feature — org work uses enhancement, status-done, W27:, [mimir:mcp], BPE-01.
• Leading the mart home page with Feature Cycle Time instead of the reconstructed cycle when Present Findings uses Streamlit.
• Mixing issue lead/cycle (business days) with MR created→merged (hours) on one sparkline without labeling units.