Skill: EDA with Pandas and Jupyter

Capability Domain: Data Science
Technology Stack: Python, Pandas, Jupyter, Plotly
Linked Activities: EDA with Pandas and Jupyter

Recipe

1. Pull via awswrangler.athena.read_sql_query(sql, database="ops_catalog", workgroup="ops-lake-analysts") — not a raw boto3 polling loop; awswrangler handles pagination and polling.
2. Pin the query text as a literal string in the notebook's first cell, and record the Athena QueryExecutionId in a markdown cell — that's what makes a finding reproducible against a catalog that keeps growing.
3. Classify the notebook's own question in a markdown cell up front — describe, classify, or predict — the acceptable rigor bar differs: describe needs clean summary stats; predict needs a held-out validation split and an explicit baseline to beat.
4. Standard opening cells, every notebook: df.info(), null-rate per column, duplicate-check on the join key (issue key / PR number) — most "surprising" operational-data findings turn out to be a duplicate-row bug.

Pitfalls

• AWS login refresh is not Athena AccessDenied. Local Jupyter/awswrangler can fail with AccessDeniedException on CreateOAuth2Token, then LoginRefreshRequired: "The refresh token has expired" / "Please reauthenticate using aws login". Wrangler never reached StartQueryExecution. This is credential session expiry, not lake IAM / workgroup deny. Procedure: (1) terminal aws login (or this org's equivalent aws sso login); (2) confirm aws sts get-caller-identity matches docs/lake-scope-contract.yaml aws_account (live opsdl: 411113550285); (3) Restart the Jupyter kernel, then Restart & Run All — boto3 clients in a running kernel keep the dead token; (4) the login credential provider also needs pip install 'botocore[crt]' (opsdl requirements-eda.txt).
• Re-running cells out of order and trusting the result is the most common way a notebook becomes unreproducible — Restart & Run All before treating any finding as final.