Handle missing and invalid data

ODA-3 Order: #3 Explore Data and Find Signal Has Dependencies

Updated 1 month, 2 weeks ago

Guidance

Detect gaps with isnull()/notnull(). Then choose deletion (dropna(), whole rows/columns only), imputation (constant/mean via masked assignment, or fillna(method="ffill"/"bfill")), selective replacement (replace()), or backfill from a secondary source (combine_first()) — the right choice depends on domain knowledge about what a gap means for that variable, not a default rule.

Details

Order:: #3
Phase:
Predecessor:: ODA-2 Inspect structure
Successor:: ODA-4 Combine and deduplicate datasets
Created:: Sep 18, 2026
Last Updated:: Sep 18, 2026

Workflow

Organize and Clean Data

Turn the selected raw datasets into one clean, combined, correctly-shaped working DataFrame — the shared substrate every later exploration step …

View Workflow

Assigned Agent

EDA Practitioner

The person or AI system running this playbook end to end — analyst, programmer, and …

Required Skills

Missing Data Handling Data Cleaning pandas

Rules

Justify the imputation method by domain knowledge justify-the-imputation-method-by-domain-knowledge
Missing values can't be dropped cell-by-cell missing-values-cant-be-dropped-cell-by-cell

Input Artifacts

No input artifacts

Output Artifacts

No output artifacts