Activities in EDA with Python, Pandas and Jupyter
All activities across workflows in this playbook
Activities 43
| Name | Abbrev. | Workflow | Phase | Actions |
|---|---|---|---|---|
| Choose the analysis type | FPM-1 | Frame the Problem | Frame the Problem | |
| Formulate the driving questions | FPM-2 | Frame the Problem | Frame the Problem | |
| Assess analyst readiness | FPM-3 | Frame the Problem | Frame the Problem | |
| Select candidate datasets from the datalake catalog | FPM-4 | Frame the Problem | Frame the Problem | |
| Assess dataset quality against meaningfulness traits | FPM-5 | Frame the Problem | Frame the Problem | |
| Write the Framing Brief | FPM-6 | Frame the Problem | Frame the Problem | |
| Load selected datasets into DataFrames | ODA-1 | Organize and Clean Data | Explore Data and Find Signal | |
| Inspect structure | ODA-2 | Organize and Clean Data | Explore Data and Find Signal | |
| Handle missing and invalid data | ODA-3 | Organize and Clean Data | Explore Data and Find Signal | |
| Combine and deduplicate datasets | ODA-4 | Organize and Clean Data | Explore Data and Find Signal | |
| Reshape data for the framing questions | ODA-5 | Organize and Clean Data | Explore Data and Find Signal | |
| Compute descriptive statistics per candidate variable | EFS-1 | Explore and Engineer Features | Explore Data and Find Signal | |
| Choose candidate target variable(s) Y | EFS-2 | Explore and Engineer Features | Explore Data and Find Signal | |
| Engineer and derive features | EFS-3 | Explore and Engineer Features | Explore Data and Find Signal | |
| Cross-tabulate categorical candidates against Y | EFS-4 | Explore and Engineer Features | Explore Data and Find Signal | |
| Compute correlation/covariance and significance between features and Y | EFS-5 | Explore and Engineer Features | Explore Data and Find Signal | |
| Interpret correlation with p-value and rule out spurious relationships | ACE-1 | Assess Information Gain and Conclude | Explore Data and Find Signal | |
| Design and run a predictive experiment | ACE-2 | Assess Information Gain and Conclude | Explore Data and Find Signal | |
| Evaluate model quality | ACE-3 | Assess Information Gain and Conclude | Explore Data and Find Signal | |
| Use clustering as an alternative when correlation is weak | ACE-4 | Assess Information Gain and Conclude | Explore Data and Find Signal | |
| Rank features by information gain and draw conclusions | ACE-5 | Assess Information Gain and Conclude | Explore Data and Find Signal | |
| Identify the analytical pattern type per conclusion | ECR-1 | Encode and Reduce Clutter | Design the Visual Story | |
| Select the optimal quantitative scale | ECR-2 | Encode and Reduce Clutter | Design the Visual Story | |
| Add reference lines and regions | ECR-3 | Encode and Reduce Clutter | Design the Visual Story | |
| Choose pre-attentive encodings by precision hierarchy | ECR-4 | Encode and Reduce Clutter | Design the Visual Story | |
| Plan trellis or multi-view layout | ECR-5 | Encode and Reduce Clutter | Design the Visual Story | |
| Reduce over-plotting | ECR-6 | Encode and Reduce Clutter | Design the Visual Story | |
| Check for spurious or missed patterns | ECR-7 | Encode and Reduce Clutter | Design the Visual Story | |
| Design a time-series display | CTE-1 | Choose Chart per Analysis Type | Design the Visual Story | |
| Design a part-to-whole/ranking display | CTE-2 | Choose Chart per Analysis Type | Design the Visual Story | |
| Design a deviation display | CTE-3 | Choose Chart per Analysis Type | Design the Visual Story | |
| Design a distribution display | CTE-4 | Choose Chart per Analysis Type | Design the Visual Story | |
| Design a correlation display | CTE-5 | Choose Chart per Analysis Type | Design the Visual Story | |
| Design a multivariate display | CTE-6 | Choose Chart per Analysis Type | Design the Visual Story | |
| Sequence the narrative | SNK-1 | Storyboard the Notebook | Design the Visual Story | |
| Draft one storyboard entry per chart | SNK-2 | Storyboard the Notebook | Design the Visual Story | |
| Review the storyboard against conclusions | SNK-3 | Storyboard the Notebook | Design the Visual Story | |
| Set up the Plotly/Jupyter environment | IVS-1 | Implement Plotly Visuals | Build the Story in Plotly | |
| Implement each storyboard entry as a Plotly figure | IVS-2 | Implement Plotly Visuals | Build the Story in Plotly | |
| Apply scale/reference-line/annotation rules via layout methods | IVS-3 | Implement Plotly Visuals | Build the Story in Plotly | |
| Build trellis/small-multiples and multi-view linking | IVS-4 | Implement Plotly Visuals | Build the Story in Plotly | |
| Assemble the Jupyter storybook notebook | IVS-5 | Implement Plotly Visuals | Build the Story in Plotly | |
| Review and export the final notebook/report | IVS-6 | Implement Plotly Visuals | Build the Story in Plotly |