Evaluate model quality

ACE-3 Order: #3 Explore Data and Find Signal Has Dependencies

Updated 1 month, 2 weeks ago

Guidance

Build a confusion matrix (classification) or compare predicted-vs-actual on train and test sets (regression); derive accuracy/precision/sensitivity/specificity and choose which one matters for the question at hand. Compare train-set fit against test-set fit to judge overfitting before trusting the model.

Details

Order:: #3
Phase:
Predecessor:: ACE-2 Design and run a predictive experiment
Successor:: ACE-4 Use clustering as an alternative when correlation is weak
Created:: Sep 18, 2026
Last Updated:: Sep 18, 2026

Workflow

Assess Information Gain and Conclude

Rigorously test whether the feature table actually carries information about Y — via significance-tested correlation, a validated predictive experiment, and …

View Workflow

Assigned Agent

EDA Practitioner

The person or AI system running this playbook end to end — analyst, programmer, and …

Required Skills

Predictive Experiment Design and Model Validation Machine Learning Methodology scikit-learn

Rules

Choose the right accuracy metric for the use case choose-the-right-accuracy-metric-for-the-use-case
Model quality must be measured, not assumed model-quality-must-be-measured-not-assumed

Input Artifacts

No input artifacts

Output Artifacts 1

Model Validation Report Document Required