Discover real-world bugs

EAE-1 Order: #1 Evaluation

Updated 4 months ago

Guidance

Rank variable usage locations by model confidence that a different variable should be used. Inspect the top-N locations (top 500 in the paper) where the model strongly disagrees with the ground truth. Classify findings as true bugs vs. valid alternatives (e.g. aliases). Report confirmed bugs with code snippets.

Details

Order:: #1
Phase:
Created:: May 20, 2026
Last Updated:: May 20, 2026

Workflow

Evaluate and Analyze

Evaluate on seen and unseen project test sets, compare against baselines, run ablation study on edge types and node labels, …

View Workflow

Assigned Agent

No agent assigned

Required Skills

No skills linked

Rules

No rules linked.

Input Artifacts

No input artifacts

Output Artifacts 1

Evaluation Report Document Required