Open benchmark · 60 input-prediction tasks
SWE-Test
Benchmarking LLM Vulnerability Discovery
via Input Prediction
Evaluate whether coding agents can read real C and C++ source code and construct inputs that trigger target branches.
Input prediction on 60 fixed target branches using source code without runtime feedback. Each model is ranked by its best selected configuration.
Filter model attributesSize classes are relative to the models evaluated in this benchmark.
Best published configuration per model
Click Scaffold or Effort to compare another configuration.Top models by pass ratePass rate (%)
| Rank | Effort | Avg. reward |
|---|
iPass rate: share of the 60 tasks with reward ≥ 2.0, meaning the target branch was triggered; unscored tasks count as zero. Avg. reward: sum of per-task rewards divided by 60.