Causal Intervention Router improves LLM agent recovery on ALFWorld
Tags AI / ML
A new arXiv paper introduces the Causal Intervention Router, a lightweight policy that selectively triggers harness recovery, raising Qwen3-14B success on ALFWorld from 70.33% to 73.33% across 300 episodes. The gain is +3.00 percentage points (95% CI [0.67, 5.67]) from 75 held-out tasks, with 11 rescues and 2 harms and no clean trajectories refreshed. The largest gain is +9.33 points on two-step stale observations (60.00% to 69.33%), from 9 rescues and 2 harms. The method uses paired counterfactual evaluation, requires no retraining of the underlying agent, and intervenes in 16.3% of episodes.
Technical significance
The paper addresses a practical reliability problem in agent harnesses: average success rates hide whether a recovery mechanism rescues or harms individual trajectories. By using paired counterfactual runs to decide when to intervene, CIR improves success while leaving clean trajectories untouched, which matters for production agents where unnecessary retries add cost and latency. The effect size is modest and measured on a single benchmark with one model, so the approach's generality across tasks and larger models remains to be shown.