www.lesswrong.com/posts/WgzwBi6DCagDuHPzP/introducing-linuxarena
1 correction found
BenchBench
This appears to be a typo: Redwood’s 2025 control setting is called BashBench, not “BenchBench.” The linked arXiv paper (2504.10374) is the Ctrl-Z paper introducing BashBench.
Full reasoning
Redwood’s own AI Control overview lists its follow-up settings as BashArena, BashBench, and LinuxArena—not “BenchBench.” In addition, the linked paper for arXiv:2504.10374 is Redwood’s Ctrl-Z: Controlling AI Agents via Resampling, whose abstract says: “We construct BashBench, a dataset of 257 challenging multi-step system administration tasks…” So the post’s reference to “BenchBench” does not match either Redwood’s project naming or the linked paper; the intended name is BashBench.
2 sources
- Redwood Research — AI Control
Since the seminal AI Control paper, Redwood Research has done a variety of follow-up work studying AI control in diverse settings such as BashArena, BashBench, and LinuxArena.
- Ctrl-Z: Controlling AI Agents via Resampling
We construct BashBench, a dataset of 257 challenging multi-step system administration tasks... Paper link: https://arxiv.org/abs/2504.10374