P PaperCompassAI 论文工作台
← 返回论文检索
ICLR 2026PosterAccept (Poster)

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

Mike Merrill, Alexander Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Di Lu, Orfeas Menis Mastromichalakis, Zhiwei Xu, Zizhao Chen, Yue Liu, Robert Zhang, Leon Liangyu Chen, Anurag Kashyap, Jan-Lucas Uslu, Jeffrey Li, Jianbo Wu, Minghao Yan, Song Bian, Vedang Sharma, Ke Sun, Steven Dillmann, Akshay Anand, Andrew Lanpouthakoun, Bardia Koopah, Changran Hu, Etash Guha, Gabriel Dreiman, Jiacheng Zhu, Karl Krauth, Li Zhong, Niklas Muennighoff, Robert Amanfu, Shangyin Tan, Shreyas Pimpalgaonkar, Tushar Aggarwal, Xiangning Lin, Xin Lan, Xuandong Zhao, Yiqing Liang, Yuanli Wang, Zilong (Ryan) Wang, Changzhi Zhou, David Heineman, Hange Liu, Harsh Trivedi, John Yang, Junhong Lin, Manish Shetty, Michael Yang, Nabil Omi, Negin Raoof, Shanda Li, Terry Yue Zhuo, Wuwei Lin, Yiwei Dai, Yuxin Wang, Wenhao Chai, Shang Zhou, Dariush Wahdany, Ziyu She, Jiaming Hu, Zhikang Dong, Yuxuan Zhu, Sasha Cui, Ahson Saiyed, Arinbjörn Kolbeinsson, Christopher Rytting, Ryan Marten, Yixin Wang, Jenia Jitsev, Alex Dimakis, Andy Konwinski, Ludwig Schmidt

Anthropic · Brigham Young University · Microsoft · Northeastern University · UC Santa Barbara + ScOp VC · Cornell University · Snorkel AI · Reflection AI · Columbia University · Carnegie Mellon University · Massachusetts Institute of Technology · Peking University · Independent Researcher · Juelich Supercomputing Center, LAION, Tuebingen University · Tencent · National Technical University of Athens · University of Michigan · Department of Computer Science, Cornell University · NUS · University of Texas at Austin · Stanford University · Amazon · University of California, Merced · Department of Computer Science, University of Wisconsin - Madison · University of Wisconsin-Madison · Fremont Unified School District · University of California, Berkeley · Sambanova Systems, Inc · Stanford University, Anthropic · Insitro · Stanford · Independent · New York University · Microsoft Research / Stanford · CMU · Michigan State University · UC Berkeley · Brown University · Boston University · UC San Diego · Allen Institute for Artificial Intelligence · Princeton University · University of California, Santa Barbara · Microsoft Research · Data61, CSIRO · OpenAI · Dartmouth College · University of California, San Diego · CISPA Helmholtz Center · State University of New York at Stony Brook · University of Illinois Urbana-Champaign · Yale University · University of Virginia, Charlottesville · UVA + K01 · Harbor · University of Michigan - Ann Arbor · LAION; Juelich Supercomputing Center, Research Center Juelich · Electrical Engineering & Computer Science Department, University of California, Berkeley · University of Washington / Stanford / Anthropic

论文页面 ↗ 登录后记笔记

PDF 由论文原始站点提供，PaperCompass 不保存论文文件。

摘要

AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not sufficiently difficult to meaningfully measure frontier models. To this end, we present Terminal-Bench 2.0: a carefully curated hard benchmark composed of 89 tasks in computer terminal environments inspired by problems from real workflows. Each task features a unique environment, human-written solution, and comprehensive tests for verification. We show that frontier models and agents score less than 65% on the benchmark and conduct an error analysis to identify areas for model and agent improvement. We publish the dataset and evaluation harness to assist developers and researchers in future work at tbench.ai.