Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
UCSD; MIT-IBM · University of California, San Diego · Qwen, Alibaba Group Georgia Tech · University of Illinois at Urbana-Champaign · Princeton University · University of Pittsburgh · UIUC · Abaka AI · Stanford University · University of California, San Diego, University of California, San Diego · Computer Science Department, Stanford University · Stanford/Massachusetts Institute of Technology
PDF 由论文原始站点提供，PaperCompass 不保存论文文件。
摘要
Existing evaluations of agents with memory typically assess **memorization** and **action** in isolation. One class of benchmarks evaluates memorization by testing recall of past conversations or text but fails to capture how memory is used to guide future decisions. Another class focuses on agents acting in single-session tasks without the need for long-term memory. However, in realistic settings, memorization and action are tightly coupled: agents acquire memory while interacting with the environment, and subsequently rely on that memory to solve future tasks. To capture this setting, We introduce MEMORYARENA, a unified evaluation gym for benchmarking agent memory in multi-session Memory-Agent-Environment loops. The benchmark consists of human-crafted agentic tasks with explicitly interdependent subtasks, where agents must learn from earlier actions and feedback by distilling experiences into memory, and subsequently use that memory to guide later actions to solve the overall task. MEMORYARENA supports evaluation across web navigation, preference-constrained planning, progressive information search, and sequential formal reasoning, and reveals that agents with near-saturated performance on existing long-context memory benchmarks like LoCoMo perform poorly in our agentic setting, exposing a gap in current evaluations for agents with memory.