MINIM: Privacy-Aware Minimal View for Agents via Trusted Local Sanitization

摘要

Modern LLM-powered autonomous agents increasingly rely on rich user interface (UI) state observations to achieve reliable action grounding in complex digital environments. However, many deployments transmit the full UI state to remote inference servers even when most elements are irrelevant to the current task, which can leak sensitive but unnecessary context such as authentication codes, private notifications, and background application states. We propose MINIM, a trusted local broker that performs privacy-aware minimization on the client side before any observation leaves the device. Grounded in Contextual Integrity (CI), MINIM learns a dual-score representation for each UI element by predicting an inherent sensitivity score (s) and a task-conditioned necessity score (n). These scores drive a trinary disclosure policy that keeps essential elements, abstracts sensitive attributes when needed, and removes task-irrelevant content. We optimize a CI-aware objective that penalizes necessity errors more strongly on high-risk content, enabling aggressive pruning while preserving task-critical information. Experiments on real-world UI observations derived from WebArena show that MINIM reduces task-irrelevant sensitive leakage to 1.1% while preserving 79.8% of task-critical semantic context. Moreover, MINIM retains 95.7% of task-critical interactive affordances, indicating that local sanitization can enforce task-conditioned minimization at both the structural and attribute levels while retaining the signals required for agent actions.