FalconCopilot: Empowering LLMs Towards Integrated Human-Machine Systems for Aviation Autonomy

摘要

Complex flight tasks demand both intricate, long-horizon decision-making and precise operations, which imposes immense cognitive, knowledge and experience demands for pilots and highlights the need for advanced copilot systems. While Large Language Models (LLMs) bring powerful potential to this area, a comprehensive LLM-based copilot system—one that addresses deficiencies in task-level adaptability and fine-grained decision support while integrating with a high-fidelity environment—is critically lacking. To address this gap, we present FalconCopilot, pioneering the first such comprehensive system, composed of two parts: 1) Textual DCS, an interface built upon Digital Combat Simulator (DCS) World that unifies multi-modal cockpit data and piloting knowledge into a stable semantic interface for LLMs. Building on this interface, we introduce 2) FalconAgent, an LLM-powered copilot agent that performs optimized task planning, incorporating capabilities for multi-crew task allocation and procedural pruning. Our built-in human-AI interaction is grounded by a bidirectional feedback loop of runtime verification and human correction. In human-in-the-loop experiment, FalconCopilot shortens task completion time while attaining a level of performance approaching that of a human instructor.