Multimodal Time Series Alignment for Error Detection in Human Robot Interactions

摘要

Errors often occur during human-robot interactions, such as failing to respond, interrupting users, or providing answers that do not meet user expectations. Detecting these issues on time is crucial for making human-robot communication more natural and user-friendly. In this technical report, we present the approach proposed by our team, CFM-HRI, for the ERR@HRI 2.0 Challenge 2025, targeting the task of interaction error detection. Specifically, we propose a lightweight and efficient time-series classification approach, empowered by cross-modal alignment, to detect interaction errors more accurately and promptly. To mitigate temporal misalignment across modalities, we adopt an upsampling alignment strategy, followed by feature fusion to obtain unified representations. A sliding-window voting mechanism is then introduced to construct training samples along with corresponding ground truth labels. Several machine learning models are employed to detect errors based on the fused features. Experimental results demonstrate the effectiveness of our approach in capturing cross-modal inconsistencies and improving detection accuracy. Our approach won first place in Sub-Challenge 2 and second place in Sub-Challenge 1 of the ERR@HRI 2.0 Challenge, held in conjunction with ACM MM 2025. We provide detailed descriptions of our data processing and experimental setup, along with an analysis of the limitations of our approach and potential directions for future work. Our code is available on https://github.com/setsaile/CFM-HRI-ERR-HRI2.0.