Enhancing Multimodal Personality Assessment with LLM-Augmented Hierarchical Fusion

摘要

This study proposes an LLM-augmented hierarchical fusion framework to enhance multimodal personality and ability assessment for the ACM MULTIMEDIA AVI CHALLENGE 2025, addressing semantic sparsity and cross-modal interaction limitations. We leverage large language models (e.g., Qwen, DeepSeek) to generate psychologically enriched text descriptions, bridging raw transcripts with expert evaluations, and integrate them with audio-visual features through early fusion and multi-path MLP ensembles. Track 1 (personality regression) employs dual-text inputs while Track 2 (multi-label ability prediction) uses parallel regression. Results show significant improvements: 23.1% MSE reduction over text-only baselines in Track 1, and 10.7%/12.5% gains over state-of-the-art fusion in Tracks 1/2, with 31.2% average improvement for cognitive traits (Q3-Q5). The framework demonstrates the effectiveness of semantic enhancement and adaptive fusion, with future work focusing on overfitting mitigation and feature optimization.