Personality Prediction via Multimodal Fusion with Sentiment Analysis Enhancement

摘要

Accurately identifying personality traits is of profound significance for gaining in-depth insights into human behavior, facilitating efficient human-computer interaction, and developing personalized intelligent systems. However, existing studies often treat personality trait prediction and emotion recognition as relatively independent tasks, neglecting the inherent correlation between them. This report proposes a multimodal fusion prediction framework through the research topic ''On the Interaction between Personality and Emotion in Human Behavior and Social Interaction''. The core goal of this framework is to explore and verify the positive gain of emotional analysis on the accuracy of personality prediction. We extracted visual and audio features based on large-scale data pre-training and fine-tuning, aggregated video features at the character level to enhance the stability of personality prediction, and then integrated the video-level emotion prediction branch. By jointly optimizing the losses of personality prediction and emotion prediction, the generalization performance of the model is improved. Experimental results show that the multi-task learning method integrating emotional information can improve the prediction performance of personality traits to a certain extent, and achieved the first place in the MER-PR validation set of the MER2025 Challenge. This provides empirical evidence for us to explore the complex interaction between emotion and personality.