Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success

摘要

Model merging combines knowledge from separately fine-tuned models, yet success factors remain poorly understood. While recent work treats mergeability as an intrinsic property, we show with an architecture-agnostic framework that it fundamentally depends on both the merging method and the partner tasks. Using linear optimization over a set of interpretable pairwise metrics (e.g., gradient $L_2$ distance), we uncover properties correlating with post-merge performance across four merging methods. We find substantial variation in success drivers (46.7\% metric overlap; 55.3\% sign agreement), revealing method-specific "fingerprints". Crucially, however, \textit{subspace overlap} and \textit{gradient alignment} metrics consistently emerge as foundational, method-agnostic prerequisites for compatibility. These findings provide a diagnostic foundation for understanding mergeability and motivate future fine-tuning strategies that explicitly encourage these properties.