Deep anchor-based multi-view clustering methods enhance the scalability of neural networks by utilizing representative anchors to reduce the computational complexity of large-scale clustering. Despite their scalability advantages, existing approaches often incorporate anchor structures in a heuristic or task-agnostic manner, either through post-hoc graph construction or as auxiliary components for message passing. Such designs overlook the core structural demands of anchor-based clustering, neglecting key optimization principles. To bridge this gap, we revisit the underlying optimization problem of large-scale anchor-based multi-view clustering and unfold its iterative solution into a novel deep network architecture, termed LargeMvC-Net. The proposed model decomposes the anchor-based clustering process into three modules: RepresentModule, NoiseModule, and AnchorModule, corresponding to representation learning, noise suppression, and anchor estimation. Each module is derived by unfolding a step of the original optimization procedure into a dedicated network component, providing structural clarity and optimization traceability. In addition, an unsupervised reconstruction loss aligns each view with the anchor-induced latent space, encouraging consistent clustering structures across views. Extensive experiments on several large-scale multi-view benchmarks show that LargeMvC-Net consistently outperforms state-of-the-art methods in terms of both effectiveness and scalability. The source data, code. https://github.com/dushide/LargeMvC-Net_ACMMM_2025, and extended version http://arxiv.org/abs/2507.20980 are available.
论文检索
输入标题、作者或关键词，从 1,620 篇学术成果中精准定位
Hard samples pose a significant challenge in person re-identification (ReID) tasks, particularly in clothing-changing person Re-ID (CC-ReID). Their inherent ambiguity or similarity, coupled with the lack of explicit definitions, makes them a fundamental bottleneck. These issues not only limit the design of targeted learning strategies but also diminish the model's robustness under clothing or viewpoint changes. In this paper, we propose a novel multimodal-guided Hard Sample Generation and Learning (HSGL) framework, which is the first effort to unify textual and visual modalities to explicitly define, generate, and optimize hard samples within a unified paradigm. HSGL comprises two core components: (1) Dual-Granularity Hard Sample Generation (DGHSG), which leverages multimodal cues to synthesize semantically consistent samples, including both coarse- and fine-grained hard positives and negatives for effectively increasing the hardness and diversity of the training data. (2) Hard Sample Adaptive Learning (HSAL), which introduces a hardness-aware optimization strategy that adjusts feature distances based on textual semantic labels, encouraging the separation of hard positives and drawing hard negatives closer in the embedding space to enhance the model's discriminative capability and robustness to hard samples. Extensive experiments on multiple CC-ReID benchmarks demonstrate the effectiveness of our approach and highlight the potential of multimodal-guided hard sample generation and learning for robust CC-ReID. Notably, HSAL significantly accelerates the convergence of the targeted learning procedure and achieves state-of-the-art performance on both PRCC and LTCC datasets. The code is available at https://github.com/undooo/TryHarder-ACMMM25.
In recent years, deep multi-view clustering methods based on contrastive learning have gained significant attention. Most existing approaches treat the anchor and its cross-view representations as positive pairs, while the anchor and other samples are considered negative pairs. However, this pairwise assignment does not account for the higher similarity between the anchor and samples that are close, which should also be treated as positive pairs. To address this issue, we introduce topology-aware positive sampling: for each anchor, both its intra-view neighbors and cross-view consistent neighbors are selected as additional positive samples, which aligns contrastive learning with the homophily principle of clustering. Additionally, to obtain reliable neighbor relationships, most existing methods construct graphs from the original data or extracted features and average them to form a consensus graph. However, this approach overlooks the fact that views of varying quality should be assigned different weights, and unreliable connections within a view should be discarded. To overcome this, we design a global-guided weak connections suppression mechanism to weaken unreliable connections in the initial graph of each view, then apply weighted graph fusion to obtain a more accurate consensus graph. We also combine the view weights from graph fusion with the corresponding view's neighbor contrastive loss to enhance consistency between the two processes. Extensive experimental results demonstrate the superiority of our proposed method.
Incomplete multi-view clustering (IMVC) deals with real-world scenarios where certain views are partially missing, posing significant challenges to effective clustering. Most existing IMVC approaches face a trade-off: imputation-free methods suffer from information bias and imbalance, while full-imputation methods risk introducing and propagating noise. To overcome these limitations, we propose Energy-Based Deep Incomplete Multi-View Clustering (Energy-DIMC), a novel selective-imputation framework that leverages energy-based models (EBMs) to guide reliable imputations and robust clustering. EBMs assess data compatibility by assigning lower energy to more coherent structures, effectively modeling complex inter-view and inter-sample dependencies. Inspired by EBMs, Energy-DIMC integrates four key components: 1) a view feature projector that learns view-specific features and projects them into a common feature space; 2) an energy-guided selective imputation module that identifies the most reliable source view for each view based on view energies, and performs feature imputation only when cross-view transfer is feasible, avoiding unreliable imputations; 3) an energy-based representation fusion module that aggregates observed and selectively imputed features across views via a view attention mechanism, generating view-coherent representations; 4) an energy-enhanced contrastive alignment module that enforces consistency between view-specific and view-coherent representations using dual-level energy signals to preserve true positives. Extensive experiments demonstrate that Energy-DIMC outperforms state-of-the-art IMVC methods across diverse missing-view scenarios. The code is available at https://github.com/sunway677/EnergyIMVC.
Tensorial Multi-view Clustering (TMC) methods have received significant attention due to their ability to capture the high-order correlation among different views. Despite their notable progress, two issues persist: 1) Most TMC methods are highly reliant on prior structure information in sorted datasets, rendering them challenging to handle the shuffled case. 2) Extremely high computational complexity arising from tensor-related operations. To address these limitations, we propose a novel framework termed Anchors Bring Stability and Efficiency: Fast Tensorial Multi-view Clustering on Shuffled Case (SE-FTMC). SE-FTMC first learns a set of anchors and a unified corresponding matrix between anchors and samples, then the low-rank tensor learning is adopted in anchor space instead of sample space to avoid reliance on the prior information hidden in sorted samples and reduce the computational complexity. Finally, SE-FTMC passes the high-order correlations learned in the anchor space into the sample space through the corresponding matrix to improve the efficiency and stability of clustering. Furthermore, SE-FTMC is solved by an efficient algorithm with linear complexity. Extensive experiments on various datasets demonstrate the effectiveness and superiority of our SE-FTMC compared with state-of-the-art methods. The code is publicly available at: https://github.com/jijintian/SE-FTMC.
Low-Rank Adaptation (LoRA) and its variants have delivered strong capability in Parameter-Efficient Transfer Learning (PETL) by minimizing trainable parameters and benefiting from reparameterization. However, their projection matrices remain unrestricted during training, causing high representation redundancy and diminishing the effectiveness of feature adaptation in the resulting subspaces. While existing methods mitigate this by manually adjusting the rank or implicitly applying channel-wise masks, they lack flexibility and generalize poorly across various datasets and architectures. Hence, we propose ReSoRA, a method that explicitly models redundancy between mapping subspaces and adaptively Regularizes Subspace redundancy of Low-Rank Adaptation. Specifically, it theoretically decomposes the low-rank submatrices into multiple equivalent subspaces and systematically applies de-redundancy constraints to the feature distributions across different projections. Extensive experiments validate that our proposed method consistently facilitates existing state-of-the-art PETL methods across various backbones and datasets in vision-language retrieval and standard visual classification benchmarks. Besides, as a training supervision, ReSoRA can be seamlessly integrated into existing approaches in a plug-and-play manner, with no additional inference costs. Code is publicly available at: https://github.com/Lucenova/ReSoRA.
Infrared and visible image fusion (IVIF) aims to extract fine details from visible images and complementary information from infrared images. Most existing methods directly extract relevant and complementary features from each modality using neural networks, often overlooking the guidance process and the distinct frequency-domain characteristics of these features. To address this, we propose HRFusion-a novel frequency-domain framework that extracts complementary features from hybrid features using prior-constrained relevant features, effectively enhancing complementary information and reducing redundancy. In HRFusion, hybrid and relevant features are robustly extracted to guide the subsequent fusion stage. By leveraging frequency differences between complementary and relevant features, we introduce the Enhanced Complementary Frequency Network (ECFNet), which uses optimized Variational Mode Decomposition (VMD) to effectively separate and process these signals for fusion. The overall architecture is built with the proposed DTBlock, which captures both global and local features. Extensive experiments show that our method achieves state-of-the-art performance on the TNO, MSRS, M3FD, and Harvard Brain datasets, outperforming recent approaches. Code is available at https://github.com/liuuuuu777/HRFusion.
Self-supervised 3D hand pose estimation methods can leverage labeled synthetic data along with unlabeled real-world data for model training, thereby alleviating the reliance on large-scale annotated datasets. Multi-view information fusion is a key factor in the success of these methods. Rule-based fixed fusion methods are simple, efficient, and generalizable, but they neglect the rich visual information in each view. Neural network-based learnable fusion methods can effectively model both intra- and inter-view semantic context, but they tend to overfit to the domain-specific feature of synthetic data and susceptible to interference of domain gaps. In this paper, we decompose multi-view fusion into two components: a learnable confidence estimation stage and a fixed confidence fusion stage. This design not only enables effective use of multi-view semantic cues but also ensures strong cross-domain generalization. To achieve accurate and robust confidence estimation, our method jointly exploits both multi-view pose consistency and pose-to-data consistency. Experiments on three public datasets demonstrate that our approach significantly outperforms existing state-of-the-art self-supervised 3D hand pose estimation methods.
Unlike Object Detection, Visual Grounding task necessitates the detection of an object described by complex free-form language. To simultaneously model such complex semantic and visual representations, recent state-of-the-art studies adopt transformer-based models to fuse features from both modalities, further introducing various modules that modulate visual features to align with the language expressions. However, in most cases, the supervision signal only adopts common Object Detection losses, solely governing the bounding box regression output, which fails to fully optimize for the above objectives. To tackle this problem, in this paper, we first analyze the attention mechanisms of transformer-based models. Building upon this, we propose a novel framework named Attention-Driven Constraint Balancing (AttBalance) to optimize the behavior of visual features within language-relevant regions. Extensive experimental results show that our method brings impressive improvements. Specifically, we achieve constant improvements over five different models evaluated on four different benchmarks. Moreover, we attain a new state-of-the-art performance by integrating our method into QRNet.
Recently, since diffusion models show great potential in image generation, many pretrained diffusion models based image composition methods have been proposed for image illumination harmonization. However, they mainly face two key challenges: 1) the effective preservation of foreground appearance (i.e., content structure and texture details, etc); 2) Reasonable generation of the foreground casting shadow. To this end, we propose a novel Image Illumination Harmonization Diffusion model called I 2 HDiffuser to achieve image illumination harmonization with high-fidelity foreground appearance and reasonable cast shadows. I 2 HDiffuser mainly consists of frequency domain feature enhancement branch (FDFEB) and illumination-shadow consistency generation branch (ISCGB). Specifically, FDFEB first introduces the Wavelet Transform Module (WTM) for decomposing composite image features into low-frequency (i.e., illumination features, etc) and high-frequency (i.e., texture and content structure features, etc) components using the Haar wavelet transform. Then the Multi-Condition Guidance Mechanism (M-CGM) is proposed to interact these components as prior conditions, which are further injected into the ISCGB with a noise-to-denoise process for guiding high-fidelity content and background illumination-aware foreground regeneration. Meanwhile, a shadow mask step-wise iterative optimization strategy is introduced to the ISCGB to explicitly provide a reasonable shadow generation space for foreground objects. Extensive experiments on public image harmonization datasets DESOBAv2 and iHarmony4 and real illumination harmonization dataset IH-SG show that the I 2HDiffuser achieves the superiority.
Spatial-spectral fusion offers a promising alternative to expensive equipment in high-resolution hyperspectral (HrHs) imaging. However, training separate models for different scaling factors remains costly. To address this, we propose the Arbitrary-scale Fusion Neural Operator (AFNO), a lightweight solution for HrHs fusion across arbitrary scalings. Instead of entities, AFNO treats low-resolution hyperspectral (LrHs) and high-resolution multispectral (HrMs) images as functions and performs meticulously designed integrations as the mapping operator. The key components include Attention-Driven Convolution Integration (ADCI) to restore discretization invariance disrupted by convolutions, Implicit Neural Functional Integration (INFI) for cross-domain interaction of spatial degradations, and Galerkin-type Integration as a decoder for high-frequency details. Additionally, the bonded activation opeartor are improved for the principle of continuous-discrete equivalence. Extensive experiments validate the superiority of our approach over cutting-edge methods. Notably, AFNO holds significantly better generalization on arbitrary scaling factors, yet requiring only 0.07M parameters.
By clustering pixels with locally similar values, superpixel-based approaches have shown great potential in processing hyperspectral images (HSI) , thereby reducing the computational burden associated with large spatial dimensions. However, specific for spatial-spectral fusion (SSF), superpixel segmentation is inherently non-differentiable and irreversible; hence it is inapplicable. To address the issues, we propose a semantic transformer-based solver, namely SpecSolver, which is basically inspired by the benefits of superpixel-based approaches, yet with the inner mechanism completely improved. The core idea lies in learning the intrinsic semantic states of HSIs hidden behind discretized pixel representations. Specifically, we propose a new Semantic-Attention to adaptively split the image domain into a series of learnable slices of flexible shapes, where image pixels under similar semantic states will be ascribed to the same slice. By calculating attention to the Semantic-Superpixel tokens encoded from slices, SpecSolver can effectively capture intricate semantic correlations from the vast number of pixels, which also empowers the solver with an endogenous capacity for modeling different magnification scales and allows for efficient computation in linear complexity. On that basis, we elaborate a SpatialNet module, which extracts multiscale local spectral information, and a FreqNet module, which supplements global information, capturing subtle details and variations across different spectra. Experiments on two benchmark SSF datasets verify the state-of-the-art (SOTA) performance of the proposed method, both visually and quantitatively. Also, ablation studies validate the mentioned contributions.
Multi-view clustering aims to extract and integrate semantic information from multiple views to improve clustering performance. While deep learning-based approaches have shown promising results, they suffer from noisy view dependency (NVD) and dominant view dependency (DVD), limiting their robustness and effectiveness. NVD arises when models fail to filter out irrelevant variations, treating noise as semantic information. DVD occurs when models over-rely on dominant views, neglecting complementary information from other perspectives. To address these challenges, we propose causal content-style representation learning for deep multi-view clustering. To mitigate NVD, we incorporate causal content-style disentanglement via a dual differential content-style network for separation of semantic information from noise. Meanwhile, to reduce DVD, we introduce causal content consistency that aligns semantic content from both intra-view and cross-view perspectives. Besides, we design a content-centered style receptive field for contrastive learning, enhancing the semantic association between positive sample pairs while preventing over-alignment to dominant views. Extensive experiments on ten benchmark datasets demonstrate that CausalMVC outperforms state-of-the-art methods, validating its effectiveness.
Knowledge distillation as an efficient knowledge transfer technique, has achieved remarkable success in unimodal scenarios. However, in cross-modal settings, conventional distillation methods encounter significant challenges due to data and statistical heterogeneities, failing to leverage the complementary prior knowledge embedded in cross-modal teacher models. This paper empirically reveals two critical issues in existing approaches: distillation path selection and knowledge drift. To address these limitations, we propose MST-Distill, a novel cross-modal knowledge distillation framework featuring a mixture of specialized teachers. Our approach employs a diverse ensemble of teacher models across both cross-modal and multimodal configurations, integrated with an instance-level routing network that facilitates adaptive and dynamic distillation. This architecture effectively transcends the constraints of traditional methods that rely on monotonous and static teacher models. Additionally, we introduce a plug-in masking module, independently trained to suppress modality-specific discrepancies and reconstruct teacher representations, thereby mitigating knowledge drift and enhancing transfer effectiveness. Extensive experiments across five diverse multimodal datasets, spanning visual, audio, and text, demonstrate that our method significantly outperforms existing state-of-the-art knowledge distillation methods in cross-modal distillation tasks. The source code is available at https://github.com/Gray-OREO/MST-Distill.
Industrial anomaly detection for 2D objects has gained significant attention and achieved progress in anomaly detection (AD) methods. However, identifying 3D depth anomalies using only 2D information is insufficient. Despite explicitly fusing depth information into RGB images or using point cloud backbone networks to extract depth features, both approaches struggle to adequately represent 3D information in multimodal scenarios due to the disparities among different modal information. Additionally, due to the scarcity of abnormal samples in industrial data, especially in multimodal scenarios, it is necessary to perform anomaly generation to simulate real-world abnormal samples. Therefore, we propose a novel unified multimodal anomaly detection framework to address these issues. Our contributions consist of 3 key aspects. (1) We extract visible depth information from 3D point cloud data simply and use 2D RGB images to represent appearance, which disentangles depth and appearance to support unified anomaly generation. (2) Benefiting from the flexible input representation, the proposed Multi-Scale Gaussian Anomaly Generator and Unified Texture Anomaly Generator can generate richer anomalies in RGB and depth. (3) All modules share parameters for both RGB and depth data, effectively bridging 2D and 3D anomaly detection. Subsequent modules can directly leverage features from both modalities without complex fusion. Experiments show our method outperforms state-of-the-art (SOTA) on MVTec-3D AD and Eyecandies datasets. Code available at: https://github.com/Xantastic/BridgeNet
Fetal MRI is often acquired with thick slices to mitigate motion artifacts, but this leads to partial volume effects and reduced through-plane spatial resolution, limiting precise anatomical analysis. To address this, various super-resolution methods have been proposed to reconstruct high-resolution volumes from thick-slice scans. Current methods face several major challenges: 1) relying on multi-stack paired data makes arbitrary super-resolution ratios difficult to achieve; 2) lacking robustness against voxel coordinate misalignment caused by partial volume effects; 3) failing to fully utilize the high in-plane resolution of MRI images. To address these issues, we propose a dual-consistency guided curriculum learning method based on implicit neural representation, which uses single-stack inputs to achieve arbitrary super-resolution. We introduce progressive consistency and volumetric consistency to mitigate voxel misalignment caused by partial volume effects and ensure smooth transitions during the model's curriculum-based training. Additionally, we design a curriculum-aware multi-scale feature interaction block to fully leverage thick-slice MRI's high in-plane resolution. Comprehensive evaluations on three fetal MRI datasets demonstrate SOTA performance, with particularly outstanding results in high-ratio super-resolution tasks.
Multimodal recommendation systems effectively address data sparsity with visual and textual features but face two primary challenges: (1) Modal noise contamination, where high-frequency noise introduced by feature extractors distorts the semantic representation of items, and (2) Limited multimodal feature fusion capability, where conventional static fusion strategies struggle to capture fine-grained user preferences across modalities. To address these challenges, we propose the Frequency-refined graph convolutional network with cross-modal wavelet denoising for Recommendation (FreRec). Specifically, for the first challenge, FreRec introduces a dual denoising framework that: (i) employs wavelet transforms to decompose multimodal features into low-frequency semantics and high-frequency details, applying adaptive soft-thresholding to remove noise while preserving essential information; and (ii) implements a degree-sensitive edge pruning strategy to eliminate noisy connections in the user-item interaction graph, preventing noise propagation during message passing. For the second challenge, FreRec proposes an adaptive multimodal fusion approach that: (i) constructs a dual-view graph convolutional framework that jointly models user-item interactions and intra-modal semantic similarities; and (ii) leverages the behavior-guided frequency fusion module to dynamically adjust modality importance based on user behavior patterns, achieving robust fusion that adapts to individual preferences. Extensive experiments on three benchmark datasets validate the superiority of our model.
Handwritten Mathematical Expression Recognition (HMER) remains a challenging task due to the structural complexity of mathematical notation and the ambiguity of handwritten symbols-e.g., ''ρ'' vs. ''p'' or ''B'' vs. ''β''. While stroke-based models offer disambiguation via temporal cues, most existing methods are constrained by coarse modality fusion and a lack of fine-grained cross-modal alignment, further hindered by limited annotated data. We introduce Art for Math (Art4Math), a novel framework that leverages the structural richness of human sketches to enhance HMER through fine-grained, modality-aware learning. Art4Math follows a two-stage training paradigm: Art Grounding (A-Grd) and Math Decoding (M-Dec). In A-Grd, the model is trained to reconstruct masked regions of sketches via joint modeling of visual and stroke-level features, encouraging sensitivity to local structural cues and inter-modality alignment. This Art Grounding cultivates a strong inductive bias for parsing abstract, sparse visual forms. M-Dec then adapts this representation to the HMER domain, enabling more precise symbol disambiguation and structural decoding with limited supervision. Extensive experiments across sketch and handwriting-related tasks, including sketch recognition, retrieval, and HMER, demonstrate that Art4Math significantly outperforms existing self-supervised methods, revealing the overlooked synergy between artistic abstraction and mathematical expression.
For skeleton-based action recognition, Graph Convolutional Networks (GCNs) are effective models. Still, their reliance on floatingpoint computations leads to high energy consumption, limiting their applicability in battery-powered devices. While energy-efficient, Spiking Neural Networks (SNNs) struggle to model skeleton dynamics, leading to suboptimal solutions. We propose Signal-SGN (Spiking Graph Convolutional Network), which utilizes the temporal dimension of skeleton sequences as the spike time steps and represents features as multi-dimensional discrete stochastic signals for temporal-frequency domain feature extraction. It combines the 1D Spiking Graph Convolution (1D-SGC) module and the Frequency Spiking Convolution (FSC) module to extract features from the skeleton represented as spiking form. Additionally, the Multi-Scale Wavelet Transform Feature Fusion (MWTF) module is proposed to extract dynamic spiking features and capture frequency-specific characteristics, enhancing classification performance. Experiments across three large-scale datasets reveal Signal-SGN exceeding state-of-the-art SNN-based methods in accuracy and computational efficiency while attaining comparable performance with GCN methods and significantly reducing theoretical energy consumption.
Graph fraud detection has garnered significant attention as Graph Neural Networks (GNNs) have proven effective in modeling complex relationships within multimodal data. However, existing graph fraud detection methods typically use preprocessed node embeddings and predefined graph structures to reveal fraudsters, which ignore the rich semantic cues contained in raw textual information. Although Large Language Models (LLMs) exhibit powerful capabilities in processing textual information, it remains a significant challenge to perform multimodal fusion of processed textual embeddings with graph structures. In this paper, we propose a Multi-level LLM Enhanced Graph Fraud Detection framework called MLED. In MLED, we utilize LLMs to extract external knowledge from textual information to enhance graph fraud detection methods. To integrate LLMs with graph structure information and enhance the ability to distinguish fraudsters, we design a multi-level LLM enhanced framework including type-level enhancer and relation-level enhancer. One is to enhance the difference between the fraudsters and the benign entities, the other is to enhance the importance of the fraudsters in different relations. The experiments on four real-world datasets show that MLED achieves state-of-the-art performance in graph fraud detection as a generalized framework that can be applied to existing methods.