Anchor graph has gained considerable attention in the field of multi-view clustering due to its capability to handle large-scale datasets and superior clustering performance. However, current approaches hypothesize that views select the equal number of anchors and only share the anchor graph of single magnitude. This limits their representability and ignores the varying data distributions from multiple views. In addition, most of anchor strategies attain the clustering labels by performing post-processing on anchor graph, which consumes additional time and exists the loss of information. So as to address the above problems, this paper presents a Dual-Constraint Multi-view Fuzzy Clustering with Scalable Anchor Graph Learning (DMFC-SAGL) that explores the anchor graphs of different magnitudes and directly derives the clustering labels via fuzzy clustering. In particular, DMFC-SAGL first learns the multi-scale anchor graphs based different quantity of anchors to accommodate the distinct data distributions and extract more comprehensive information. Then, by carrying out fuzzy clustering for anchor graphs, the soft labels can be generated directly without additional post-processing. Moreover, DMFC-SAGL imposes the dual-constraint of low-rank tensor and orthogonality during label learning to ensure the information of consistency and diversity among multi-scale anchor graphs. Experiments with advanced baselines on the seven multi-view datasets indicate the superiority of the proposed method. The code is available at https://github.com/whbdmu/DMFC-SAGL.
论文检索
输入标题、作者或关键词，从 1,620 篇学术成果中精准定位
Multi-view clustering based on anchor graph has gained a lot of attention because of its ability to handle large-scale datasets in linear time. However, the existing methods learn anchors from linear space, ignore the nonlinear manifold characteristics of the original data, and fail to fully consider the geometric topological relationship of anchors in the space, which limits the representability of the anchor graph and makes these algorithms unable to deal with the complex data distribution in real scenarios. In addition, most methods involve the control of multiple hyper-parameters, which often require a significant amount of time for tuning, resulting in algorithms that are not as flexible and scalable as they could be. To address the above problems, this paper proposes Scalable multi-view clustering based on tight anchor distribution (TAD-MVC), where TAD-MVC closely associates neighboring anchors together and learns the degree of closeness between all anchors. In particular, TAD-MVC adaptively evaluates the tightness between anchors and spreads this tight structure with the aim of constraining anchors with a high degree of similarity to the same cluster, while anchors in different clusters will exclude each other as much as possible in order to learn anchors that are sufficiently representative. Experimental results on numerous datasets significantly outperforms the state-of-the-art methods. The code is available at https://github.com/whbdmu/TAD-MVC.
The oracle bone inscription (OBI) recognition plays a significant role in understanding the history and culture of ancient China. However, the existing OBI datasets suffer from a long-tail distribution problem, leading to biased performance of OBI recognition models across majority and minority classes. With recent advancements in generative models, OBI synthesis-based data augmentation has become a promising avenue to expand the sample size of minority classes. Unfortunately, current OBI datasets lack large-scale structure-aligned image pairs for generative model training. To address these problems, we first present the Oracle-P15K, a structure-aligned OBI dataset for OBI generation and denoising, consisting of 14,542 images infused with domain knowledge from OBI experts. Second, we propose a diffusion model-based pseudo OBI generator, called OBIDiff, to achieve realistic and controllable OBI generation. Given a clean glyph image and a target rubbing-style image, it can effectively transfer the noise style of the original rubbing to the glyph image. Extensive experiments on OBI downstream tasks and user preference studies show the effectiveness of the proposed Oracle-P15K dataset and demonstrate that OBIDiff can accurately preserve inherent glyph structures while transferring authentic rubbing styles effectively. The dataset, code, and pre-trained models are available at https://github.com/LJHolyGround/Oracle-P15K.
While recent advancements in supervised gait recognition have yielded promising results, these approaches rely heavily on annotated walking data, limiting their generalizability to complex environments. This paper presents a self-supervised gait recognition framework using human poses as input to address this challenge, focusing on high-quality pretrained data and self-supervised learning strategies. We first introduce StreamGait, a large-scale, unlabelled dataset that captures in-the-wild distributions of walking sequences. This dataset is curated from Internet livestreams across diverse geographic and environmental scenarios, reflecting variations in real-world camera angles, weather, and pedestrian behavior. Our framework, MirrorGait, conducts self-supervised learning by integration with 2D-to-3D pose reconstruction to synthesize multi-view perspectives for effective 3D-aware contrastive learning. With specific designs of temporal position embedding and gait partition head on a Transformer backbone, the encoder can readily adapt to the periodic and fine-grained nature of gait. Extensive experiments on three widely used gait datasets, Gait3D, GREW, and OUMVLP-Pose, demonstrate that our method, with minimal fine-tuning on the pretrained model, achieves state-of-the-art performance among pose-based gait recognition approaches. The dataset, code, and models are available at https://github.com/BNU-IVC/StreamGait.
Layout generation plays a crucial role in enhancing both user experience and design efficiency. However, current approaches suffer from task-specific generation capabilities and perceptually misaligned evaluation metrics, leading to limited applicability and ineffective measurement. In this paper, we propose Uni-Layout, a novel framework that achieves unified generation, human-mimicking evaluation and alignment between the two. For universal generation, we incorporate various layout tasks into a single taxonomy and develop a unified generator that handles background or element contents constrained tasks via natural language prompts. To introduce human feedback for the effective evaluation of layouts, we build Layout-HF100k, the first large-scale human feedback dataset with 100,000 expertly annotated layouts. Based on Layout-HF100k, we introduce a human-mimicking evaluator that integrates visual and geometric information, employing a Chain-of-Thought mechanism to conduct qualitative assessments alongside a confidence estimation module to yield quantitative measurements. For better alignment between the generator and the evaluator, we integrate them into a cohesive system by adopting Dynamic-Margin Preference Optimization (DMPO), which dynamically adjusts margins based on preference strength to better align with human judgments. Extensive experiments show that Uni-Layout significantly outperforms both task-specific and general-purpose methods. Our code is publicly available at https://github.com/JD-GenX/Uni-Layout.
Local motion blur in digital images originates from the relative motion between dynamic objects and static imaging systems during exposure. Existing deblurring methods face significant challenges in addressing this problem due to their inefficient allocation of computational resources and inadequate handling of spatially varying blur patterns. To overcome these limitations, we first propose a trainable mask predictor that identifies blurred regions in the image. During training, we employ blur masks to exclude sharp regions. For inference optimization, we implement structural reparameterization by converting 3 x 3 convolutions to computationally efficient 1 x 1 convolutions, enabling pixel-level pruning of sharp areas to reduce computation. Second, we develop an intra-frame motion analyzer that translates relative pixel displacements into motion trajectories, establishing adaptive guidance for region-specific blur restoration. Our method is trained end-to-end using a combination of reconstruction loss, reblur loss, and mask loss guided by annotated blur masks. Extensive experiments demonstrate superior performance over state-of-the-art methods on both local and global blur datasets while reducing FLOPs by 49% compared to SOTA models (e.g., LMD-ViT). The source code is available at https://github.com/shangwei5/M2AENet.
Recent advances in deep neural networks and generative AI have enabled the creation of highly realistic synthetic speech, raising significant concerns about the misuse of deepfake in deception and fraud. This paper addresses the generalization challenge of active defense against cross-task audio deepfake under black-box conditions. We systematically analyze the common characteristics of state-of-the-art acoustic synthesis models and propose PhonoFence, an active defense framework that introduces fine-grained phoneme-level adversarial perturbations to prevent unauthorized synthesis. PhonoFence employs a dual-domain strategy to perturb both the time domain and frequency domain via an iterative cross-training framework, leveraging complementary acoustic features to enhance the generalization of perturbations. To further improve the transferability of perturbations, we ensemble speaker encoders with a novel Multi-Middle-Layer loss. Additionally, a psychoacoustic masking algorithm is employed to enhance the perceptual quality of protected speech and conceal perturbations. Extensive experiments on leading acoustic synthesis models demonstrate that PhonoFence reduces identity similarity and word error rate to 17.91% and 49.77%, respectively, achieving relative improvements of 7.97% and 9.67% over the best existing methods. To assess the effectiveness of our method in real-world, we test PhonoFence in commercial speaker recognition system, where it reduces deepfake attack success rates by 73.49%. Moreover, PhonoFence shows strong robustness against adaptive attacks involving compression, denoising, and re-recording.
Camera-based 3D Semantic Scene Completion (SSC) is a critical task in autonomous driving systems, assessing voxel-level geometry and semantics for holistic scene perception. While existing voxel-based and plane-based SSC methods have achieved considerable progress, they struggle to capture physical regularities for realistic geometric details. On the other hand, neural reconstruction methods like NeRF and 3DGS demonstrate superior physical awareness, but suffer from high computational cost and slow convergence when handling large-scale, complex autonomous driving scenes, leading to inferior semantic accuracy. To address these issues, we propose the Semantic-PHysical Engaged REpresentation (SPHERE) for camera-based SSC, which integrates voxel and Gaussian representations for joint exploitation of semantic and physical information. First, the Semantic-guided Gaussian Initialization (SGI) module leverages dual-branch 3D scene representations to locate focal voxels as anchors to guide efficient Gaussian initialization. Then, the Physical-aware Harmonics Enhancement (PHE) module incorporates semantic spherical harmonics to model physical-aware contextual details and promote semantic-geometry consistency through focal distribution alignment, generating SSC results with realistic details. Extensive experiments and analyses on the popular SemanticKITTI and SSCBench-KITTI-360 benchmarks validate the effectiveness of SPHERE. The code is available at https://github.com/PKU-ICST-MIPL/SPHERE_ACMMM2025.
Online face recognition systems usually store face features in the server database for authentication, which are vulnerable to face reconstruction attacks. Various face privacy protection approaches have been proposed to address this issue, where transformation-based schemes are shown to be promising. However, the existing transformation-based schemes are all hand-crafted approaches which are difficult to balance the privacy protection and face recognition. In this paper, we propose to learn a set of discrepant convolutional neural networks (DCNNs) to protect the privacy of face features. We randomly split the original face features into different sub-features. Each of the DCNNs transforms an original sub-feature into a protected one. We adopt appropriate strategies to make the DCNNs as diverse as possible to improve the ability of our protected features to resist different face reconstruction attacks, where a face recognition loss and a privacy protection loss are designed for training. The former ensures that the protected feature can be matched directly using the existing face recognizers, while the latter incorporates a shadow face reconstruction model to interrupt the correlation between the protected features and the face images. Experimental results demonstrate the advantage of our method over existing schemes for face privacy protection. Our protected features can be accurately matched using existing face recognizers, which are capable of resisting both black-box and white-box face reconstruction attacks.
For camera-based image capturing, the impact of exposure or camera parameters (ISO sensitivity, shutter speed, and aperture F-number) on imaging quality is decisive. Such parameters interact in a coupled manner during the imaging process to determine the exposure quality and the degree of blur in a photograph. Naturally, decoupling such parameters from images holds significant value for applications like image quality assessment and illumination optimization. However, there has been no systematic research dedicated to this topic. In this paper, we propose a new benchmark, Cam-Bench, for estimating camera parameters on images directly. It collects an image dataset Cam-10K with various indoor scenes and accurate labels of camera parameters. Based on Cam-10K, we propose a camera parameter estimation network to decouple and regress recorded exposure information. To the best of our knowledge, Cam-Bench is the first benchmark for camera parameter estimation. Experiments demonstrate that it can enhance the performance of various downstream applications.The source code has been made publicly available at: https://github.com/pengquanhong/CamBench.
As multimedia content expands, the demand for unified multimodal retrieval (UMR) in real-world applications increases. Recent work leverages multimodal large language models to tackle this task. However, the large number of parameters leads to high training resource demands and low inference efficiency. To address this issue, we propose the PUMA: Layer-Pruned Language Model for Efficient Unified Multimodal Retrieval with Modality-Adaptive Learning, an efficient approach to enhancing the unified retrieval capabilities from both structure and learning perspectives: 1) From the perspective of model structure, we analyze and propose Layer-Pruned Self-Distillation approach. It structurally prunes the model by preserving only the shallow layers, substantially reducing the parameters of MLLM. To mitigate representational degradation, we apply self-distillation, reusing the feature from the original model as the teacher signal to guide the retrieval embedding token from the pruned model, enabling effective capacity retention in a compact model. 2) From the perspective of model learning, to mitigate representation degradation caused by rapid convergence during multimodal contrastive learning, we propose Modality-Adaptive Contrastive Learning Loss (MAC-Loss). It adaptively separates in-batch negative candidates into harder intra-modality and simpler inter-modality groups based on each query's target modality. Assigning each group a temperature coefficient with different strategies allows queries to adaptively focus on harder in-batch negatives, reducing the resource demands of multimodal contrastive learning when UMR training. Experiments demonstrate that our approach achieves double efficiency, significantly reduces resource consumption while maintaining most of the performance.
Visual perception of nighttime images is often compromised by co-existing low-light and blur degradations. While recent methods have made progress in jointly solving these degradations, the diversity of patterns and intensities in degradation has not been properly considered, leading to inconsistent illumination and unintended artifacts. In response, we propose to integrate perceptual cues with mixture-of-experts (IPCMoE) to achieve flexible processing for low-light blurry images. By exploiting the perceptual cues, we strategically combine dedicated experts with the selective collaboration approach for feature enlightening and texture restoration. To this end, we develop perceptual-integrated MoEs by designing customized routers and task-depended experts. Specifically, the texture memorial MoE is developed to preserve valuable features to restore high-fidelity details, and the enhancement MoE that adaptively integrates enlightening cues and texture cues is designed to formulate the relationship between feature enlightening and texture restoration, thereby achieving dynamic image processing. Extensive experiments show that our method achieves state-of-the-art performance on LOL-Blur and Real-LOL-Blur datasets.
Source-Free Domain Adaptive Object Detection addresses cross-domain detection on an unlabeled target domain without accessing source data. Existing methods implement self-training with Mean Teacher but are bottlenecked by error accumulation from noisy pseudo-labels generated via recursive teacher-student updates. This issue is handled through the proposed dual enhancements: (1) External Guidance via Multimodal Foundation Models (FMs); (2) Internal Regulation through Fast-Slow Teacher. First, despite FMs' multimodal comprehension, their semantic misalignment with a specific task introduces noise during adaptation. Bidirectional Distillation mitigates this by calibrating the FM using task-specific knowledge transferred from the source detector. The aligned cross-modal knowledge then propagates through high-quality pseudo-label generation. Second, the conventional Mean Teacher suffers from plasticity-stability dilemma, where rapid adaptation corrupts historical knowledge. Fast-Slow Teacher introduces dual-velocity knowledge consolidation: The Fast Teacher dynamically captures emerging domain features, while the Slow Teacher preserves stable historical knowledge and periodically resets the Fast Teacher, establishing an error-correcting dynamic equilibrium. Experiments show our method achieves significant improvements over SOTA.
Physical adversarial attacks on person detectors reveal critical vulnerabilities in safety-critical vision systems such as autonomous driving and surveillance. While recent methods enhance attack efficacy and robustness, they often neglect realistic garment deformations and motion blur, limiting real-world performance. In this work, we propose FAB-Attack (Fabric-a ware and Blur-resistant Attack), a new adversarial attack that simulates realistic garment deformation during training and targets both person detectors and image deblurring models. To enhance attack effectiveness under varying clothing deformations, we introduce a Fabric-aware Texture Appliance (FTA) module, which applies adversarial textures to clothing regions and simulates realistic fabric dynamics via physics-inspired TPS. To better emulate real-world conditions, we develop a differentiable pipeline incorporating motion blur and deblurring processes. Moreover, we demonstrate the stability of low-frequency information during motion blur's generation and removal. Based on this insight, we design a frequency band separation mechanism that suppresses high-frequency components in adversarial patterns to enhance further robustness against motion blur. Experimental results demonstrate that our approach achieves SOTA performance, reducing AP to 25.2% on the COCO dataset and achieving a 94.4% ASR in the real world under severe motion blur.
Video instance segmentation presents significant challenges in complex and dynamic environments, where instances experience progressive occlusion, either from objects obstructing each other or due to changes in the camera's viewpoint. Current state-of-the-art methods rely on memory bank mechanisms, but we still look forward to new paradigms that have the ability to capture and utilize structural information, the ability to model complex relationships, and the flexibility to adapt to dynamic scenarios. To this end, we propose the Weighted Structure Inference method for Video Instance Segmentation. We build on high-order structural relationships by constructing hypergraphs for each video frame, enabling the capture of complex interactions that go beyond traditional pairwise methods. To model intricate dynamics, we introduce Weighted Sheaf Hypergraph Convolution, which enhances the hierarchical and structural information embedded in the hypergraph. Furthermore, we ensure spatio-temporal consistency by employing a dynamic inference mechanism based on Weighted Sliced Wasserstein distance to compare structural features across adjacent frames. Our method preserves the topological characteristics of occlusion instances and improves the reliability of instance tracking across frames. Experimental results demonstrate that our method outperforms existing video instance segmentation frameworks in both Video Instance and Panoptic Segmentation tasks.
3D hand pose estimation has garnered great attention in recent years due to its critical applications in human-computer interaction, virtual reality, and related fields. Accurate estimation of hand joints is essential for high-quality hand pose estimation. However, existing methods neglect the importance of Distal Phalanx Tip (TIP) and Wrist in predicting hand joints overall and often fail to account for the phenomenon of error accumulation for distal joints in gesture estimation, which can cause certain joints to incur larger errors, resulting in misalignments and artifacts in pose estimation and degrading the overall reconstruction quality. To address this challenge, we propose a novel segmented architecture for enhanced hand pose estimation (EHPE). We perform a local extraction of the TIP and wrist, thus alleviating the effect of error accumulation on the prediction of the TIP and further reduce the predictive errors for all joints on this basis. EHPE consists of two key stages: In the TIP and Wrist Joints Extraction stage (TW-stage), the positions of the TIP and wrist joints are estimated to provide an initial accurate joint configuration; In the Prior Guided Joints Estimation stage (PG-stage), a dual-branch interaction network is employed to refine the positions of the remaining joints. Extensive experiments on two widely used benchmarks demonstrate that EHPE achieves state-of-the-art performance.
Fake document detection is an important area in image forensics. Most of the existing techniques focus on the detection of digitally forged documents. In this paper, we look into the forgery of generalized physical occlusion, which is a simple and effective strategy to generate fake document images. We propose an Adversary Decomposition Network (ADDNet) to effectively extract generalized physical occlusion features from various types of documents, where two adversarial classifiers are designed and trained for feature decomposition. On top of the ADDNet, we further propose a lightweight Document Adapter (DA) for flexible and scalable fake document detection, which works well when we encounter a new type of document with limited samples for fine-tuning. To facilitate the research, we newly construct a dataset for physical occlusion detection on different types of documents. Various experiments are carried out to demonstrate the advantage of our proposed scheme over the existing schemes for physical occlusion detection, especially when the document is unseen or has limited samples in training.
The domain of time series forecasting has gained significant attention due to its critical applications in multimedia-rich web traffic (including video streaming workloads and dynamic content delivery) and cross-platform advertisement click predictions, which are essential for web operations planning. While models like TimeSieve have demonstrated strong capabilities in predicting web visitation metrics, they suffer from critical unfaithfulness issues, including sensitivity to random seeds, input noise, layer noise, and parametric perturbations. To address these limitations, we propose Faithful TimeSieve (FTS), an enhanced framework designed to improve prediction reliability and robustness. Our approach systematically detects and mitigates unfaithfulness in TimeSieve, significantly enhancing its stability and consistency. Experimental results demonstrate that FTS substantially improves the model's faithfulness, setting a new standard for temporal forecasting methods. This advancement not only increases TimeSieve's reliability but also contributes to more robust temporal modeling, particularly crucial for web traffic forecasting where prediction accuracy directly impacts operational decisions. Our work thus represents a significant step toward more dependable time series predictions in web-related applications.
3D Gaussian Splatting (3D-GS) achieves real-time photorealistic novel view synthesis, yet struggles with complex scenes due to over-reconstruction artifacts, manifesting as local blurring and needle-shape distortions. While recent approaches attribute these issues to insufficient splitting of large-scale Gaussians, we identify two fundamental limitations: gradient magnitude dilution during densification and the primitive frozen phenomenon, where essential Gaussian densification is inhibited in complex regions while suboptimally scaled Gaussians become trapped in local optima. To address these challenges, we introduce ReAct-GS, a method founded on the principle of re-activation. Our approach features: (1) an importance-aware densification criterion incorporating α-blending weights from multiple viewpoints to re-activate stalled primitive growth in complex regions, and (2) a re-activation mechanism that revitalizes frozen primitives through adaptive parameter perturbations. Comprehensive experiments across diverse real-world datasets demonstrate that ReAct-GS effectively eliminates over-reconstruction artifacts and achieves state-of-the-art performance on standard novel view synthesis metrics while preserving intricate geometric details. Additionally, our re-activation mechanism yields consistent improvements when integrated with other 3D-GS variants such as Pixel-GS, demonstrating its broad applicability.
The proliferation of satellite-related commonsense data on the internet. Traditional analytical methods are challenging to integrate and effectively uncover implicit knowledge within them. However, current deep learning and LLM-based approaches often struggle with errors and hallucinations when performing multi-hop reasoning in domain-specific contexts. To address these limitations, we propose a novel multi-hop reasoning framework for implicit commonsense mining. This framework aims to uncover the underlying meta-knowledge behind reasoning problems, thereby providing enhanced interpretability of the reasoning process. Specifically, we design an extraction-retrieval-principle multi-step reasoning method that generates different levels of meta-knowledge in stages to support the reasoning process effectively. We further design the mixture of expert knowledge graph construction to construct a satellite knowledge graph that supports multi-hop reasoning. Experimental results demonstrate that our approach outperforms baselines on satellite knowledge graph reasoning.