The Federated Domain Generalization for Person re-identification (FedDG-ReID) aims to learn a global server model that can be effectively generalized to source and target domains through distributed source domain data. Existing methods mainly improve the diversity of samples through style transformation, which to some extent enhances the generalization performance of the model. However, we discover that not all styles contribute to the generalization performance. Therefore, we define styles that are beneficial/harmful to the model's generalization performance as positive/negative styles. Based on this, new issues arise: How to effectively screen and continuously utilize the positive styles. To solve these problems, we propose a Style Screening and Continuous Utilization (SSCU) framework. Firstly, we design a Generalization Gain-guided Dynamic Style Memory (GGDSM) for each client model to screen and accumulate generated positive styles. Specifically, the memory maintains a prototype initialized from raw data for each category, then screens positive styles that enhance the global model during training, and updates these positive styles into the memory using a momentum-based approach. Meanwhile, we propose a style memory recognition loss to fully leverage the positive styles memorized by GGDSM. Furthermore, we propose a Collaborative Style Training (CST) strategy to make full use of positive styles. Unlike traditional learning strategies, our approach leverages both newly generated styles and the accumulated positive styles stored in memory to train client models on two distinct branches. This training strategy is designed to effectively promote the rapid acquisition of new styles by the client models, ensuring that they can quickly adapt to and integrate novel stylistic variations. Simultaneously, this strategy guarantees the continuous and thorough utilization of positive styles, which is highly beneficial for the model's generalization performance. Extensive experimental results demonstrate that our method outperforms existing methods in both the source domain and the target domain.
论文检索
输入标题、作者或关键词，从 1,620 篇学术成果中精准定位
3D point cloud segmentation aims to assign semantic labels to individual points in a scene for fine-grained spatial understanding. Existing methods typically adopt data augmentation to alleviate the burden of large-scale annotation. However, most augmentation strategies only focus on local transformations or semantic recomposition, lacking the consideration of global structural dependencies within scenes. To address this limitation, we propose a graph-guided data augmentation framework with dual-level constraints for realistic 3D scene synthesis. Our method learns object relationship statistics from real-world data to construct guiding graphs for scene generation. Local-level constraints enforce geometric plausibility and semantic consistency between objects, while global-level constraints maintain the topological structure of the scene by aligning the generated layout with the guiding graph. Extensive experiments on indoor and outdoor datasets demonstrate that our framework generates diverse and high-quality augmented scenes, leading to consistent improvements in point cloud segmentation performance across various models. Code is available at: https://github.com/alexander7xu/DualLevelAug
As a form of multimedia creation, visual novel (VN) conveys engaging narratives through the integrated presentation of text, images, and music, and has shown promise across various application domains. Recent advances in generative AI have fueled interest in automating VN creation using LLMs and other foundation models. However, fully end-to-end VN creation (i.e., from user description to executable VN) remains underexplored and presents several key challenges: 1) the hallucination and limited capacity of LLMs hinder the generation of long and coherent plots; 2) current models lack effective mechanisms for ensuring cross-modal consistency between plot, visual, and audio elements. To address these issues, we propose a hierarchical end-to-end framework for automatic VN generation and assembly, which employs an outline-guided autoregressive generation mechanism that transforms high-level user prompts into coherent plots, while a vision LLM-based self-correction mechanism ensures consistency between multimedia assets and plot content. Additionally, we introduce a script validation mechanism to ensure the executable of the final VN application. Experiments demonstrate that our framework generates high-quality VN applications with coherent storylines and consistent multimedia content.
Early screening for Alzheimer's Disease (AD) through speech presents a promising non-invasive approach. However, challenges such as limited data and the lack of fine-grained, adaptive feature selection often hinder performance. To address these issues, we propose MoTAS, a robust framework designed to enhance AD screening efficiency. MoTAS leverages Text-to-Speech (TTS) augmentation to increase data volume and employs a Mixture of Experts (MoE) mechanism to improve multimodal feature selection, jointly enhancing model generalization. The process begins with automatic speech recognition (ASR) to obtain accurate transcriptions. TTS is then used to synthesize speech that enriches the dataset. After extracting acoustic and text embeddings, the MoE mechanism dynamically selects the most informative features, optimizing feature fusion for improved classification. Evaluated on the ADReSSo dataset, MoTAS achieves a leading accuracy of 85.71%, outperforming existing baselines. Ablation studies further validate the individual contributions of TTS augmentation and MoE in boosting classification performance. These findings highlight the practical value of MoTAS in real-world AD screening scenarios, particularly in data-limited settings.
Facial image steganography is crucial for privacy-preserving media transmission. Traditional embedding methods degrade image quality and are vulnerable to steganalysis, while GAN-based non-embedding approaches lack controllability and realism. Diffusion-based methods using textual prompts face two key issues: (1) security risks from interpretable prompts and (2) poor preservation of facial details. This paper presents Featurized Denoising Diffusion Implicit Models (F-DDIM), a novel non-embedding steganography framework. First, F-DDIM replaces explicit textual prompts with implicit image-based encoding, enhancing security. Second, it selectively refines facial regions for natural and high-quality recovery through iterative reconstruction. Third, it enables indistinguishable encryption without secret key sharing via a novel sub-code embedding algorithm. Fourth, a refinement step post-decoding improves the clarity and accuracy of recovered facial image details. Experimental results demonstrate that F-DDIM achieves superior image fidelity and robustness against transmission interference.
Recent advancements in Generalizable Gaussian Splatting have enabled robust 3D reconstruction from sparse input views by utilizing feed-forward Gaussian Splatting models, achieving superior cross-scene generalization. However, while many methods focus on geometric consistency, they often neglect the potential of text-driven guidance to enhance semantic understanding, which is crucial for accurately reconstructing fine-grained details in complex scenes. To address this limitation, we propose TextSplat-the first text-driven Generalizable Gaussian Splatting framework. Specifically, our framework employs three parallel modules to obtain complementary representations: the Diffusion Prior Depth Estimator for accurate depth information, the Semantic Aware Segmentation Network for detailed semantic information, and the Multi-View Interaction Network for refined cross-view features. Then, in the Text-Guided Semantic Fusion Module, these representations are integrated via the text-guided and attention-based feature aggregation mechanism, resulting in enhanced 3D Gaussian parameters enriched with detailed semantic cues. Experimental results on various benchmark datasets demonstrate improved performance compared to existing methods across multiple evaluation metrics, validating the effectiveness of our framework. The code will be publicly available.
Forged videos are often subjected to double compression. When a forger maliciously or unintentionally increases the video's bitrate during re-encoding, the resulting videos are termed fake bitrate videos. Detecting these videos offers a generalized approach for efficiently identifying potentially forged content within large datasets. However, previous research has largely focused on video-level detection of fully fake bitrate videos, where an entire video is re-encoded at a higher bitrate after content modification or the creation of fake high-definition (HD) footage. In practice, a skilled forger may adjust the bitrate of only specific video segments, generating partial fake bitrate videos-a common manipulation in tampering processes like video splicing. Existing methods face difficulties in detecting such partial modifications at the frame level and in pinpointing the manipulated segments. Our study addresses this gap by introducing a novel frame-level detection approach, which significantly enhances forensic precision. We simultaneously account for two types of abnormal frames arising from re-encoding and bitrate escalation and, for the first time, define fake bitrate video detection as a triple classification problem. To meet the challenges of this task, we extract anomalous bitrate-compression traces that capture subtle differences among the three frame types. Additionally, we propose the Trident Transformer Network (TTNet), a model designed to effectively integrate and learn high-frequency information within the encoding domain. Our approach achieves substantial improvements in accuracy, surpassing state-of-the-art methods by 3.62% and 11.95% in video-level and frame-level detection scenarios, respectively.
Multimodal Sentiment Analysis (MSA) aims to integrate textual, audio, and visual data to capture nuanced sentimental cues. Although text dominates in existing approaches, audio and visual modalities inherently contain both shared semantics (overlapping with text) and private semantics. Existing methods struggle to precisely find semantic boundaries and lack explicit mechanisms for modeling interaction between shared/private semantics and different modalities. To address this, we propose DiffuFuse, a framework that uses a diffusion denoising model to leverage textual information to predict shared semantic features, dynamically and adaptively delineate semantic boundaries for non-textual features, and employs a dual-stream fusion strategy to accurately model the interactions between different modalities and semantic types. Finally, adopt an orthogonal projection method to reduce redundancy and eliminate overlapping information between the two streams. DiffuFuse is evaluated on the MOSI and MOSEI datasets, and the experimental results demonstrate that our proposed DiffuFuse achieves superior performance.
Transformer-based networks have achieved strong performance in low-level vision tasks like image deraining by utilizing spatial or channel-wise self-attention. However, irregular rain patterns and complex geometric overlaps challenge single-paradigm architectures, necessitating a unified framework to integrate complementary global-local and spatial-channel representations. To address this, we propose a novel Cross Paradigm Representation and Alignment Transformer (CPRAformer). Its core idea is the hierarchical representation and alignment, leveraging the strengths of both paradigms (spatial-channel and global-local) to aid image reconstruction. It bridges the gap within and between paradigms, aligning and coordinating them to enable deep interaction and fusion of features. Specifically, we use two types of self-attention in the Transformer blocks: sparse prompt channel self-attention (SPC-SA) and spatial pixel refinement self-attention (SPR-SA). SPC-SA enhances global channel dependencies through dynamic sparsity, while SPR-SA focuses on spatial rain distribution and fine-grained texture recovery. To address the feature misalignment and knowledge differences between them, we introduce the Adaptive Alignment Frequency Module (AAFM), which aligns and interacts with features in a two-stage progressive manner, enabling adaptive guidance and complementarity. This reduces the information gap within and between paradigms. Through this unified cross-paradigm dynamic interaction framework, we achieve the extraction of the most valuable interactive fusion information from the two paradigms. Extensive experiments demonstrate that our model achieves state-of-the-art performance on eight benchmark datasets and further validates CPRAformer's robustness in other image restoration tasks and downstream applications.
Dynamic fluid scene reconstruction remains challenging in multimedia applications and digital content creation due to complex motions and changing topology. While Neural Radiance Fields (NeRF) methods are computationally expensive and 3D Gaussian Splatting (3DGS) approaches struggle with fluid phenomena, we propose Fluid-GS, a flexible, efficient end-to-end framework for sparse-view fluid reconstruction that tightly couples density field modeling with velocity estimation via differentiable advection. Our key innovation is a hybrid Lagrangian-Eulerian Gaussian primitive representation that combines the rendering efficiency of 3DGS with physically-accurate fluid motion tracking on Eulerian grid, that enables us to formulate physics-informed constraints derived from Navier-Stokes equations, enforcing temporal coherence and fluid incompressibility. Moreover, to address the inherent challenges of sparse-view reconstruction, we introduce a fluid-specific Gaussian kernel constraint that preserves the spatial characteristics of fluid phenomena, and dynamically adjusts the anisotropic kernel of Gaussian primitives based on local velocity fields, preventing non-physical artifacts. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods in both reconstruction quality and computational efficiency.
Point cloud completion is crucial for downstream tasks in 3D visual perception. However, existing methods often struggle to generalize to real-world scans due to their heavy reliance on abundant paired point clouds for training and their neglect of the distribution shift between training and testing datasets. To address these limitations, this paper explores a practical and challenging setting: ''source-free domain adaptive point cloud completion'', where a well-trained source model must adapt to the target data distribution without access to source data, aiming to improve completion performance. To tackle this problem, we propose a novel method called ''Dual-Stage Preservation and Fusion'' (DSPF), which comprises two key training stages tailored to this new setting. In the source preservation stage, we introduce graph structural alignment and marginal feature alignment to preserve and transfer essential knowledge from the source domain. In the target fusion stage, we design a self-supervised loss to capture the geometric structure of target instances and establish a bidirectional interaction mechanism to transfer partial source knowledge to the target distribution. Extensive experiments on various cross-domain point cloud completion benchmarks demonstrate that our proposed DSPF significantly outperforms existing methods, validating its effectiveness and robustness in source-free domain adaptation scenarios. Our code is available at https://github.com/ZhiXia-SEU/DSPF.
We present NIVM, a lightweight and efficient view morphing framework that learns coordinate transforms between views, enabling real-time, user-controlled perspective shifts on resource-constrained devices. Unlike existing view interpolation methods that compromise visual quality or require high data overhead, NIVM integrates seamlessly into multi-view video streams as compact metadata per frame, enabling the synthesis of high-quality intermediate views and interactive transitions from sparse viewpoints. To avoid dependence on explicit 3D geometry, which may be unavailable, we introduce a dual-branch training strategy: a teacher network operates in rectified stereo space to supervise the morpher in the original image domain. By inheriting the monotonicity constraints of epipolar geometry, our morphing network produces visually plausible pixel flows while avoiding the reprojection artifacts prevalent in depth-based methods. Compared to recent pose-free sparse-view Gaussian Splatting approaches, NIVM achieves competitive results without the need to construct or transmit volumetric representations. Experiments show that NIVM achieves the lowest memory footprint, highest inference efficiency, and top-tier visual quality across multiple benchmark datasets.
Advancements in Generative AI offers new opportunities for FashionAI, surpassing traditional recommendation systems that often lack transparency and struggle to integrate expert knowledge, leaving the potential for personalized fashion styling remain untapped. To address these challenges, we present PAFA (Principle-Aware Fashion), a multi-granular knowledge base that organizes professional styling expertise into three levels of metadata, domain principles, and semantic relationships. Using PAFA, we develop StePO-Rec, a knowledge-guided method for multi-step outfit recommendation. StePO-Rec provides structured suggestions using a scenario-dimension-attribute framework, employing recursive tree construction to align recommendations with both professional principles and individual preferences. A preference-trend re-ranking system further adapts to fashion trends while maintaining the consistency of the user's original style. Experiments on the widely used personalized outfit dataset IQON show a 28% increase in Recall@1 and 32.8% in MAP. Furthermore, case studies highlight improved explainability, traceability, result reliability, and the seamless integration of expertise and personalization.
Model quantization is a promising method for accelerating and compressing diffusion models. Nevertheless, since post-training quantization (PTQ) fails catastrophically at low-bit cases, quantization-aware training (QAT) is essential. Unfortunately, the wide range and time-varying activations in diffusion models sharply increase the complexity of quantization, making existing QAT methods inefficient. Equivalent scaling can effectively reduce activation range, but previous methods remain the overall quantization error unchanged. More critically, these methods significantly disrupt the original weight distribution, resulting in poor weight initialization and challenging convergence during QAT training. In this paper, we propose a novel QAT framework for diffusion models, called DilateQuant. Specifically, we propose Weight Dilation (WD) that maximally dilates the unsaturated in-channel weights to a constrained range through equivalent scaling. WD decreases the activation range while preserving the original weight range, which steadily reduces the quantization error and ensures model convergence. To further enhance accuracy and efficiency, we design a Temporal Parallel Quantizer (TPQ) to address the time-varying activations and introduce a Block-wise Knowledge Distillation (BKD) to reduce resource consumption in training. Extensive experiments demonstrate that DilateQuant significantly outperforms existing methods in terms of accuracy and efficiency.
Dynamic Residual Encoding with Slide-Level Contrastive Learning for End-to-End Whole Slide Image Representation
PDF ↗Whole Slide Image (WSI) representation is critical for cancer subtyping, cancer recognition and mutation prediction.Training an end-to-end WSI representation model poses significant challenges, as a standard gigapixel slide can contain tens of thousands of image tiles, making it difficult to compute gradients of all tiles in a single mini-batch due to current GPU limitations. To address this challenge, we propose a method of dynamic residual encoding with slide-level contrastive learning (DRE-SLCL) for end-to-end WSI representation. Our approach utilizes a memory bank to store the features of tiles across all WSIs in the dataset. During training, a mini-batch usually contains multiple WSIs. For each WSI in the batch, a subset of tiles is randomly sampled and their features are computed using a tile encoder. Then, additional tile features from the same WSI are selected from the memory bank. The representation of each individual WSI is generated using a residual encoding technique that incorporates both the sampled features and those retrieved from the memory bank. Finally, the slide-level contrastive loss is computed based on the representations and histopathology reports ofthe WSIs within the mini-batch. Experiments conducted over cancer subtyping, cancer recognition, and mutation prediction tasks proved the effectiveness of the proposed DRE-SLCL method.
Salient object detection (SOD) plays a crucial role in image understanding and visual guidance. However, due to the complexity of underwater environments, the accuracy of underwater salient object detection is often low. To improve the accuracy and robustness of underwater salient object detection, different from the existing spatial domain aware RGB-D methods that rely on pixel-level probabilities, we propose a novel Fourier-Spatial Entangled Conditional Diffusion model (FSCDiff) for underwater salient object detection. The FSCDiff aims to address the insufficient representation and boundary shift issues in underwater salient object detection by leveraging Fourier-domain information and the powerful multi-step iterative generation capability of diffusion models. The FSCDiff framework consists of two key components: the Dual-Domain Entanglement Enhancement Block (DTEB) and the Stable Time-step Mask Prediction Module (STMP). DTEB utilizes Fourier-spatial entanglement learning to fully exploit the Fourier and spatial domain information of RGB images and depth maps, thereby optimizing feature representation. STMP takes advantage of the excellent multi-step iterative mechanism of diffusion models to enhance the accuracy and robustness of the segmentation results. Comprehensive experimental results indicate that our FSCDiff method outperforms the state-of-the-art approaches on the USOD10K and USOD datasets. The source code is available at: https://github.com/lgwplay/FSCDiff.
Due to the high cost of collection and labeling, there are relatively few datasets for camouflaged object detection (COD). In particular, for certain specialized categories, the available image dataset is insufficiently populated. Synthetic datasets can be utilized to alleviate the problem of limited data to some extent. However, directly training with synthetic datasets compared to real datasets can lead to a degradation in model performance. To tackle this problem, in this work, we investigate a new task, namely Syn-to-Real Camouflaged Object Detection (S2R-COD). In order to improve the model performance in real world scenarios, a set of annotated synthetic camouflaged images and a limited number of unannotated real images must be utilized. We propose the Cycling Syn-to-Real Domain Adaptation Framework (CSRDA), a method based on the student-teacher model. Specially, CSRDA propagates class information from the labeled source domain to the unlabeled target domain through pseudo labeling combined with consistency regularization. Considering that narrowing the intra-domain gap can improve the quality of pseudo labeling, CSRDA utilizes a recurrent learning framework to build an evolving real domain for bridging the source and target domain. Extensive experiments demonstrate the effectiveness of our framework, mitigating the problem of limited data and handcraft annotations in COD. Our code is publicly available at https://github.com/Muscape/S2R-COD.
In recent years, adversarial attacks on video recognition models have attracted increasing attention. However, most existing strategies are extensions of image-based methods, where adversarial perturbations are computed independently and embedded into individual frames. This independent per-frame perturbation process wastes computational resources and leads to excessive query consumption. To address this problem, we introduce Frequency Domain Distributed Perturbations (FDP), a straightforward yet effective black-box video attack method using temporal correlations between video frames. Specifically, FDP first converts the input video into the frequency domain and calculates globally coordinated adversarial perturbations in the spectral space. By conducting global optimization in the frequency domain, FDP improves the effectiveness of each query, significantly decreasing the total number of queries needed. The resulting perturbations are temporally distributed across frames to preserve the spatiotemporal structure. Furthermore, we introduce a frequency-sensitive mask to identify the spectral regions most critical to the model's predictions. By applying perturbations only to these key frequency bands, FDP further reduces the perturbation search space and improves query efficiency. Extensive experiments demonstrate that our method significantly reduces query consumption while achieving higher attack success rates than state-of-the-art approaches.
3D Gaussian Splatting has exhibited remarkable capabilities in 3D scene reconstruction. However, reconstructing high-quality 3D scenes from motion-blurred images caused by camera motion poses a significant challenge. The performance of existing 3DGS-based deblurring methods are limited due to their inherent mechanisms, such as extreme dependence on the accuracy of camera poses and inability to effectively control erroneous Gaussian primitives densification caused by motion blur. To solve these problems, we introduce a novel framework, Bi-Stage 3D Gaussian Splatting, to accurately reconstruct 3D scenes from motion-blurred images. BSGS contains two stages. First, Camera Pose Refinement roughly optimizes camera poses to reduce motion-induced distortions. Second, with fixed rough camera poses, Global Rigid Transformation further corrects motion-induced blur distortions. To alleviate multi-subframe gradient conflicts, we propose a subframe gradient aggregation strategy to optimize both stages. Furthermore, a space-time bi-stage optimization strategy is introduced to dynamically adjust primitive densification thresholds and prevent premature noisy Gaussian generation in blurred regions. Comprehensive experiments verify the effectiveness of our proposed deblurring method and show its superiority over the state of the arts.
3D Gaussian Splatting (3DGS) data compression is crucial for enabling efficient storage and transmission in 3D scene modeling. However, its development remains limited due to inadequate entropy models and suboptimal quantization strategies for both lossless and lossy compression scenarios, where existing methods have yet to 1) fully leverage hyperprior information to construct robust conditional entropy models, and 2) apply fine-grained, element-wise quantization strategies for improved compression granularity. In this work, we propose a novel Mixture of Priors (MoP) strategy to simultaneously address these two challenges. Specifically, inspired by the Mixture-of-Experts (MoE) paradigm, our MoP approach processes hyperprior information through multiple lightweight MLPs to generate diverse prior features, which are subsequently integrated into the MoP feature via a gating mechanism. To enhance lossless compression, the resulting MoP feature is utilized as a hyperprior to improve conditional entropy modeling. Meanwhile, for lossy compression, we employ the MoP feature as guidance information in an element-wise quantization procedure, leveraging a prior-guided Coarse-to-Fine Quantization (C2FQ) strategy with a predefined quantization step value. Specifically, we expand the quantization step value into a matrix and adaptively refine it from coarse to fine granularity, guided by the MoP feature, thereby obtaining a quantization step matrix that facilitates element-wise quantization. Extensive experiments demonstrate that our proposed 3DGS data compression framework achieves state-of-the-art performance across multiple benchmarks, including Mip-NeRF360, BungeeNeRF, DeepBlending, and Tank&Temples.