Zero-shot multimodal anomaly detection is critical for identifying structural defects that are often invisible to traditional RGB images, particularly in scenarios lacking target domain samples. However, existing methodologies face two significant impediments: the prohibitive computational overhead caused by relying on multi-view rendering for depth processing, and the inability of static text prompts to adapt to fine-grained local anomalies. To address these challenges, a unified zero-shot multimodal anomaly detection framework GRASP is proposed. Firstly, a Frequency Domain Enhancement module is introduced to replace costly rendering with spectral transformations, directly synthesizing high-fidelity depth images for efficient utilization. Secondly, a Prompt-Conditioned Variational module is designed to bridge the semantic gap by grounding global textual descriptions into local visual nuances. Finally, a Dual Cross-Injection Alignment module is proposed to enable robust feature fusion for enhancing anomaly classification performance, while a Pyramid Anomaly Map Recalibration module further refines anomaly localization across multiple scales. Extensive experiments on MVTec 3D-AD and Eyecandies demonstrate that GRASP establishes a new state-of-the-art, yielding substantial improvements of 3.0 points in I-AUROC and 2.7 points in AUPRO compared to the SOTA method.
论文检索
输入标题、作者或关键词，从 960 篇学术成果中精准定位
Spatiotemporal fusion (STF) bridges the gap between temporal and spatial resolutions in satellite imagery, enabling effective monitoring of Earth's surface dynamics. However, existing methods rely on cloud-free reference images, a constraint that fails in realistic, cloud-prone scenarios. To overcome this, we propose the Cloud-Aware Wavelet Generative Adversarial Network (CLAW-GAN), a novel framework for high-fidelity reconstruction under cloud-contaminated conditions. CLAW-GAN introduces Regression Wavelet Analysis (RWA) to decouple spectral backgrounds from structural details. While a change-aware gated mechanism accounts for land-cover changes, the Frequency-Separated Fusion (FSF) module then independently integrates these components. To ensure visual realism, a multi-scale discriminator operates in the wavelet domain, enforcing consistency across high-frequency subbands to minimize artifacts. Evaluated on the newly introduced Global Cloud-shrouded Agricultural Regions (GCAR) benchmark and the simulated Daxing dataset, CLAW-GAN achieves state-of-the-art performance and demonstrates superior robustness across varying cloud coverage.
Adapting large, adversarially pre-trained models to specialized domains via transfer learning is a promising path toward building secure AI systems. However, a critical challenge arises when fine-tuning on limited downstream data: models often suffer from catastrophic forgetting of robustness, where the generalizable robust features from pre-training are erased, leading to severe robust overfitting. To address this, we first establish the underlying principle that adversarial vulnerability is not diffuse, but is concentrated in a low-dimensional subspace defined by the principal components of the model's weight matrices. We then introduce Principal component-based Implicit regularization with Low-rank Optimization (PILO), a novel parameter-efficient framework that operationalizes this insight. PILO employs a principled decoupling of learning objectives: a spectrally-guided adversarial branch, initialized via Singular Value Decomposition (SVD), performs a surgical update on the identified vulnerable subspace to enhance robustness. This is complemented by a transient, standard low-rank branch that gently adapts the model to natural data, preserving clean accuracy. This dual-branch design acts as a form of implicit spectral regularization, anchoring the model to its robust pre-trained state. Extensive experiments show that PILO significantly outperforms state-of-the-art full-parameter and parameter-efficient methods in robust accuracy across multiple benchmarks, all while reducing trainable parameters by up to 71%. Our work thus establishes a new, more effective paradigm for robust transfer learning through principled and targeted parameter optimization.
As a prominent paradigm for large-scale unsupervised learning, anchor-based multi-view clustering aims to reveal the latent structures across heterogeneous data representations with high efficiency. Despite achieving some progress, existing methods typically suffer from the following two limitations. Firstly, they rely on pre-constructed anchors or rigid constraints (e.g., orthogonality), whereas the intrinsic topological correlations among anchors are completely discarded. Secondly, most existing methods either perform clustering directly on the bipartite graph or treat different views equally, thereby failing to capture the global structural information. To this end, the Dual-tOpology learning with adapTive Anchors (DOTA) is proposed, which not only learns the sample-anchor relationship (i.e., bipartite graph) but also preserves the topology structure among anchors, significantly enhancing the discriminability of learned representation while preserving the underlying data manifold. By integrating an adaptive view-weighting strategy to balance view contributions, DOTA derives discriminative global sample embeddings by propagating spectral information through the bipartite graph. Extensive experiments on benchmark datasets demonstrate the superiority of DOTA.
Multi-view learning aims to enhance performance by integrating information from multiple sources. While different views offer complementary perspectives, extracting consistent and discriminative representations remains a significant challenge due to discrepancies in representation and presence of noise. Existing methods typically separate the denoising and alignment processes, mapping the denoised heterogeneous views to a shared subspace for alignment. This means that noise may propagate or even amplify during the alignment stage, ultimately leading to suboptimal solutions. To address this issue, we propose center-guided spectral diffusion, which replaces traditional alignment with generative modeling. This method avoids noise propagation in multi-view alignment process and prevents multi-view features be aligned to the noise subspace during fusion process. Specifically, we first performs unconditional diffusion in both the feature space and a low-rank spectral space to learn a stable consensus centre anchor. This anchor is then used to condition a guided generative diffusion process, enabling the model to generate more consistent and realistic sample representations. By combining conditional and unconditional diffusion, the proposed method alleviates the noise amplification problem commonly found in traditional alignment methods. Experimental results show that the proposed method outperforms state-of-the-art methods on several datasets.
Open-vocabulary 3D understanding aims to align 3D representations with a unified vision-language semantic space. However, existing methods suffer from the challenge of structural asymmetry caused by sparse observations and holistic geometries. Additionally, the inherent semantic chasm between discrete coordinates and abstract natural language hinders cross-modal mapping. This work introduces SpecBridge, a 3D-2D-Text pre-training framework that leverages CLIP priors as a foundational bridge to connect three modalities by synergizing spectral graph theory with transitive semantic learning. The method consists of two main sub-modules. First, Spectral Eigen-Modal Alignment is presented to construct cross-modal correlations within the intrinsic eigen-spectral space via Laplacian eigendecomposition. By aligning low-frequency geometric harmonics with CLIP-guided observation inference, it maintains structural consistency between 3D geometries and 2D projections. Second, a Transitive Spectral-Semantic Alignment method is developed to establish a 3D→2D→Text propagation chain across the CLIP priors bridge. It distills dense CLIP priors into 3D representations through transitive distillation, effectively mitigating the semantic chasm between geometry and language. Extensive experiments confirm that the proposed SpecBridge demonstrates state-of-the-art performance by overcoming challenges triggered by modality discrepancies.
Diffusion-based image compression has exhibited robust performance. However, most existing methods primarily emphasize spatial domain, causing frequency dependencies to be learned only implicitly. We revisit this problem from a frequency perspective, and observe pronounced heterogeneity between global and local frequency dependencies. Global frequency patterns describe the energy distribution over the entire image, whereas patch level frequency relations govern the coupling of local details. These two forms of dependency differ substantially in both scale and semantic level, yet most methods overlook this distinction. To address this issue, we propose a learning heterogeneous global local frequency dependencies in diffusion-based image compression which uses Fourier and Mamba jointly models both global and local frequency correlations(FMDiff). The core of FMDiff is the dual branch FMBlock. In the frequency branch, features are decomposed into magnitude and phase, then fed into Mamba separately. Its scanning mechanism captures cross frequency coupling within each patch while aggregating long range frequency context along the sequence. Magnitude phase interactive modulation is then used to explicitly restore spectral information corrupted by compression. The spatial branch provides high level semantic constraints and is fused with the frequency branch during diffusion reconstruction. Extensive experiments show that FMDiff consistently improves performance across multiple datasets.
Hyperspectral image (HSI) clustering facilitates the unsupervised discrimination of complex surface materials but traditionally relies on the idealized assumption of centralized data availability. In real-world scenarios, however, this assumption clashes with data privacy regulations and the physical distribution of data across isolated silos. While Federated Learning offers a decentralized solution, existing frameworks in remote sensing are predominantly confined to supervised paradigms and struggle to address the heavy reliance on annotations and Non-IID distributions inherent among clients. To overcome these limitations, we propose a novel framework named Robust Federated HSI Clustering(RFHC) that enables collaborative unsupervised learning without requiring raw data exchange. Specifically, we design a dual-encoder architecture that incorporates a Federated Model Weight Augmentation strategy(FMWA), which generates consistent views through local-global network interactions to mitigate the spectral distortion introduced by traditional data augmentation. Furthermore, we develop a hybrid optimization objective that synergizes prototype relationship optimization with contrastive learning. This mechanism utilizes global prototypes to guide local training, effectively stabilizing feature learning against data heterogeneity while ensuring intra-cluster compactness. Extensive experiments on three benchmark HSI datasets demonstrate the effectiveness and superiority of the proposed method against state-of-the-art model(SOTA) federated approaches.
Dynamic Graph Neural Networks (DyGNNs) are facing challenges with distribution shifts between training and test data that are similar but not identical. Existing DyGNNs for out-of-distribution scenarios primarily focus on discovering invariant patterns in the spatial domain, overlooking its impact on the structural properties in the spectral domain. In this paper, we propose a spectral-based graph augmentation framework designed to investigate and improve generalization behavior in dynamic graphs under distribution shifts. Specifically, we augment the input graph spectra into a mixture of shift components by maximizing the variance of spectral distance and propose an efficient approximation to reduce the computational cost brought by eigen-decomposition. Building on this, we develop a multi-encoder architecture in which each encoder targets a specific spectral shift component to generate referential representations. Our method adopts a new learning objective to encourage the model to rely on robust spectral properties for better generalization under distribution shifts. Extensive experiments on both real-world and synthetic datasets demonstrate that our proposed method significantly outperforms state-of-the-art approaches in node classification and link prediction tasks. Source codes are available at https://github.com/SSQiana/DSPA.
We present S³-TIR (Coupled Spatial-Spectral Splatting), a physics-inspired framework for high-fidelity thermal infrared (TIR) novel view synthesis. Existing 3D Gaussian splatting methods struggle to capture the unique duality of TIR imagery: the coexistence of diffusive smoothness from heat conduction and structural sharpness from material discontinuities. To address this issue, we leverage the intrinsic frequency properties of thermal scenes, where conduction creates smooth fields while discontinuities cause sharp signal jumps. Accordingly, S³-TIR couples adaptive spatial support with internal spectral modulation via a unified generalized-exponential Gabor primitive. This design utilizes spatial support as a physical truncation controller, confining high-frequency textures within sharp boundaries while maintaining smooth diffusion elsewhere. Furthermore, we propose a frequency-aware deferred splatting scheme that enforces multi-view frequency consistency, anchoring frequency distributions to the 3D structure rather than view-dependent appearance. Experiments on diverse benchmarks demonstrate that S³-TIR achieves state-of-the-art performance, notably reducing LPIPS by 42.5% and improving PSNR by 3.2% on the RGBT-Scenes dataset, while yielding structurally stable thermal sequences free from jittering artifacts.
DINOv3-Powered Multi-Task Foundation Model for Quantitative Remote Sensing Estimation (Student Abstract)
PDF ↗Quantitative remote sensing estimation is critical for environmental monitoring, providing continuous measures of vegetation indices, canopy height, and carbon stock. Traditional radiative-transfer models and empirical regressions require expert knowledge and generalize poorly, while deep learning methods remain task-specific. We propose SatelliteCalculator+, a DINOv3-powered multi-task foundation model for continuous regression of spectral and structural variables. The framework combines prompt-driven cross-attentive adapters with lightweight MLP decoders, enabling efficient dense prediction from frozen features. To overcome limited supervision, we synthesize over one million paired samples from SPOT 6/7 imagery using physically defined formulas. On the Open-Canopy dataset, SatelliteCalculator+ achieves competitive accuracy across eight ecological variables while reducing inference cost, demonstrating the promise of self-supervised transformers and scalable multi-task learning for large-scale Earth observation.
Modern generative and diffusion models produce highly realistic images that can mislead human perception and even sophisticated automated detection systems. Most detection methods operate in RGB space and thus analyze only three spectral channels. We propose HSI-Detect, a two-stage pipeline that reconstructs a 31-channel hyperspectral image from a standard RGB input and performs detection in the hyperspectral domain. Expanding the input representation into denser spectral bands amplifies manipulation artifacts that are often weak or invisible in the RGB domain, particularly in specific frequency bands. We evaluate HSI-Detect across FaceForensics++ dataset and show the consistent improvements over RGB-only baselines, illustrating the promise of spectral-domain mapping for Deepfake detection.
Life, Machine Learning, and the Search for Habitability: Predicting Biosignature Fluxes for the Habitable Worlds Observatory
PDF ↗Future direct-imaging flagship missions, such as NASA's Habitable Worlds Observatory (HWO), face critical decisions in prioritizing observations due to extremely stringent time and resource constraints. In this paper, we introduce two advanced machine-learning architectures tailored for predicting biosignature species fluxes from exoplanetary reflected-light spectra: a Bayesian Convolutional Neural Network (BCNN) and our novel model architecture, the Spectral Query Adaptive Transformer (SQuAT). The BCNN robustly quantifies both epistemic and aleatoric uncertainties, offering reliable predictions under diverse observational conditions, whereas SQuAT employs query-driven attention mechanisms to enhance interpretability by explicitly associating spectral features with specific biosignature species. We demonstrate that both models achieve comparably high predictive accuracy on an augmented dataset spanning a wide range of exoplanetary conditions, while highlighting their distinct advantages in uncertainty quantification and spectral interpretability. These capabilities position our methods as promising tools for accelerating target triage, optimizing observation schedules, and maximizing scientific return for upcoming flagship missions such as HWO.
Humans easily apply learned skills to different situations, a flexibility that AI systems still struggle to achieve. Current AI models are often confined to their training setup, leading to isolated developments and a narrow scope of application. This largely restricts the creation of flexible and general-purpose AI systems. Deep Model Reuse presents a novel solution. Imagine tapping into a vast library of pre-trained models, each a master in its specialized domain. Our approach re-purposes these existing models, extracting and transforming their knowledge for the development of novel AI systems. In this talk, we explore the essential techniques of this transformative process, highlighting the shift towards versatile and efficient AI that mirrors human cognition's adaptability. We introduce three foundational pillars of deep model reuse: understanding, composing, and refining. First, we investigate the internal behavior of neural networks—using language models as explainers and analyzing the representation space of diffusion models—to uncover how and what models have learned. Second, we develop methods to transform and compose models through weight mapping, knowledge distillation, and model dissection, enabling the creation of new capabilities by reassembling existing expertise. Third, we enhance reliability by editing model behaviors and mitigating biases, ensuring robustness in complex and dynamic environments. We demonstrate the power of this paradigm in generative AI, where model reuse leads to efficient diffusion models free from spectral bias, improved compositional understanding in video generation, and the repurposing of 2D/3D models for 3D/4D content creation. By shifting from training from scratch to intelligently reusing and recombining models, we move closer to adaptive, scalable, and human-like AI systems—ushering in a new era of sustainable and general intelligence.
This talk surveys my research journey toward building reliable machine learning systems that behave safely and predictably in the open world. While modern machine learning models—including foundation models (FMs)—have demonstrated unprecedented capabilities, they often suffer from reliability failures under distribution shift, leading to overconfident mispredictions, hallucinated generations, or susceptibility to adversarial prompts. My research rethinks reliability not as an afterthought, but as a first-class algorithmic principle, to be optimized alongside accuracy with minimal human supervision. The talk is organized around three key threads. To respect the allotted 20-30 minutes, the first and second parts will be briefly discussed. 1. Unknown-Aware Learning via Outlier Synthesis. I introduce a class of learning algorithms that synthesize “virtual outliers” in representation or pixel space to explicitly teach models what they don’t know. This includes the VOS, NPOS, and Dream-OOD frameworks, which shape the energy landscape around in-distribution data to avoid overconfidence on OOD. 2. Learning in the Wild with Unlabeled Data. I present theoretical insights and practical algorithms for leveraging unlabeled in-the-wild data to improve reliability. This includes SAL framework, which uses a gradient-based spectral method to separate potential outliers, and SCONE, which handles semantic and covariate shifts via constrained optimization. These results turn unlabeled data contamination into a learning signal. 3. Reliable Foundation Models. I explore reliability failures in LLMs and multimodal systems. I introduce HaloScope for hallucination detection via subspace separation on LLM representations, and TSV that performs LLM latent steering for improved hallucination detection. I will also briefly cover the LLM security and alignment, which includes VLMGuard for detecting malicious prompts in vision-language models and a data-centric paradigm for AI alignment through source-aware feedback cleaning. Throughout the talk, I highlight how representation learning, data generation, and theoretical guarantees intersect to produce scalable, label-efficient reliability methods. I will also reflect on my broader vision: designing proactive and collaborative AI systems that anticipate uncertainty and support rich human-AI interaction—especially for underrepresented communities and emerging scientific domains. This talk will be accessible to a broad AAAI audience, combining foundational algorithmic insights with real-world applications and forward-looking perspectives on the future of responsible AI.
mmJEPA-ECG: Cross-Posture Robust Contactless Electrocardiogram Monitoring via Millimeter Wave Radar Sensing
PDF ↗Continuous cardiac monitoring during sleep is vital for detecting silent arrhythmia and other nocturnal cardiac events. While electrocardiogram (ECG) is the clinical gold standard, its reliance on electrodes and physical contact makes it intrusive for daily long-term use. Millimeter-wave (mmWave) radar offers a compelling non-contact alternative by capturing cardiac-induced chest-wall micro-vibrations. Existing radar-to-ECG methods often rely on direct waveform regression, assuming posture-stable mappings that break under natural sleep movements and obscure true cardiac rhythms. Inspired by the modality-invariant perception observed in speech and vision, we introduce mmJEPA-ECG, a physiology-guided framework for reconstructing clinical ECGs by anchoring radar sensing to invariant cardiac dynamics. It addresses two fundamental challenges: (i) disentangling robust cardiac representations from posture-induced artifacts, and (ii) generalizing ECG reconstruction across individuals under signal ambiguity. To address these challenges, Physiology-Oriented Self-Supervised Pretraining builds on a Joint Embedding Predictive Architecture (JEPA) with domain-informed masking and heart rate consistency to extract posture-robust cardiac embeddings. Conditional Diffusion-based ECG Reconstruction then generates personalized ECG waveforms through a hierarchical conditional diffusion process by spectral fidelity and denoising constraints. Extensive experiments on both public and self-collected multi-subject datasets demonstrate that our method outperforms state-of-the-art across waveform and rhythm metrics, halving R-R peak errors even under posture shifts and arrhythmic conditions.
MTRL-CG: Multi-Task Reinforcement Learning Method with Spectral Clustering-Based Task Grouping
PDF ↗Multi-task reinforcement learning (RL) aims to enhance agent performance across multiple tasks by enabling effective knowledge transfer. However, these methods adopt a fully shared policy across all tasks without explicitly distinguishing between related and conflicting ones, making them suffer from negative interference issue, where updates beneficial to one task adversely affect others and lead to degraded overall performance. In this paper, we propose a multi-task reinforcement learning method with spectral clustering-based task grouping (MTRL-CG), which leverages spectral clustering to group related tasks and separate conflicting ones, enabling group-wise policy learning to mitigate negative interference. We first quantify inter-task affinity by measuring the influence of task-specific updates on others within a shared model, and construct an affinity matrix to capture these relationships. Spectral clustering is then applied to partition tasks via spectral embedding and k-means clustering. Each task group is trained with a dedicated policy network to promote focused learning. Built upon the Soft Actor-Critic (SAC) algorithm, MTRL-CG can be readily integrated into existing SAC-based multi-task RL methods. Extensive experiments on the Meta-World benchmark demonstrate the effectiveness of the proposed MTRL-CG method.
Large Language Models (LLMs) are widely adopted, but their high training cost leads many developers to fine-tune existing open-source models. While most adhere to open-source licenses, some falsely claim original training despite clear derivation from public models, raising pressing concerns about intellectual property protection and the need to verify model provenance. In this paper, we propose GhostSpec, a lightweight yet effective method for verifying LLM lineage without access to training data or modification of model behavior. Our approach constructs compact and robust fingerprints by applying singular value decomposition (SVD) to invariant products of internal attention weight matrices. Unlike watermarking or output-based methods, GhostSpec is fully data-free, non-invasive, and computationally efficient. Extensive experiments show it is robust to fine-tuning, pruning, expansion, and adversarial transformations, reliably tracing lineage with minimal overhead. By offering a practical solution for model verification, our method contributes to intellectual property protection and fosters a transparent, trustworthy LLM ecosystem.
SpecDetect: Simple, Fast, and Training-Free Detection of LLM-Generated Text via Spectral Analysis
PDF ↗The proliferation of high-quality text from Large Language Models (LLMs) demands reliable and efficient detection methods. While existing training-free approaches show promise, they often rely on surface-level statistics and overlook fundamental signal properties of the text generation process. In this work, we reframe detection as a signal processing problem, introducing a novel paradigm that analyzes the sequence of token log-probabilities in the frequency domain. By systematically analyzing the signal's spectral properties using the global Discrete Fourier Transform (DFT) and the local Short-Time Fourier Transform (STFT), we find that human-written text consistently exhibits significantly higher spectral energy. This higher energy reflects the larger-amplitude fluctuations inherent in human writing compared to the suppressed dynamics of LLM-generated text. Based on this key insight, we construct SpecDetect, a detector built on a single, robust feature from the global DFT: DFT total energy. We also propose an enhanced version, SpecDetect++, which incorporates a sampling discrepancy mechanism to further boost robustness. Extensive experiments show that our approach outperforms the state-of-the-art model while running in nearly half the time. Our work introduces a new, efficient, and interpretable pathway for LLM-generated text detection, showing that classical signal processing techniques offer a surprisingly powerful solution to this modern challenge.
MGT-Prism: Enhancing Domain Generalization for Machine-Generated Text Detection via Spectral Alignment
PDF ↗Large Language Models have shown growing ability to generate fluent and coherent texts that are highly similar to the writing style of humans. Current detectors for Machine-Generated Text (MGT) perform well when they are trained and tested in the same domain but generalize poorly to unseen domains, due to domain shift between data from different sources. In this work, we propose MGT-Prism , an MGT detection method from the perspective of the frequency domain for better domain generalization. Our key insight stems from analyzing text representations in the frequency domain, where we observe consistent spectral patterns across diverse domains, while significant discrepancies in magnitude emerge between MGT and human-written texts (HWTs). The observation initiates the design of a low frequency domain filtering module for filtering out the document-level features that are sensitive to domain shift, and a dynamic spectrum alignment strategy to extract the task-specific and domain-invariant features for improving the detector's performance in domain generalization. Extensive experiments demonstrate that MGT-Prism outperforms state‑of‑the‑art baselines by an average of 0.90% in accuracy and 0.92% in F1 score on 11 test datasets across three domain‑generalization scenarios.