RWKV3D: An RWKV-Based Model with Multiple Training Strategies for Point Cloud Analysis

摘要

Transformer-based models have achieved dominance in point cloud analysis, yet their quadratic computational complexity remains a fundamental limitation for practical applications. Recently, RWKV has emerged as a promising alternative for sequence modeling due to its linear computational complexity. However, it has yet to be effectively adapted to handle the unordered and sparse nature of point cloud data. In this paper, we propose RWKV3D, an innovative and computational framework tailored for point cloud analysis, which is adaptable to three training strategies: training from scratch, single-modal pre-training, and cross-modal pre-training. First, we replace the MLP layer with an advanced Local Feature Mixer (LFM), which not only enhances fine-grained feature extraction but also reduces the number of parameters. Second, we introduce a Bidirectional Multi-head Shift (BMS) mechanism to expand the receptive field, effectively capturing richer contextual information. Additionally, to enhance high-level feature processing, we strategically incorporate a Multi-head Self-Attention (MSA) block before the first RWKV3D block. Experimental results demonstrate that RWKV3D outperforms Transformer-based and Mamba-based methods while maintaining lower parameter counts and computational costs. Notably, it achieves several state-of-the-art results, including overall accuracies of 95.3% (training from scratch) and 95.9% (cross-modal pre-training) on the ModelNet40 dataset, as well as 95.28% (single-modal pre-training) on the ScanObjectNN (PB_T50_RS) dataset. These results underscore the superior efficacy of the RWKV architecture in 3D vision tasks and highlight its potential for broader multimodal learning scenarios.