Rotation Invariant Spatial Networks for Single-View Point Cloud Classification

Feng Luan, Jiarui Hu, Changshi Zhou, Zhipeng Wang, Jiguang Yue, Yanmin Zhou, Bin He

Shanghai Research Institute for Intelligent Autonomous Systems National Key Laboratory of Autonomous Intelligent Unmanned Systems, Tongji University Frontiers Science Center for Intelligent Autonomous Systems College of Electronics and Information Engineering, Tongji University

Code

Abstract: Point cloud classification is critical for three-dimensional scene understanding. However, in real-world scenarios, depth cameras often capture partial, single-view point clouds of objects with different poses, making their accurate classification a challenge. In this paper, we propose a novel point cloud classification network that captures the detailed spatial structure of objects by constructing tetrahedra, which is different from point-wise operations. Specifically, we propose a RISpaNet block to extract rotation-invariant features. A rotation-invariant property generation module is designed in RISpaNet for constructing rotation-invariant tetrahedron properties (RITPs). Meanwhile, a multi-scale pooling module and a hybrid encoder are used to process RITPs to generate integrated rotation-invariant features. Further, for single-view point clouds, a complete point cloud auxiliary branch and a part-whole correlation module are jointly employed to obtain complete point cloud features from partial point clouds. Experimental results show that this network performs better than other state-of-the-art methods, evaluated on four public datasets. We achieved an overall accuracy of 94.7\% (+2.0\%) on ModelNet40, 93.4\% (+5.9\%) on MVP, 94.7\% (+6.3\%) on PCN and 94.8\% (+1.7\%) on ScanObjectNN.

Fig. 1: The process of constructing rotation-invariant tetrahedron properties.

Fig. 2: RISpaNet block structure. RISpaNet consists of three modules: the rotation-invariant property generation module (RIPGM), the multi-scale pooling module (MSPM) and the hybrid encoder. The RIPGM is used to generate the rotational-invariant tetrahedron properties (RITP). The multi-scale pooling module is applied to process the RITP to obtain multi-scale features. The hybrid encoder is designed to extract the integrated rotational-invariant features.

Fig. 3: Two-branch structure. The proposed network contains two branches: the partial point cloud branch (PPCB) and the complete point cloud auxiliary branch (CPCAB). PPCB is used to extract the rotation-invariant features of single-view point clouds. CPCAB is used to extract the rotation-invariant features of complete point clouds. A part-whole correlation module (PWCM) is used to construct the link between the complete and partial features to get the predicted complete point cloud features. Decoder is used to output the classification results.

TABLE. 1: FLOPS analysis

Contributions:

*Rotation-invariant Tetrahedron Properties. A highly expressive rotation-invariant property is designed by constructing local tetrahedra to capture spatial information of objects.

*RISpaNet. RISpaNet is a fundamental block in our model. This block includes a rotation-invariant property generation module, a multi-scale pooling module and a hybrid encoder. Different coding mechanisms are embedded in the hybrid encoder to extract the integrated rotation-invariant features from RITPs.

*Two-branch design. The complete point cloud auxiliary branch is designed to acquire complete point cloud features during the training phase. The part-whole correlation module is used to achieve the mapping from missing point cloud features to complete point cloud features. The two-branch structure can improve the single-view point cloud classification accuracy.