Reference grid-assisted network for 3D point signature learning from point clouds

Learning a robust 3D point signature from point clouds is an interesting but challenging task in the computer vision field due to the irregular and unordered structure characteristics of the point cloud data. In this paper, we propose to learn a 3D point signature by exploring the implicit relation between keypoints and their neighbors (grouped as patches) among the given scene point clouds. We design a uniform reference grid to represent the raw relation between each keypoint and its neighbors from the raw point clouds. In order to learn a 3D point signature gradually from expanding perceptive region, we create a novel siamese framework with a multi-layer perceptron (MLP)-based unit feature network and a 3D convolutional neural network (CNN)-based grid feature network. Specifically, the unit feature network aims to dig the connections among points fallen into the same unit of the reference grid, while the grid feature network is used to discover the grid-wise relations across the whole reference grid with concatenation of the learned unit- wise features. Moreover, we introduce an attention network upon the unit feature network to enhance the discriminative ability of our learned 3D point signature. Our proposed 3D point signature achieves superior performance over other state-of-the-art methods on keypoint matching and geometric registration on the real-world scenes datasets, e.g. SUN3D, 7-scenes and the synthetic scan augmented scenes in ICL-NUIM dataset. More importantly, our learned 3D point signature successfully handles the point cloud fragment alignment challenges by producing correct transformations with RANSAC algorithm.