Abstract:We propose a novel approach to self-supervised learning of point cloud representations by differentiable neural rendering. Motivated by the fact that informative point cloud features should be able to encode rich geometry and appearance cues and render realistic images, we train a point-cloud encoder within a devised point-based neural renderer by comparing the rendered images with real images on massive RGB-D data. The learned point-cloud encoder can be easily integrated into various downstream tasks, including not only high-level tasks like 3D detection and segmentation, but low-level tasks like 3D reconstruction and image synthesis. Extensive experiments on various tasks demonstrate the superiority of our approach compared to existing pre-training methods.
| Comments: | Project page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2301.00157 [cs.CV] |
| (or arXiv:2301.00157v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2301.00157 arXiv-issued DOI via DataCite |
Submission history
From: Di Huang [view email]
[v1]
Sat, 31 Dec 2022 08:58:39 UTC (18,573 KB)
[v2]
Thu, 26 Oct 2023 15:56:50 UTC (18,455 KB)