Abstract:Neural radiance fields (NeRF) achieve highly photo-realistic novel-view synthesis, but it's a challenging problem to edit the scenes modeled by NeRF-based methods, especially for dynamic scenes. We propose editable neural radiance fields that enable end-users to easily edit dynamic scenes and even support topological changes. Input with an image sequence from a single camera, our network is trained fully automatically and models topologically varying dynamics using our picked-out surface key points. Then end-users can edit the scene by easily dragging the key points to desired new positions. To achieve this, we propose a scene analysis method to detect and initialize key points by considering the dynamics in the scene, and a weighted key points strategy to model topologically varying dynamics by joint key points and weights optimization. Our method supports intuitive multi-dimensional (up to 3D) editing and can generate novel scenes that are unseen in the input sequence. Experiments demonstrate that our method achieves high-quality editing on various dynamic scenes and outperforms the state-of-the-art. Our code and captured data are available at this https URL.
| Comments: | Accepted by CVPR 2023 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2212.04247 [cs.CV] |
| (or arXiv:2212.04247v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2212.04247 arXiv-issued DOI via DataCite |
Submission history
From: Chengwei Zheng [view email]
[v1]
Wed, 7 Dec 2022 06:08:03 UTC (5,506 KB)
[v2]
Tue, 28 Mar 2023 05:14:33 UTC (5,753 KB)