neural-3d-videos
Papers with tag neural-3d-videos
2022
- Streaming Radiance Fields for 3D Video SynthesisLingzhi Li, Zhen Shen, Zhongshu Wang, Li Shen, and Ping TanIn 2022
We present an explicit-grid based method for efficiently reconstructingstreaming radiance fields for novel view synthesis of real world dynamicscenes. Instead of training a single model that combines all the frames, weformulate the dynamic modeling problem with an incremental learning paradigm inwhich per-frame model difference is trained to complement the adaption of abase model on the current frame. By exploiting the simple yet effective tuningstrategy with narrow bands, the proposed method realizes a feasible frameworkfor handling video sequences on-the-fly with high training efficiency. Thestorage overhead induced by using explicit grid representations can besignificantly reduced through the use of model difference based compression. Wealso introduce an efficient strategy to further accelerate model optimizationfor each frame. Experiments on challenging video sequences demonstrate that ourapproach is capable of achieving a training speed of 15 seconds per-frame withcompetitive rendering quality, which attains 1000 \times speedup over thestate-of-the-art implicit methods. Code is available athttps://github.com/AlgoHunt/StreamRF.
提升训练速度. 提出一个explicit-grid based method, 将3d videos表示为a based model and per-frame model difference.
neural-3d-videos@inproceedings{Streaming_RF, title = {Streaming Radiance Fields for 3D Video Synthesis}, author = {Li, Lingzhi and Shen, Zhen and Wang, Zhongshu and Shen, Li and Tan, Ping}, year = {2022}, tags = {neural-3d-videos}, sida = {提升训练速度. 提出一个explicit-grid based method, 将3d videos表示为a based model and per-frame model difference.}, } - NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance FieldsLiangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas GeigerIn 2022
Visually exploring in a real-world 4D spatiotemporal space freely in VR hasbeen a long-term quest. The task is especially appealing when only a few oreven single RGB cameras are used for capturing the dynamic scene. To this end,we present an efficient framework capable of fast reconstruction, compactmodeling, and streamable rendering. First, we propose to decompose the 4Dspatiotemporal space according to temporal characteristics. Points in the 4Dspace are associated with probabilities of belonging to three categories:static, deforming, and new areas. Each area is represented and regularized by aseparate neural field. Second, we propose a hybrid representations basedfeature streaming scheme for efficiently modeling the neural fields. Ourapproach, coined NeRFPlayer, is evaluated on dynamic scenes captured by singlehand-held cameras and multi-camera arrays, achieving comparable or superiorrendering performance in terms of quality and speed comparable to recentstate-of-the-art methods, achieving reconstruction in 10 seconds per frame andreal-time rendering.
用一个hybrid representation-based feature streaming scheme表示动态场景.
neural-3d-videos@inproceedings{NeRFPlayer, title = {NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields}, author = {Song, Liangchen and Chen, Anpei and Li, Zhong and Chen, Zhang and Chen, Lele and Yuan, Junsong and Xu, Yi and Geiger, Andreas}, year = {2022}, tags = {neural-3d-videos}, sida = {用一个hybrid representation-based feature streaming scheme表示动态场景.}, }