4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos

4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos

4D3R in a nutshell

Turning single‑camera videos of moving scenes into crisp 3D views is hard—especially without known camera poses. 4D3R tackles this with a pose‑free, motion‑aware pipeline.

  • Two-stage design: first estimates camera and rough geometry using 3D foundation models, then refines with motion cues.
  • MA‑BA: Motion‑Aware Bundle Adjustment blends transformer priors and SAM2 segmentation to separate moving objects and sharpen camera pose.
  • MA‑GS: Motion‑Aware Gaussian Splatting uses compact control points, a deformation field MLP, and linear blend skinning for efficient, high‑quality motion.

Why it matters: sharper novel views from everyday videos of dynamic scenes—no precomputed poses or multi‑camera setups.

  • Up to 1.8 dB PSNR boost over leading methods
  • About 5× lower compute for dynamic modeling

By Mengqi Guo, Bo Xu, Yanyan Li, Gim Hee Lee. Paper: http://arxiv.org/abs/2511.05229v1

Paper: http://arxiv.org/abs/2511.05229v1

Register: https://www.AiFeta.com

#ComputerVision #3D #4D #DynamicScenes #NeRF #GaussianSplatting #MonocularVideo #NovelViewSynthesis #AIResearch

Read more