Urban scene reconstruction using Geometry-aware Gaussian primitives.

Abstract

Recent advancements in radiance fields, particularly with the emergence of Gaussian splatting, have highlighted their significant potential for 3D scene reconstruction and novel view synthesis. However, existing methods encounter substantial challenges when addressing dynamic environments, especially in complex urban settings with both rigid and non-rigid participants. To tackle these challenges, we propose a geometry-aware framework that integrates Gaussian primitives with a template mesh to effectively represent dynamic objects. This integration facilitates the efficient and accurate reconstruction of urban scenes, ensuring that the geometric integrity of dynamic elements is maintained. We first decompose the scene into a dynamic scene graph and fit the template vertices to observations to construct topologically consistent 3D models. Then, we build Gaussian radiance fields for dynamic nodes based on the template meshes, optimizing the vertex offset of dynamic participants to align with their geometric surfaces. We further project the appearance attributes into the 2D texture space based on topological relationships preserved in the Gaussians, enabling finer reconstruction of small-scale details and smoother appearance generalization on unseen surfaces. To validate the effectiveness of our proposed method, we conduct extensive evaluations on the Waymo Open Dataset (Ettinger et al., 2021) and the KITTI Dataset (Geiger et al., 2013). Our results demonstrate superior performance compared to mainstream dynamic reconstruction methods. We believe our work establishes a foundation for more realistic and geometrically complete urban scene reconstruction.