DuetGS: Two-Stage Controllable 3D Human Reconstruction from Dual Images.

Abstract

Creating realistic and fully detailed 3D human models using a minimal number of views has long been a challenging goal in 3D human reconstruction. Reconstructing a realistic human model from only two images (front and back) is particularly difficult due to the limited 3D information available, leading to two major challenges: (1) it is difficult to establish spatial consistency for reconstruction due to the lack of sufficient images for reliable matching, and (2) incomplete field of view results in missing color information. To address these challenges, we propose DuetGS, a novel pipeline that divides the reconstruction process into two stages: geometry reconstruction and color reconstruction. For geometry reconstruction, we employ a data-driven neural network to recover a full-body mesh from the front and back images, providing the spatial positioning for Gaussians. For color reconstruction, we adapt Gaussian Splatting and integrate our proposed unsupervised color propagation method to establish the color details of the Gaussians. Furthermore, our Gaussians are directly mapped to the mesh, allowing us to control their rotation and translation through mesh manipulation. This mapping ensures compatibility with various animation techniques. Extensive experiments on the THUman, CustomHumans, and PeopleSnapshot datasets demonstrate that our approach outperforms existing methods in terms of reconstruction accuracy and visual quality.