Physically Informed 3D Food Reconstruction: Methods and Results.

Abstract

Accurate food portion size estimation is a critical challenge in nutrition analysis and dietary assessment. Recent 3D reconstruction methods primarily focus on surface geometry, often neglecting the volumetric accuracy necessary for precise portion size estimation. In this work, we present three methods, NS-DRS, HR-CMS, and GS-GP, for real-scale 3D food reconstruction and volume estimation from limited 2D inputs. All three methods follow a shared three-stage pipeline consisting of 3D reconstruction, scale estimation using physical references such as visible checkerboards, and mesh refinement. While this high-level structure is consistent, the methods differ in their reconstruction backbones and scale-recovery mechanisms, and are designed for different input conditions, including both single-view and multi-view images. We evaluate each method on a diverse set of food items exhibiting variations in texture, shape, and camera pose, and assess their performance in terms of volume estimation and geometric accuracy. The results reveal complementary strengths among the pipelines, with NS-DRS performing better in volume estimation and GS-GP being more effective in 3D reconstruction. Across all settings, the three methods achieve 18-23% lower volume estimation error compared to the current state-of-the-art model. These findings demonstrate the effectiveness of physically informed and explainable reconstruction pipelines for accurate portion estimation, and support their potential use as scalable tools for dietary monitoring and clinical nutrition analysis.