Edge_MVSFormer: Edge-Aware Multi-View Stereo Plant Reconstruction Based on Transformer Networks.

Abstract

With the rapid advancements in computer vision and deep learning, multi-view stereo (MVS) based on conventional RGB cameras has emerged as a promising and cost-effective tool for botanical research. However, existing methods often struggle to capture the intricate textures and fine edges of plants, resulting in suboptimal 3D reconstruction accuracy. To overcome this challenge, we proposed Edge_MVSFormer on the basis of TransMVSNet, which particularly focuses on enhancing the accuracy of plant leaf edge reconstruction. This model integrates an edge detection algorithm to augment edge information as input to the network and introduces an edge-aware loss function to focus the network's attention on a more accurate reconstruction of edge regions, where depth estimation errors are obviously more significant. Edge_MVSFormer was pre-trained on two public MVS datasets and fine-tuned with our private data of 10 model plants collected for this study. Experimental results on 10 test model plants demonstrated that for depth images, the proposed algorithm reduces the edge error and overall reconstruction error by 2.20 ± 0.36 mm and 0.46 ± 0.07 mm, respectively. For point clouds, the edge and overall reconstruction errors were reduced by 0.13 ± 0.02 mm and 0.05 ± 0.02 mm, respectively. This study underscores the critical role of edge information in the precise reconstruction of plant MVS data.