SegMeshNet: Joint heart segmentation and mesh reconstruction with task-aware shared attention.

Abstract

Segmentation and 3D mesh reconstruction of heart images are important for both medical diagnosis and subsequent medical research. By analyzing the common network processes in the two tasks, we find that they contain similar structures, and the efficiency of data utilization can be greatly improved if the two tasks are implemented simultaneously on this basis. To this end, we propose a simple and efficient joint learning framework for heart segmentation and heart mesh reconstruction(SegMeshNet). To improve the accuracy of heart mesh reconstruction and segmentation, we propose a task-aware shared attention module (TSA), which uses different pooling methods to extract more suitable feature representations for segmentation and reconstruction, and adaptively assigns weights through the attention mechanism to achieve cross-task feature interactions. We also designed the multi-scale feature fusion module (MSF), which further improves the feature representation capability by aggregating features of different scales. For the original mesh reconstruction loss, we propose the curvature-weighted hyperbolic chamfer distance loss (wHCD) to improve the reconstruction quality. Evaluations on existing CT and MR datasets demonstrate that our model outperforms most current state-of-the-art networks. More importantly, SegMeshNet exhibits favorable adaptability and versatility across different imaging modalities without the need for modality-specific architectures.