Integrating 2D Dermatological Photography with 3D Anatomical Surfaces.

Abstract

Standardized 2D photography plays an essential role in dermatologic practice, supporting longitudinal documentation, patient monitoring, and consensus-based clinical scoring. However, photographs taken from limited views often suffer from reduced anatomical context and missing body-location information. 3D representations enable unified spatial interpretation of multi-view imagery. Recent developments in computer vision have made it feasible to infer dense correspondences between 2D images and a 3D human mesh. In this study, we explored integrating 2D dermatological images with a 3D surface model using DensePose, a deep learning-based human dense correspondence framework. This creates an anatomically grounded representation that supports mesh-level analyses and recovers spatial context for each image. We use a dataset including four full body photographs (front, back, and each side) from each of 147 subjects with chronic graft-versus-host disease, for a total of 588 images. Our method integrates these multiple 2D full-body photographs captured across varied body shapes and camera angles into a 3D mesh. We further showed that the resulting 3D mesh enables quantification of the extent to which individual 2D images, or their combinations, represent the complete body surface. On average, a single full body view captures 28% of the body surface, while adding a second, third, and fourth view increases average coverage to 50%, 72%, and 80%, respectively. To assess spatial consistency, we annotated up to 10 anatomical landmarks per patient on 80 images across 20 patients and reported a median pairwise geodesic distance between corresponding landmarks of 4.6 cm. These findings can guide how dermatology images are captured and support future opportunities in monitoring, education, and communication using existing infrastructure.