How augmented laparoscopy helps surgeons see hidden liver tumors

Abstract

We address the hidden tumour visualization problem in augmented monocular liver laparoscopy. Conveying a hidden tumour's depth correctly to the surgeon in augmented monocular laparoscopy is extremely difficult and still forms an unsolved problem. The depth conveyance can be split into two subsequent problems. First, designing a visualization that convinces the user to see the tumour inside the organ. Second, enhancing this visualization so that it also provides metric depth perception. The most promising visualization methods rely on a preoperative CT organ model with the tumour to be registered to an intraoperative laparoscopic image. Such a registration allows the organ's intraoperative shape mesh to be overlaid on top of the augmented tumour. The overlaid organ mesh guarantees a partial occlusion on the augmented tumour. This provides a powerful depth cue for the surgeon's perception. However, this type of registration, especially in liver laparoscopy, is usually not real-time and sometimes not possible. This is because of the liver deformation and lack of matchable features between the multimodal images. Subsequently, the tumour augmentation cannot be carried out continuously to guide the surgeon. We propose a novel visualization method to address these limitations. The proposed method replaces the deformable preoperative to intraoperative liver registration with a rigid tumour registration via laparoscopic ultrasound imaging. The proposed method handles surgical tool occlusions, runs faster, and outperforms the state of the art in terms of depth perception, as shown in the user study.