Abstract
Augmented Reality (AR) is a promising way to precisely locate the internal structures of an organ in laparoscopy. Several methods have been proposed to register a preoperative 3D model reconstructed from MRI or CT to the intraoperative laparoscopy 2D images. These methods assume a fixed topology of the 3D model. They thus quickly fail once the organ is cut to remove pathological internal structures. We propose to add image-based incision detection in the registration pipeline, in order to update the topology of the organ model. Whenever an incision is detected, it is transferred to the 3D model, whose topology is then updated accordingly, and registration started. We trained a UNet as incision detector from 181 labelled incision images, collected from 10 myomectomy procedures. It obtains a mean precision, recall and f1 score of 0.05, 0.36, and 0.08 from 10-fold cross-validation. Overall, topology updating improves 3D registration accuracy by 5% on average.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Griwodz, C., et al.: Alicevision meshroom: an open-source 3D reconstruction pipeline. In: Proceedings of the 12th ACM Multimedia Systems Conference - MMSys 2021. ACM Press (2021). https://doi.org/10.1145/3458305.3478443
Photoscan (2015). https://www.agisoft.com
Adagolodjo, Y., Trivisonne, R., Haouchine, N., Cotin, S., Courtecuisse, H.: Silhouette-based pose estimation for deformable organs application to surgical augmented reality. In: IROS, pp. 539–544. IEEE (2017)
Amir-Khalili, A., Nosrati, M.S., Peyrat, J.-M., Hamarneh, G., Abugharbieh, R.: Uncertainty-encoded augmented reality for robot-assisted partial nephrectomy: a phantom study. In: Liao, H., Linte, C.A., Masamune, K., Peters, T.M., Zheng, G. (eds.) AE-CAI/MIAR -2013. LNCS, vol. 8090, pp. 182–191. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40843-4_20
Bartoli, A.: Maximizing the predictivity of smooth deformable image warps through cross-validation. JMIV 31(2–3), 133–145 (2008). https://doi.org/10.1007/s10851-007-0062-1
Collins, T., et al.: A system for augmented reality guided laparoscopic tumour resection with quantitative ex-vivo user evaluation. In: Peters, T., et al. (eds.) CARE 2016. LNCS, vol. 10170, pp. 114–126. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54057-3_11
Collins, T., et al.: Augmented reality guided laparoscopic surgery of the uterus. IEEE Trans. Med. Imaging 40(1), 371–380 (2020)
François, T., et al.: Detecting the occluding contours of the uterus to automatise augmented laparoscopy: score, loss, dataset, evaluation and user study. Int. J. Comput. Assist. Radiol. Surg. 15(7), 1177–1186 (2020). https://doi.org/10.1007/s11548-020-02151-w
Garcia-Peraza-Herrera, L.C., et al.: ToolNet: holistically-nested real-time segmentation of robotic surgical tools. In: IROS, pp. 5717–5722. IEEE (2017)
Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct estimation of nonrigid registrations with image-based self-occlusion reasoning. TPAMI 32(1), 87–104 (2008)
Han, L., Wang, H., Liu, Z., Chen, W., Zhang, X.: Vision-based cutting control of deformable objects with surface tracking. IEEE/ASME Transactions on Mechatronics (2020)
Haouchine, N., Dequidt, J., Berger, M.O., Cotin, S.: Monocular 3D reconstruction and augmentation of elastic surfaces with self-occlusion handling. IEEE Trans. Vis. Comput. Graph. 21(12), 1363–1376 (2015)
Haouchine, N., Dequidt, J., Peterlik, I., Kerrien, E., Berger, M.O., Cotin, S.: Image-guided simulation of heterogeneous tissue deformation for augmented reality during hepatic surgery. In: ISMAR, pp. 199–208. IEEE (2013)
Hattab, G., et al.: Kidney edge detection in laparoscopic image data for computer-assisted surgery. Int. J. Comput. Assist. Radiol. Surg. 15(3), 379–387 (2019). https://doi.org/10.1007/s11548-019-02102-0
Litjens, G.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Paulus, C.J., Haouchine, N., Kong, S.-H., Soares, R.V., Cazier, D., Cotin, S.: Handling topological changes during elastic registration. Int. J. Comput. Assist. Radiol. Surg. 12(3), 461–470 (2016). https://doi.org/10.1007/s11548-016-1502-4
Pizarro, D., Bartoli, A.: Feature-based deformable surface detection with self-occlusion reasoning. IJCV 97(1), 54–70 (2012). https://doi.org/10.1007/s11263-011-0452-0
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
de Vos, B.D., Berendsen, F.F., Viergever, M.A., Sokooti, H., Staring, M., Išgum, I.: A deep learning framework for unsupervised affine and deformable image registration. Med. Image Anal. 52, 128–143 (2019)
Wu, J., Westermann, R., Dick, C.: A survey of physically based simulation of cuts in deformable bodies. Comput. Graph. Forum 34(6), 161–187 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
François, T., Calvet, L., Sève-d’Erceville, C., Bourdel, N., Bartoli, A. (2021). Image-Based Incision Detection for Topological Intraoperative 3D Model Update in Augmented Reality Assisted Laparoscopic Surgery. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12904. Springer, Cham. https://doi.org/10.1007/978-3-030-87202-1_62
Download citation
DOI: https://doi.org/10.1007/978-3-030-87202-1_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87201-4
Online ISBN: 978-3-030-87202-1
eBook Packages: Computer ScienceComputer Science (R0)
