{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T19:09:26Z","timestamp":1735585766174},"reference-count":79,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T00:00:00Z","timestamp":1666915200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T00:00:00Z","timestamp":1666915200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010661","name":"EC | Horizon 2020 Framework Programme","doi-asserted-by":"publisher","award":["813782"],"id":[{"id":"10.13039\/100010661","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"publisher","award":["ANR-20-CHIA-0029-01","ANR-10-IAHU-02"],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"Abstract<\/jats:title>Hundreds of millions of operations are performed worldwide each year, and the rising uptake in minimally invasive surgery has enabled fiber optic cameras and robots to become both important tools to conduct surgery and sensors from which to capture information about surgery. Computer vision (CV), the application of algorithms to analyze and interpret visual data, has become a critical technology through which to study the intraoperative phase of care with the goals of augmenting surgeons\u2019 decision-making processes, supporting safer surgery, and expanding access to surgical care. While much work has been performed on potential use cases, there are currently no CV tools widely used for diagnostic or therapeutic applications in surgery. Using laparoscopic cholecystectomy as an example, we reviewed current CV techniques that have been applied to minimally invasive surgery and their clinical applications. Finally, we discuss the challenges and obstacles that remain to be overcome for broader implementation and adoption of CV in surgery.<\/jats:p>","DOI":"10.1038\/s41746-022-00707-5","type":"journal-article","created":{"date-parts":[[2022,10,28]],"date-time":"2022-10-28T11:11:55Z","timestamp":1666955515000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":68,"title":["Computer vision in surgery: from potential to clinical value"],"prefix":"10.1038","volume":"5","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-7288-3023","authenticated-orcid":false,"given":"Pietro","family":"Mascagni","sequence":"first","affiliation":[]},{"given":"Deepak","family":"Alapatt","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0002-5993-468X","authenticated-orcid":false,"given":"Luca","family":"Sestini","sequence":"additional","affiliation":[]},{"given":"Maria S.","family":"Altieri","sequence":"additional","affiliation":[]},{"given":"Amin","family":"Madani","sequence":"additional","affiliation":[]},{"given":"Yusuke","family":"Watanabe","sequence":"additional","affiliation":[]},{"given":"Adnan","family":"Alseidi","sequence":"additional","affiliation":[]},{"given":"Jay A.","family":"Redan","sequence":"additional","affiliation":[]},{"given":"Sergio","family":"Alfieri","sequence":"additional","affiliation":[]},{"given":"Guido","family":"Costamagna","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0001-8194-2670","authenticated-orcid":false,"given":"Ivo","family":"Bo\u0161koski","sequence":"additional","affiliation":[]},{"given":"Nicolas","family":"Padoy","sequence":"additional","affiliation":[]},{"ORCID":"http:\/\/orcid.org\/0000-0003-4725-3104","authenticated-orcid":false,"given":"Daniel A.","family":"Hashimoto","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,28]]},"reference":[{"key":"707_CR1","doi-asserted-by":"publisher","first-page":"S11","DOI":"10.1016\/S0140-6736(15)60806-6","volume":"385","author":"TG Weiser","year":"2015","unstructured":"Weiser, T. G. et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet 385, S11 (2015).","journal-title":"Lancet"},{"key":"707_CR2","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.surg.2015.04.011","volume":"158","author":"JG Meara","year":"2015","unstructured":"Meara, J. G. et al. Global Surgery 2030: Evidence and solutions for achieving health, welfare, and economic development. Surgery 158, 3\u20136 (2015).","journal-title":"Surgery"},{"key":"707_CR3","doi-asserted-by":"publisher","first-page":"e176233","DOI":"10.1001\/jamasurg.2017.6233","volume":"153","author":"CP Childers","year":"2018","unstructured":"Childers, C. P. & Maggard-Gibbons, M. Understanding costs of care in the operating room. JAMA Surg. 153, e176233 (2018).","journal-title":"JAMA Surg."},{"key":"707_CR4","doi-asserted-by":"publisher","DOI":"10.1186\/1754-9493-5-13","volume":"5","author":"M Zegers","year":"2011","unstructured":"Zegers, M. et al. The incidence, root-causes, and outcomes of adverse events in surgical units: implication for potential prevention strategies. Patient Saf. Surg. 5, 13 (2011).","journal-title":"Patient Saf. Surg."},{"key":"707_CR5","doi-asserted-by":"publisher","first-page":"S260","DOI":"10.21037\/jss.2019.09.31","volume":"6","author":"K-U Lewandrowski","year":"2020","unstructured":"Lewandrowski, K.-U. et al. Regional variations in acceptance, and utilization of minimally invasive spinal surgery techniques among spine surgeons: results of a global survey. J. Spine Surg. 6, S260\u2013S274 (2020).","journal-title":"J. Spine Surg."},{"key":"707_CR6","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1097\/SLA.0b013e31828faa66","volume":"258","author":"O Bardakcioglu","year":"2013","unstructured":"Bardakcioglu, O., Khan, A., Aldridge, C. & Chen, J. Growth of laparoscopic colectomy in the United States: analysis of regional and socioeconomic factors over time. Ann. Surg. 258, 270\u2013274 (2013).","journal-title":"Ann. Surg."},{"key":"707_CR7","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1001\/jamasurg.2014.1791","volume":"150","author":"MK Richards","year":"2015","unstructured":"Richards, M. K. et al. A national review of the frequency of minimally invasive surgery among general surgery residents: assessment of ACGME case logs during 2 decades of general surgery resident training. JAMA Surg. 150, 169\u2013172 (2015).","journal-title":"JAMA Surg."},{"key":"707_CR8","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1007\/s00464-011-2011-8","volume":"26","author":"M Zhou","year":"2012","unstructured":"Zhou, M. et al. Effect of haptic feedback in laparoscopic surgery skill acquisition. Surg. Endosc. 26, 1128\u20131134 (2012).","journal-title":"Surg. Endosc."},{"key":"707_CR9","doi-asserted-by":"publisher","unstructured":"Balvardi, S. et al. The association between video-based assessment of intraoperative technical performance and patient outcomes: a systematic review. Surg. Endosc. https:\/\/doi.org\/10.1007\/s00464-022-09296-6 (2022).","DOI":"10.1007\/s00464-022-09296-6"},{"key":"707_CR10","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1016\/j.jamcollsurg.2021.06.018","volume":"233","author":"P Mascagni","year":"2021","unstructured":"Mascagni, P. et al. Intraoperative time-out to promote the implementation of the critical view of safety in laparoscopic cholecystectomy: A video-based assessment of 343 procedures. J. Am. Coll. Surg. 233, 497\u2013505 (2021).","journal-title":"J. Am. Coll. Surg."},{"key":"707_CR11","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.amjsurg.2020.06.027","volume":"221","author":"CM Pugh","year":"2021","unstructured":"Pugh, C. M., Hashimoto, D. A. & Korndorffer, J. R. Jr. The what? How? And Who? Of video based assessment. Am. J. Surg. 221, 13\u201318 (2021).","journal-title":"Am. J. Surg."},{"key":"707_CR12","doi-asserted-by":"publisher","first-page":"3285","DOI":"10.1007\/s00464-020-07628-y","volume":"34","author":"LS Feldman","year":"2020","unstructured":"Feldman, L. S. et al. SAGES Video-Based Assessment (VBA) program: a vision for life-long learning for surgeons. Surg. Endosc. 34, 3285\u20133288 (2020).","journal-title":"Surg. Endosc."},{"key":"707_CR13","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.jss.2017.05.067","volume":"223","author":"G Sharma","year":"2018","unstructured":"Sharma, G. et al. A cadaveric procedural anatomy simulation course improves video-based assessment of operative performance. J. Surg. Res. 223, 64\u201371 (2018).","journal-title":"J. Surg. Res."},{"key":"707_CR14","doi-asserted-by":"publisher","first-page":"1253","DOI":"10.1016\/j.surg.2020.10.039","volume":"169","author":"TM Ward","year":"2021","unstructured":"Ward, T. M. et al. Computer vision in surgery. Surgery 169, 1253\u20131256 (2021).","journal-title":"Surgery"},{"key":"707_CR15","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.gie.2020.06.059","volume":"93","author":"C Hassan","year":"2021","unstructured":"Hassan, C. et al. Performance of artificial intelligence in colonoscopy for adenoma and polyp detection: a systematic review and meta-analysis. Gastrointest. Endosc. 93, 77\u201385.e6 (2021).","journal-title":"Gastrointest. Endosc."},{"key":"707_CR16","doi-asserted-by":"publisher","first-page":"3797","DOI":"10.1007\/s00330-021-07892-z","volume":"31","author":"KG van Leeuwen","year":"2021","unstructured":"van Leeuwen, K. G., Schalekamp, S., Rutten, M. J. C. M., van Ginneken, B. & de Rooij, M. Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur. Radiol. 31, 3797\u20133804 (2021).","journal-title":"Eur. Radiol."},{"key":"707_CR17","doi-asserted-by":"publisher","first-page":"2175","DOI":"10.1007\/s00464-017-5974-2","volume":"32","author":"PH Pucher","year":"2018","unstructured":"Pucher, P. H. et al. Outcome trends and safety measures after 30 years of laparoscopic cholecystectomy: a systematic review and pooled data analysis. Surg. Endosc. 32, 2175\u20132183 (2018).","journal-title":"Surg. Endosc."},{"key":"707_CR18","doi-asserted-by":"publisher","first-page":"e6457","DOI":"10.1136\/bmj.e6457","volume":"345","author":"B T\u00f6rnqvist","year":"2012","unstructured":"T\u00f6rnqvist, B., Str\u00f6mberg, C., Persson, G. & Nilsson, M. Effect of intended intraoperative cholangiography and early detection of bile duct injury on survival after cholecystectomy: population based cohort study. BMJ 345, e6457 (2012).","journal-title":"BMJ"},{"key":"707_CR19","doi-asserted-by":"crossref","unstructured":"A prospective analysis of 1518 laparoscopic cholecystectomies. N. Engl. J. Med. 324, 1073\u20131078 (1991).","DOI":"10.1056\/NEJM199104183241601"},{"key":"707_CR20","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.surg.2006.01.008","volume":"140","author":"SO Rogers Jr.","year":"2006","unstructured":"Rogers, S. O. Jr. et al. Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery 140, 25\u201333 (2006).","journal-title":"Surgery"},{"key":"707_CR21","doi-asserted-by":"publisher","first-page":"1051","DOI":"10.1007\/s00464-012-2767-5","volume":"27","author":"G Berci","year":"2013","unstructured":"Berci, G. et al. Laparoscopic cholecystectomy: first, do no harm; second, take care of bile duct stones. Surg. Endosc. 27, 1051\u20131054 (2013).","journal-title":"Surg. Endosc."},{"key":"707_CR22","doi-asserted-by":"publisher","first-page":"1521","DOI":"10.1007\/s00464-020-08168-1","volume":"35","author":"R Anteby","year":"2021","unstructured":"Anteby, R. et al. Deep learning visual analysis in laparoscopic surgery: a systematic review and diagnostic test accuracy meta-analysis. Surg. Endosc. 35, 1521\u20131533 (2021).","journal-title":"Surg. Endosc."},{"key":"707_CR23","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1097\/SLA.0000000000004425","volume":"273","author":"CR Garrow","year":"2021","unstructured":"Garrow, C. R. et al. Machine learning for surgical phase recognition: A systematic review. Ann. Surg. 273, 684\u2013693 (2021).","journal-title":"Ann. Surg."},{"key":"707_CR24","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","volume":"36","author":"AP Twinanda","year":"2017","unstructured":"Twinanda, A. P. et al. EndoNet: A deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36, 86\u201397 (2017).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"707_CR25","doi-asserted-by":"publisher","first-page":"556","DOI":"10.1109\/TMI.2019.2931158","volume":"39","author":"S Kannan","year":"2020","unstructured":"Kannan, S., Yengera, G., Mutter, D., Marescaux, J. & Padoy, N. Future-State Predicting LSTM for early surgery type recognition. IEEE Trans. Med. Imaging 39, 556\u2013566 (2020).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"707_CR26","unstructured":"Yengera, G., Mutter, D., Marescaux, J. & Padoy, N. Less is more: Surgical phase recognition with less annotations through self-supervised pre-training of CNN-LSTM networks. arXiv [cs.CV] (2018)."},{"key":"707_CR27","doi-asserted-by":"crossref","unstructured":"Meireles, O. R. et al. SAGES consensus recommendations on an annotation framework for surgical video. Surg. Endosc. In Press, (2021).","DOI":"10.1007\/s00464-021-08578-9"},{"key":"707_CR28","doi-asserted-by":"publisher","first-page":"102433","DOI":"10.1016\/j.media.2022.102433","volume":"78","author":"CI Nwoye","year":"2022","unstructured":"Nwoye, C. I. et al. Rendezvous: Attention mechanisms for the recognition of surgical action triplets in endoscopic videos. Med. Image Anal. 78, 102433 (2022).","journal-title":"Med. Image Anal."},{"key":"707_CR29","doi-asserted-by":"publisher","unstructured":"Yeung, S. et al. A real-time spatiotemporal AI model analyzes skill in open surgical videos. Res. Square https:\/\/doi.org\/10.21203\/rs.3.rs-1129461\/v1 (2021).","DOI":"10.21203\/rs.3.rs-1129461\/v1"},{"key":"707_CR30","doi-asserted-by":"publisher","first-page":"e93","DOI":"10.1097\/SLA.0000000000004736","volume":"274","author":"P Mascagni","year":"2021","unstructured":"Mascagni, P. et al. A computer vision platform to automatically locate critical events in surgical videos: Documenting safety in laparoscopic cholecystectomy. Ann. Surg. 274, e93\u2013e95 (2021).","journal-title":"Ann. Surg."},{"key":"707_CR31","doi-asserted-by":"publisher","unstructured":"Mascagni, P. et al. Multicentric validation of EndoDigest: a computer vision platform for video documentation of the critical view of safety in laparoscopic cholecystectomy. Surg. Endosc. https:\/\/doi.org\/10.1007\/s00464-022-09112-1 (2022).","DOI":"10.1007\/s00464-022-09112-1"},{"key":"707_CR32","doi-asserted-by":"publisher","unstructured":"Yu, T. & Padoy, N. Encode the Unseen: Predictive Video Hashing for Scalable Mid-stream Retrieval. In: Ishikawa, H., Liu, CL., Pajdla, T., Shi, J. (eds) Computer Vision \u2013 ACCV 2020. ACCV 2020. Lecture Notes in Computer Science, vol 12626. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-030-69541-5_26 (2021).","DOI":"10.1007\/978-3-030-69541-5_26"},{"key":"707_CR33","unstructured":"Yu, T. et al. Live laparoscopic video retrieval with compressed uncertainty. Preprint at: https:\/\/arxiv.org\/abs\/2203.04301 (2022)."},{"key":"707_CR34","doi-asserted-by":"publisher","unstructured":"Berlet, M. et al. Surgical reporting for laparoscopic cholecystectomy based on phase annotation by a convolutional neural network (CNN) and the phenomenon of phase flickering: a proof of concept. Int. J. Comput. Assist. Radiol. Surg. https:\/\/doi.org\/10.1007\/s11548-022-02680-6 (2022).","DOI":"10.1007\/s11548-022-02680-6"},{"key":"707_CR35","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1007\/s11548-020-02285-x","volume":"16","author":"C Loukas","year":"2021","unstructured":"Loukas, C., Frountzas, M. & Schizas, D. Patch-based classification of gallbladder wall vascularity from laparoscopic images using deep learning. Int. J. Comput. Assist. Radiol. Surg. 16, 103\u2013113 (2021).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"707_CR36","doi-asserted-by":"publisher","unstructured":"Ward, T. M., Hashimoto, D. A., Ban, Y., Rosman, G. & Meireles, O. R. Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation. Surg. Endosc. https:\/\/doi.org\/10.1007\/s00464-022-09009-z (2022).","DOI":"10.1007\/s00464-022-09009-z"},{"key":"707_CR37","doi-asserted-by":"publisher","unstructured":"Jin, A. et al. Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks, IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, pp. 691\u2013699. https:\/\/doi.org\/10.1109\/WACV.2018.00081 (2018).","DOI":"10.1109\/WACV.2018.00081"},{"key":"707_CR38","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-84295-6","volume":"11","author":"JL Lavanchy","year":"2021","unstructured":"Lavanchy, J. L. et al. Automation of surgical skill assessment using a three-stage machine learning algorithm. Sci. Rep. 11, 5197 (2021).","journal-title":"Sci. Rep."},{"key":"707_CR39","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1002\/jso.26496","volume":"124","author":"TM Ward","year":"2021","unstructured":"Ward, T. M. et al. Surgical data science and artificial intelligence for surgical education. J. Surg. Oncol. 124, 221\u2013230 (2021).","journal-title":"J. Surg. Oncol."},{"key":"707_CR40","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1016\/j.surg.2021.10.001","volume":"171","author":"M Vannucci","year":"2022","unstructured":"Vannucci, M. et al. Statistical models to preoperatively predict operative difficulty in laparoscopic cholecystectomy: A systematic review. Surgery 171, 1158\u20131167 (2022).","journal-title":"Surgery"},{"key":"707_CR41","doi-asserted-by":"publisher","first-page":"1651","DOI":"10.1007\/s00464-020-07548-x","volume":"35","author":"T Tokuyasu","year":"2021","unstructured":"Tokuyasu, T. et al. Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg. Endosc. 35, 1651\u20131658 (2021).","journal-title":"Surg. Endosc."},{"key":"707_CR42","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1097\/SLA.0000000000004594","volume":"276","author":"A Madani","year":"2022","unstructured":"Madani, A. et al. Artificial intelligence for intraoperative guidance. Ann. Surg. 276, 363\u2013369 (2022).","journal-title":"Ann. Surg."},{"key":"707_CR43","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1097\/01.SLA.0000060680.92690.E9","volume":"237","author":"LW Way","year":"2003","unstructured":"Way, L. W. et al. Causes and prevention of laparoscopic bile duct injuries. Ann. Surg. 237, 460\u2013469 (2003).","journal-title":"Ann. Surg."},{"key":"707_CR44","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1097\/SLA.0000000000003791","volume":"272","author":"LM Brunt","year":"2020","unstructured":"Brunt, L. M. et al. Safe Cholecystectomy Multi-society Practice Guideline and State of the Art Consensus Conference on Prevention of Bile Duct Injury During Cholecystectomy. Ann. Surg. 272, 3\u201323 (2020).","journal-title":"Ann. Surg."},{"key":"707_CR45","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1097\/SLA.0000000000004351","volume":"275","author":"P Mascagni","year":"2022","unstructured":"Mascagni, P. et al. Artificial intelligence for surgical safety. Ann. Surg. 275, 955\u2013961 (2022).","journal-title":"Ann. Surg."},{"key":"707_CR46","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s11548-021-02441-x","volume":"17","author":"F Aspart","year":"2022","unstructured":"Aspart, F. et al. ClipAssistNet: bringing real-time safety feedback to operating rooms. Int. J. Comput. Assist. Radiol. Surg. 17, 5\u201313 (2022).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"707_CR47","doi-asserted-by":"publisher","first-page":"1069","DOI":"10.1109\/TMI.2018.2878055","volume":"38","author":"AP Twinanda","year":"2019","unstructured":"Twinanda, A. P., Yengera, G., Mutter, D., Marescaux, J. & Padoy, N. RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Trans. Med. Imaging 38, 1069\u20131078 (2019).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"707_CR48","doi-asserted-by":"publisher","first-page":"4008","DOI":"10.1007\/s00464-020-07833-9","volume":"35","author":"TM Ward","year":"2021","unstructured":"Ward, T. M. et al. Automated operative phase identification in peroral endoscopic myotomy. Surg. Endosc. 35, 4008\u20134015 (2021).","journal-title":"Surg. Endosc."},{"key":"707_CR49","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1016\/j.amjsurg.2014.02.014","volume":"208","author":"MN Mavros","year":"2014","unstructured":"Mavros, M. N. et al. Opening Pandora\u2019s box: understanding the nature, patterns, and 30-day outcomes of intraoperative adverse events. Am. J. Surg. 208, 626\u2013631 (2014).","journal-title":"Am. J. Surg."},{"key":"707_CR50","doi-asserted-by":"publisher","first-page":"1090","DOI":"10.1007\/s00464-021-08375-4","volume":"36","author":"L Mazer","year":"2022","unstructured":"Mazer, L., Varban, O., Montgomery, J. R., Awad, M. M. & Schulman, A. Video is better: why aren\u2019t we using it? A mixed-methods study of the barriers to routine procedural video recording and case review. Surg. Endosc. 36, 1090\u20131097 (2022).","journal-title":"Surg. Endosc."},{"key":"707_CR51","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1002\/bjs.11198","volume":"106","author":"ASHM van Dalen","year":"2019","unstructured":"van Dalen, A. S. H. M., Legemaate, J., Schlack, W. S., Legemate, D. A. & Schijven, M. P. Legal perspectives on black box recording devices in the operating environment. Br. J. Surg. 106, 1433\u20131441 (2019).","journal-title":"Br. J. Surg."},{"key":"707_CR52","unstructured":"United States Code of Federal Regulation. 42 CFR Ch I, Part 3. https:\/\/www.ecfr.gov\/current\/title-42\/chapter-I\/subchapter-A\/part-3."},{"key":"707_CR53","doi-asserted-by":"publisher","first-page":"48658","DOI":"10.1109\/ACCESS.2021.3068852","volume":"9","author":"I Rivas-Blanco","year":"2021","unstructured":"Rivas-Blanco, I., Perez-Del-Pulgar, C. J., Garcia-Morales, I. & Munoz, V. F. A review on deep learning in minimally invasive surgery. IEEE Access 9, 48658\u201348678 (2021).","journal-title":"IEEE Access"},{"key":"707_CR54","doi-asserted-by":"publisher","first-page":"15","DOI":"10.3390\/jimaging7020015","volume":"7","author":"T Shimizu","year":"2021","unstructured":"Shimizu, T., Hachiuma, R., Kajita, H., Takatsume, Y. & Saito, H. Hand motion-aware surgical tool localization and classification from an egocentric camera. J. Imaging 7, 15 (2021).","journal-title":"J. Imaging"},{"key":"707_CR55","first-page":"1373","volume":"2020","author":"M Zhang","year":"2020","unstructured":"Zhang, M. et al. Using computer vision to automate hand detection and tracking of surgeon movements in videos of open surgery. AMIA Annu. Symp. Proc. 2020, 1373\u20131382 (2020).","journal-title":"AMIA Annu. Symp. Proc."},{"key":"707_CR56","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1007\/s11548-022-02559-6","volume":"17","author":"A Goldbraikh","year":"2022","unstructured":"Goldbraikh, A., D\u2019Angelo, A.-L., Pugh, C. M. & Laufer, S. Video-based fully automatic assessment of open surgery suturing skills. Int. J. Comput. Assist. Radiol. Surg. 17, 437\u2013448 (2022).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"707_CR57","unstructured":"Mascagni, P. et al. Surgical data science for safe cholecystectomy: a protocol for segmentation of hepatocystic anatomy and assessment of the critical view of safety. Preprint at: https:\/\/arxiv.org\/abs\/2106.10916 (2021)."},{"key":"707_CR58","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1080\/24699322.2021.1937320","volume":"26","author":"TM Ward","year":"2021","unstructured":"Ward, T. M. et al. Challenges in surgical video annotation. Comput Assist Surg. (Abingdon) 26, 58\u201368 (2021).","journal-title":"Comput Assist Surg. (Abingdon)"},{"key":"707_CR59","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","volume":"25","author":"A Esteva","year":"2019","unstructured":"Esteva, A. et al. A guide to deep learning in healthcare. Nat. Med. 25, 24\u201329 (2019).","journal-title":"Nat. Med."},{"key":"707_CR60","doi-asserted-by":"publisher","first-page":"102306","DOI":"10.1016\/j.media.2021.102306","volume":"76","author":"L Maier-Hein","year":"2022","unstructured":"Maier-Hein, L. et al. Surgical data science - from concepts toward clinical translation. Med. Image Anal. 76, 102306 (2022).","journal-title":"Med. Image Anal."},{"key":"707_CR61","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1007\/s11548-019-01962-w","volume":"14","author":"A Rau","year":"2019","unstructured":"Rau, A. et al. Implicit domain adaptation with conditional generative adversarial networks for depth prediction in endoscopy. Int. J. Comput. Assist. Radiol. Surg. 14, 1167\u20131176 (2019).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"707_CR62","doi-asserted-by":"crossref","unstructured":"Shorten, C. & Khoshgoftaar, T. M. A survey on image data augmentation for deep learning. J. Big Data 6, (2019).","DOI":"10.1186\/s40537-019-0197-0"},{"key":"707_CR63","doi-asserted-by":"crossref","unstructured":"Kassem, H. et al. Federated cycling (FedCy): Semi-supervised Federated Learning of surgical phases. Preprint at: https:\/\/arxiv.org\/abs\/2203.07345 (2022).","DOI":"10.1109\/TMI.2022.3222126"},{"key":"707_CR64","unstructured":"Taleb, A. et al. 3D self-supervised methods for medical imaging. In Proceedings of the 34th International Conference on Neural Information Processing Systems (pp. 18158\u201318172) (2020)."},{"key":"707_CR65","unstructured":"Yu, T., Mutter, D., Marescaux, J. & Padoy, N. Learning from a tiny dataset of manual annotations: a teacher\/student approach for surgical phase recognition. Preprint at: https:\/\/arxiv.org\/abs\/1812.00033 (2018)."},{"key":"707_CR66","doi-asserted-by":"publisher","first-page":"102158","DOI":"10.1016\/j.media.2021.102158","volume":"73","author":"X Shi","year":"2021","unstructured":"Shi, X., Jin, Y., Dou, Q. & Heng, P.-A. Semi-supervised learning with progressive unlabeled data excavation for label-efficient surgical workflow recognition. Med. Image Anal. 73, 102158 (2021).","journal-title":"Med. Image Anal."},{"key":"707_CR67","doi-asserted-by":"publisher","first-page":"1675","DOI":"10.1109\/TNNLS.2017.2677468","volume":"29","author":"J Zhang","year":"2018","unstructured":"Zhang, J., Sheng, V. S., Li, T. & Wu, X. Improving crowdsourced label quality using noise correction. IEEE Trans. Neural Netw. Learn. Syst. 29, 1675\u20131688 (2018).","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"707_CR68","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1007\/s11548-019-01958-6","volume":"14","author":"CI Nwoye","year":"2019","unstructured":"Nwoye, C. I., Mutter, D., Marescaux, J. & Padoy, N. Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos. Int. J. Comput. Assist. Radiol. Surg. 14, 1059\u20131067 (2019).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"707_CR69","doi-asserted-by":"publisher","unstructured":"Deng, J. et al. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009). https:\/\/doi.org\/10.1109\/cvpr.2009.5206848.","DOI":"10.1109\/cvpr.2009.5206848"},{"key":"707_CR70","doi-asserted-by":"publisher","first-page":"e190043","DOI":"10.1148\/ryai.2020190043","volume":"2","author":"M Reyes","year":"2020","unstructured":"Reyes, M. et al. On the interpretability of artificial intelligence in radiology: Challenges and opportunities. Radio. Artif. Intell. 2, e190043 (2020).","journal-title":"Radio. Artif. Intell."},{"key":"707_CR71","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-17478-w","volume":"11","author":"DC Castro","year":"2020","unstructured":"Castro, D. C., Walker, I. & Glocker, B. Causality matters in medical imaging. Nat. Commun. 11, 3673 (2020).","journal-title":"Nat. Commun."},{"key":"707_CR72","doi-asserted-by":"publisher","first-page":"e384","DOI":"10.1016\/S2589-7500(22)00003-6","volume":"4","author":"X Liu","year":"2022","unstructured":"Liu, X. et al. The medical algorithmic audit. Lancet Digit Health 4, e384\u2013e397 (2022).","journal-title":"Lancet Digit Health"},{"key":"707_CR73","doi-asserted-by":"publisher","first-page":"101079","DOI":"10.1109\/ACCESS.2020.2997831","volume":"8","author":"L Sun","year":"2020","unstructured":"Sun, L., Jiang, X., Ren, H. & Guo, Y. Edge-cloud computing and artificial intelligence in internet of medical things: Architecture, technology and application. IEEE Access 8, 101079\u2013101092 (2020).","journal-title":"IEEE Access"},{"key":"707_CR74","doi-asserted-by":"crossref","unstructured":"Gerke, S., Minssen, T. & Cohen, G. Ethical and legal challenges of artificial intelligence-driven healthcare. In Artificial Intelligence in Healthcare 295\u2013336 (Elsevier, 2020).","DOI":"10.1016\/B978-0-12-818438-7.00012-5"},{"key":"707_CR75","doi-asserted-by":"publisher","unstructured":"Gallant, J.-N., Brelsford, K., Sharma, S., Grantcharov, T. & Langerman, A. Patient Perceptions of Audio and Video Recording in the Operating Room. Ann. Surg. https:\/\/doi.org\/10.1097\/SLA.0000000000004759 (2021).","DOI":"10.1097\/SLA.0000000000004759"},{"key":"707_CR76","doi-asserted-by":"publisher","first-page":"e406","DOI":"10.1016\/S2589-7500(22)00063-2","volume":"4","author":"JW Gichoya","year":"2022","unstructured":"Gichoya, J. W. et al. AI recognition of patient race in medical imaging: a modelling study. Lancet Digit Health 4, e406\u2013e414 (2022).","journal-title":"Lancet Digit Health"},{"key":"707_CR77","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1038\/s41591-020-01192-7","volume":"27","author":"E Pierson","year":"2021","unstructured":"Pierson, E., Cutler, D. M., Leskovec, J., Mullainathan, S. & Obermeyer, Z. An algorithmic approach to reducing unexplained pain disparities in underserved populations. Nat. Med. 27, 136\u2013140 (2021).","journal-title":"Nat. Med."},{"key":"707_CR78","unstructured":"Reznick, R. et al. Task Force Report on Artificial Intelligence and Emerging Digital Technologies. Published at: https:\/\/www.royalcollege.ca\/rcsite\/documents\/health-policy\/rc-ai-task-force-e.pdf (2021)."},{"key":"707_CR79","unstructured":"The topol review \u2014 NHS health education England. The Topol Review \u2014 NHS Health Education England. Published at: https:\/\/topol.hee.nhs.uk\/ (2019)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00707-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00707-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00707-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,10]],"date-time":"2023-03-10T12:35:44Z","timestamp":1678451744000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-022-00707-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,28]]},"references-count":79,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["707"],"URL":"https:\/\/doi.org\/10.1038\/s41746-022-00707-5","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,28]]},"assertion":[{"value":"15 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The Authors declare the following Competing Financial Interests: AM is a consultant for Activ Surgical and Genesis MedTech. NP is a scientific advisor for Caresyntax and his laboratory receives a PhD fellowship from Intuitive Surgical. DAH is a consultant for Johnson & Johnson Institute and Activ Surgical. He previously received research support from Olympus Corporation. The Authors declare also the following Competing Non-Financial Interests: PM, MSA, AM, YW, AA, and DAH serve on the board of directors for the Global Surgical AI Collaborative, a non-profit organization that oversees a data sharing and analytics platform for surgical data.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"163"}}