{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,11,27]],"date-time":"2024-11-27T17:12:47Z","timestamp":1732727567138,"version":"3.29.0"},"reference-count":28,"publisher":"SAGE Publications","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IFS"],"published-print":{"date-parts":[[2022,8,10]]},"abstract":"Recent developments in deep learning can be used in skill assessments for laparoscopic surgeons. In Minimally Invasive Surgery (MIS), surgeons should acquire many skills before carrying out a real operation. The Laparoscopic Surgical Box-Trainer allows surgery residents to train on specific skills that are not traditionally taught to them. This study aims to automatically detect the tips of laparoscopic instruments, localize a point, evaluate the detection accuracy to provide valuable assessment and expedite the development of surgery skills and assess the trainees\u2019 performance using a Multi-Input-Single-Output Fuzzy Logic Supervisor system. The output of the fuzzy logic assessment is the performance evaluation for the surgeon, and it is quantified in percentages. Based on the experimental results, the trained SSD Mobilenet V2 FPN can identify each instrument at a score of 70% fidelity. On the other hand, the trained SSD ResNet50 V1 FPN can detect each instrument at the score of 90% fidelity, in each location within a region of interest, and determine their relative distance with over 65% and 80% reliability, respectively. This method can be applied in different types of laparoscopic tooltip detection. Because there were a few instances when the detection failed, and the system was designed to generate pass-fail assessment, we recommend improving the measurement algorithm and the performance assessment by adding a camera to the system and measuring the distance from multiple perspectives.<\/jats:p>","DOI":"10.3233\/jifs-213243","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T16:26:00Z","timestamp":1652804760000},"page":"4741-4756","source":"Crossref","is-referenced-by-count":0,"title":["Fuzzy logic supervisor \u2013 A surgical skills assessment system using multi-class detection of laparoscopic box-trainer instruments"],"prefix":"10.1177","volume":"43","author":[{"given":"Fatemeh Rashidi","family":"Fathabadi","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering Department, Western Michigan University, USA"}]},{"given":"Janos L.","family":"Grantner","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering Department, Western Michigan University, USA"}]},{"given":"Saad A.","family":"Shebrain","sequence":"additional","affiliation":[{"name":"Department of Surgery, of the Homer Stryker M.D. School of Medicine, Western Michigan University, USA"}]},{"given":"Ikhlas","family":"Abdel-Qader","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering Department, Western Michigan University, USA"}]}],"member":"179","reference":[{"issue":"2","key":"10.3233\/JIFS-213243_ref1","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1109\/TMRB.2019.2913282","article-title":"Autonomy for surgical robots: Concepts and paradigms","volume":"1","author":"Haidegger","year":"2019","journal-title":"IEEE Trans Med Robot Bionics"},{"issue":"2","key":"10.3233\/JIFS-213243_ref2","first-page":"1","article-title":"Surgical simulation training systems: box trainers, virtual reality and augmented reality simulators","volume":"1","author":"Lahanas","year":"2016","journal-title":"Int J Adv Robot Autom"},{"key":"10.3233\/JIFS-213243_ref3","first-page":"22","article-title":"Achieving interface and environment fidelity inthe Virtual Basic Laparoscopic Surgical Trainer","volume":"96","author":"Chellali","year":"2016","journal-title":"Int J HumComput Stud"},{"issue":"9","key":"10.3233\/JIFS-213243_ref4","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1089\/lap.2016.0161","article-title":"Surgical techniques for totally laparoscopic caudate lobectomy","volume":"26","author":"Oh","year":"2016","journal-title":"J Laparoendosc Adv Surg Tech"},{"issue":"1","key":"10.3233\/JIFS-213243_ref6","doi-asserted-by":"crossref","first-page":"37","DOI":"10.3390\/robotics10010037","article-title":"Visual Intelligence: Prediction of Unintentional Surgical-Tool-Induced Bleeding during Robotic and Laparoscopic Surgery","volume":"10","author":"Daneshgar Rahbar","year":"2021","journal-title":"Robotics"},{"issue":"4","key":"10.3233\/JIFS-213243_ref7","doi-asserted-by":"crossref","first-page":"724","DOI":"10.1016\/j.jsurg.2016.12.009","article-title":"Box-or virtual-reality trainer: which tool results in better transfer of laparoscopic basic skills?\u2014 A prospective randomized trial","volume":"74","author":"Brinkmann","year":"2017","journal-title":"J Surg Educ"},{"issue":"6","key":"10.3233\/JIFS-213243_ref8","doi-asserted-by":"crossref","first-page":"661","DOI":"10.3138\/jvme.2018-0004","article-title":"Veterinary Student Self-Assessment of Basic Surgical Skills as an Experiential Learning Tool","volume":"47","author":"Tobias","year":"2020","journal-title":"J Vet Med Educ"},{"key":"10.3233\/JIFS-213243_ref10","doi-asserted-by":"crossref","unstructured":"Grantner J.L. , Kurdi A.H. , Al-Gailani M. , Abdel-Qader I. , Sawyer R.G. and Shebrain S. , Multi-Thread Implementation of Tool Tip Tracking for Laparoscopic Surgical Box-Trainer Intelligent Performance Assessment System, Acta Polytech Hungarica 16(9), 2019.","DOI":"10.1109\/INES46365.2019.9109507"},{"key":"10.3233\/JIFS-213243_ref11","doi-asserted-by":"crossref","unstructured":"Fathabadi F.R. , Grantner J.L. , Abdel-Qader I. and ShebrainBox-Trainer S.A. , Assessment System with Real-Time Multi-Class Detectionand Tracking of Laparoscopic Instruments, using CNN, ActaPolytech Hungarica 19(2), 2022.","DOI":"10.12700\/APH.19.2.2022.2.1"},{"issue":"6","key":"10.3233\/JIFS-213243_ref12","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1007\/s11548-015-1194-1","article-title":"Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty","volume":"10","author":"Ahmidi","year":"2015","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"2","key":"10.3233\/JIFS-213243_ref13","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1007\/s11548-011-0650-9","article-title":"et al., Decomposition and analysis of laparoscopic suturing task using tool-motion analysis (TMA): improving the objective assessment","volume":"7","author":"Pagador","year":"2012","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"4","key":"10.3233\/JIFS-213243_ref14","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1177\/1553350618777045","article-title":"Development of a laparoscopic box trainer based on open source hardware and artificial intelligence for objective assessment of surgical psychomotor skills","volume":"25","author":"Alonso-Silverio","year":"2018","journal-title":"Surg Innov"},{"issue":"1","key":"10.3233\/JIFS-213243_ref15","first-page":"1","article-title":"Automation of surgical skill assessment using a three-stage machine learning algorithm","volume":"11","author":"Lavanchy","year":"2021","journal-title":"Sci Rep"},{"issue":"6","key":"10.3233\/JIFS-213243_ref16","doi-asserted-by":"crossref","first-page":"1629","DOI":"10.1016\/j.jsurg.2019.06.011","article-title":"Automated methods of technical skill assessment in surgery: A systematic review","volume":"76","author":"Levin","year":"2019","journal-title":"J Surg Educ"},{"issue":"6","key":"10.3233\/JIFS-213243_ref18","first-page":"2020","article-title":"Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations","volume":"9","author":"Lee","year":"1964","journal-title":"J Clin Med"},{"issue":"11","key":"10.3233\/JIFS-213243_ref19","doi-asserted-by":"crossref","first-page":"1993","DOI":"10.1007\/s11548-019-01977-3","article-title":"Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions","volume":"14","author":"Holden","year":"2019","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"7","key":"10.3233\/JIFS-213243_ref20","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1007\/s11548-019-01995-1","article-title":"Video-based surgical skill assessment using 3D convolutional neural networks","volume":"14","author":"Funke","year":"2019","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"10.3233\/JIFS-213243_ref21","doi-asserted-by":"crossref","first-page":"101994","DOI":"10.1016\/j.media.2021.101994","article-title":"Detection, segmentation, and 3D pose estimation of surgical tools using convolutional neural networks and algebraic geometry","volume":"70","author":"Hasan","year":"2021","journal-title":"Med Image Anal"},{"issue":"5","key":"10.3233\/JIFS-213243_ref22","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1016\/j.jamcollsurg.2020.01.037","article-title":"Automated Surgical Instrument Detection from Laparoscopic Gastrectomy Video Images Using an Open Source Convolutional Neural Network Platform","volume":"230","author":"Yamazaki","year":"2020","journal-title":"J Am Coll Surg"},{"key":"10.3233\/JIFS-213243_ref23","doi-asserted-by":"crossref","first-page":"104384","DOI":"10.1016\/j.compbiomed.2021.104384","article-title":"Automatic tip detection of surgical instruments in biportal endoscopic spine surgery","volume":"133","author":"Cho","year":"2021","journal-title":"Comput Biol Med"},{"issue":"12","key":"10.3233\/JIFS-213243_ref24","doi-asserted-by":"crossref","first-page":"4245","DOI":"10.3390\/app10124245","article-title":"Development of a Deep Learning-Based Algorithm to Detect the Distal End of a Surgical Instrument","volume":"10","author":"Sugimori","year":"2020","journal-title":"Appl Sci"},{"key":"10.3233\/JIFS-213243_ref25","doi-asserted-by":"crossref","first-page":"23748","DOI":"10.1109\/ACCESS.2020.2969885","article-title":"Surgical Tools Detection Based on Modulated Anchoring Network in Laparoscopic Videos","volume":"8","author":"Zhang","year":"2020","journal-title":"IEEE Access"},{"key":"10.3233\/JIFS-213243_ref27","first-page":"2021","article-title":"Real-time 3D tracking of laparoscopy training instruments for assessment and feedback","volume":"8","author":"Gautier","journal-title":"Front Robot AI"},{"issue":"6","key":"10.3233\/JIFS-213243_ref28","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1007\/s11548-019-01958-6","article-title":"Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos","volume":"14","author":"Nwoye","year":"2019","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"23","key":"10.3233\/JIFS-213243_ref30","doi-asserted-by":"crossref","first-page":"6937","DOI":"10.3390\/s20236937","article-title":"LAPKaans: Tool-Motion Tracking and Gripping Force-Sensing Modular Smart Laparoscopic Training System","volume":"20","author":"Olivas-Alanis","year":"2020","journal-title":"Sensors"},{"issue":"8","key":"10.3233\/JIFS-213243_ref38","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1007\/s11548-020-02214-y","article-title":"Object extraction via deep learning-based marker-free tracking framework of surgical instruments for laparoscope-holder robots","volume":"15","author":"Zhang","year":"2020","journal-title":"Int J Comput Assist Radiol Surg"},{"issue":"2","key":"10.3233\/JIFS-213243_ref41","first-page":"14","article-title":"Implementation and comparison of Perturband Observe, and Fuzzy Logic Control onMaximum Power Point Tracking (MPPT) for a Small Satellite","volume":"8","author":"Balal","year":"2021","journal-title":"JSoft Comput Decis Support Syst"},{"key":"10.3233\/JIFS-213243_ref42","first-page":"01","article-title":"Towards Automated Performance Assessment for Laparoscopic Box Trainer using Cross-Stage Partial Network","author":"Alkhamaiseh","journal-title":"2021 Digital Image Computing: Techniques and Applications (DICTA)"}],"container-title":["Journal of Intelligent & Fuzzy Systems"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JIFS-213243","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,26]],"date-time":"2024-11-26T13:55:48Z","timestamp":1732629348000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.medra.org\/servlet\/aliasResolver?alias=iospress&doi=10.3233\/JIFS-213243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,10]]},"references-count":28,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.3233\/jifs-213243","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"type":"print","value":"1064-1246"},{"type":"electronic","value":"1875-8967"}],"subject":[],"published":{"date-parts":[[2022,8,10]]}}}