Abstract
Lymph node (LN) assessment is a critical yet very challenging task in the routine clinical workflow of radiology and oncology. Accurate LN analysis is essential for cancer diagnosis, staging and treatment planning. Finding scatteredly distributed, low-contrast clinically relevant LNs in 3D CT is difficult even for experienced physicians under high inter-observer variations. Previous automatic LN detection typically yields limited recall and high false positives (FPs) due to adjacent anatomies with similar image intensities, shapes or textures (vessels, muscles, esophagus, etc.). In this work, we propose a new LN DEtection TRansformer, named LN-DETR, with location debiased query selection and contrastive query learning to enhance the representation ability of LN queries, important to increase the detection sensitivity and reduce FPs or duplicates. We also enhance LN-DETR by adapting an efficient multi-scale 2.5D fusion scheme to incorporate the 3D context. Trained and tested on 3D CT scans of 1067 patients (with \(10,000+\) labeled LNs) via combining seven LN datasets from different body parts (neck, chest, and abdomen) and pathologies/cancers, our method significantly improves the performance of previous leading methods by >4\(\sim \)5% average recall at the same FP rates in both internal and external testing. We further evaluate on the universal lesion detection task using DeepLesion benchmark, and our method achieves the top performance of 88.46% averaged recall, compared with other leading reported results.
Q. Yu, Y. Wang—Equal contribution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ardila, D., et al.: End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat. Med. 25(6), 954–961 (2019)
Barbu, A., Suehling, M., Xu, X., Liu, D., Zhou, S.K., Comaniciu, D.: Automatic detection and segmentation of lymph nodes from CT data. IEEE Trans. Med. Imaging 31(2), 240–250 (2011)
Baumgartner, M., Jäger, P.F., Isensee, F., Maier-Hein, K.H.: nnDetection: a self-configuring method for medical object detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 530–539. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_51
Bouget, D., Jørgensen, A., Kiss, G., Leira, H.O., Langø, T.: Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging. Int. J. Comput. Assist. Radiol. Surg. 14, 977–986 (2019)
Bouget, D., Pedersen, A., Vanel, J., Leira, H.O., Langø, T.: Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guiding. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 11(1), 44–58 (2023)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13
Chang, J.M., Leung, J.W., Moy, L., Ha, S.M., Moon, W.K.: Axillary nodal evaluation in breast cancer: state of the art. Radiology 295(3), 500–515 (2020)
Chao, C.-H., et al.: Lymph node gross tumor volume detection in oncology imaging via relationship learning using graph neural network. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 772–782. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_75
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)
Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R.: Masked-attention mask transformer for universal image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1290–1299 (2022)
Cheng, C.T., et al.: A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs. Nat. Commun. 12(1), 1066 (2021)
Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392(10162), 2388–2396 (2018)
Detterbeck, F.C., Boffa, D.J., Kim, A.W., Tanoue, L.T.: The eighth edition lung cancer stage classification. Chest 151(1), 193–203 (2017)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
El-Sherief, A.H., Lau, C.T., Wu, C.C., Drake, R.L., Abbott, G.F., Rice, T.W.: International association for the study of lung cancer (IASLC) lymph node map: radiologic review with CT illustration. Radiographics 34(6), 1680–1691 (2014)
Feuerstein, M., Glocker, B., Kitasaka, T., Nakamura, Y., Iwano, S., Mori, K.: Mediastinal atlas creation from 3-d chest computed tomography images: application to automated detection and station mapping of lymph nodes. Med. Image Anal. 16(1), 63–74 (2012)
Feulner, J., Zhou, S.K., Hammon, M., Hornegger, J., Comaniciu, D.: Lymph node detection and segmentation in chest CT data using discriminative learning and a spatial prior. Med. Image Anal. 17(2), 254–270 (2013)
Guo, D., et al.: Thoracic lymph node segmentation in CT imaging via lymph node station stratification and size encoding. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. LNCS, vol. 13435, pp. 55–65. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16443-9_6
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)
Jia, D., et al.: DETRs with hybrid matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19702–19712 (2023)
Jiang, B., Luo, R., Mao, J., Xiao, T., Jiang, Y.: Acquisition of localization confidence for accurate object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 784–799 (2018)
Jin, D., et al.: Artificial intelligence in radiology. In: Artificial Intelligence in Medicine, pp. 265–289. Elsevier (2021)
Kann, B.H., et al.: Multi-institutional validation of deep learning for pretreatment identification of extranodal extension in head and neck squamous cell carcinoma. J. Clin. Oncol. 38(12), 1304–1311 (2020)
Khosla, P., et al.: Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33, 18661–18673 (2020)
Kuhn, H.W.: The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)
Li, F., Zhang, H., Liu, S., Guo, J., Ni, L.M., Zhang, L.: Dn-DETR: accelerate DETR training by introducing query denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13619–13627 (2022)
Li, F., et al.: Mask DINO: towards a unified transformer-based framework for object detection and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3041–3050 (2023)
Li, H., Chen, L., Han, H., Chi, Y., Zhou, S.K.: Conditional training with bounding map for universal lesion detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 141–152. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_14
Li, H., Chen, L., Han, H., Kevin Zhou, S.: SATr: slice attention with transformer for universal lesion detection. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. LNCS, vol. 13433, pp. 163–174. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16437-8_16
Li, J., Dai, H., Shao, L., Ding, Y.: From voxel to point: iou-guided 3d object detection for point cloud with voxel-to-point decoder. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4622–4631 (2021)
Li, J., Chen, J., Tang, Y., Wang, C., Landman, B.A., Zhou, S.K.: Transforming medical imaging with transformers? A comparative review of key properties, current progresses, and future perspectives. Med. Image Anal. 102762 (2023)
Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., Yang, J.: Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection. Adv. Neural Inf. Process. Syst. 33, 21002–21012 (2020)
Li, Z., Zhang, S., Zhang, J., Huang, K., Wang, Y., Yu, Y.: MVP-Net: multi-view FPN with position-aware attention for deep universal lesion detection. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 13–21. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_2
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)
Liu, J., et al.: Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest. Med. Phys. 43(7), 4362–4374 (2016)
Liu, S., et al.: Dab-detr: dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329 (2022)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Mathai, T.S., et al.: Detection of lymph nodes in T2 MRI using neural network ensembles. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds.) MLMI 2021. LNCS, vol. 12966, pp. 682–691. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87589-3_70
McLoud, T., et al.: Bronchogenic carcinoma: analysis of staging in the mediastinum with CT by correlative lymph node mapping and sampling. Radiology 182(2), 319–323 (1992)
Meng, D., et al.: Conditional DETR for fast training convergence. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3651–3660 (2021)
Mitani, A., et al.: Detection of anaemia from retinal fundus images via deep learning. Nat. Biomed. Eng. 4(1), 18–27 (2020)
Mountain, C.F., Dresler, C.M.: Regional lymph node classification for lung cancer staging. Chest 111(6), 1718–1723 (1997)
Oda, H., et al.: Dense volumetric detection and segmentation of mediastinal lymph nodes in chest CT images. In: Medical Imaging 2018: Computer-Aided Diagnosis, vol. 10575, p. 1057502. SPIE (2018)
Oord, A.V.D., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Pu, Y., et al.: Rank-detr for high quality object detection. Adv. Neural Inf. Process. Syst. 36 (2024)
Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)
Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28 (2015)
Rice, T.W., Ishwaran, H., Ferguson, M.K., Blackstone, E.H., Goldstraw, P.: Cancer of the esophagus and esophagogastric junction: an eighth edition staging primer. J. Thorac. Oncol. 12(1), 36–42 (2017)
Roth, H.R., et al.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2015)
Schwartz, L., et al.: Evaluation of lymph nodes with RECIST 1.1. Eur. J. Cancer 45(2), 261–267 (2009)
Shamshad, F., et al.: Transformers in medical imaging: a survey. Med. Image Anal. 102802 (2023)
Sheoran, M., Dani, M., Sharma, M., Vig, L.: DKMA-ULD: domain knowledge augmented multi-head attention based robust universal lesion detection. arXiv preprint arXiv:2203.06886 (2022)
Takeuchi, H., et al.: Validation study of radio-guided sentinel lymph node navigation in esophageal cancer. Ann. Surg. 249(5), 757–763 (2009)
Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627–9636 (2019)
Wang, S., et al.: Global-local attention network with multi-task uncertainty loss for abnormal lymph node detection in MR images. Med. Image Anal. 77, 102345 (2022)
Wang, Y., Zhang, X., Yang, T., Sun, J.: Anchor detr: query design for transformer-based detector. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2567–2575 (2022)
Wu, C., et al.: Integrating features from lymph node stations for metastatic lymph node detection. Comput. Med. Imaging Graph. 101, 102108 (2022)
Yan, K., Bagheri, M., Summers, R.M.: 3D context enhanced region-based convolutional neural network for end-to-end lesion detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 511–519. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_58
Yan, K., et al.: Learning from multiple datasets with heterogeneous and partial labels for universal lesion detection in CT. IEEE Trans. Med. Imaging 40(10), 2759–2770 (2021)
Yan, K., et al.: Anatomy-aware lymph node detection in chest CT using implicit station stratification. arXiv preprint arXiv:2307.15271 (2023)
Yan, K., et al.: MULAN: multitask universal lesion analysis network for joint lesion detection, tagging, and segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 194–202. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_22
Yan, K., Wang, X., Lu, L., Summers, R.M.: Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5(3), 036501–036501 (2018)
Yang, J., He, Y., Kuang, K., Lin, Z., Pfister, H., Ni, B.: Asymmetric 3D context fusion for universal lesion detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 571–580. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_55
Yao, Z., Ai, J., Li, B., Zhang, C.: Efficient DETR: improving end-to-end object detector with dense prior. arXiv preprint arXiv:2104.01318 (2021)
Ye, M., et al.: Cascade-detr: delving into high-quality universal object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6704–6714 (2023)
Zhang, H., et al.: DINO: detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605 (2022)
Zhang, H., et al.: MP-Former: mask-piloted transformer for image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18074–18083 (2023)
Zhang, H., Wang, Y., Dayoub, F., Sunderhauf, N.: Varifocalnet: an iou-aware dense object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8514–8523 (2021)
Zhang, S., Li, Z., Zhou, H.Y., Ma, J., Yu, Y.: Advancing 3d medical image analysis with variable dimension transform based supervised 3d pre-training. Neurocomputing 529, 11–22 (2023)
Zhao, P., Li, H., Jin, R., Zhou, S.K.: DiffULD: diffusive universal lesion detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2023). https://api.semanticscholar.org/CorpusID:257771843
Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)
Zhu, Z., et al.: Lymph node gross tumor volume detection and segmentation via distance-based gating using 3D CT/PET imaging in radiotherapy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 753–762. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_73
Zhu, Z., et al.: Detecting scatteredly-distributed, small, andcritically important objects in 3D oncologyimaging via decision stratification. arXiv preprint arXiv:2005.13705 (2020)
Zlocha, M., Dou, Q., Glocker, B.: Improving RetinaNet for CT lesion detection with dense masks from weak RECIST labels. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 402–410. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_45
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yu, Q. et al. (2025). Effective Lymph Nodes Detection in CT Scans Using Location Debiased Query Selection and Contrastive Query Representation in Transformer. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15100. Springer, Cham. https://doi.org/10.1007/978-3-031-72946-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-72946-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72945-4
Online ISBN: 978-3-031-72946-1
eBook Packages: Computer ScienceComputer Science (R0)