Effective Lymph Nodes Detection in CT Scans Using Location Debiased Query Selection and Contrastive Query Representation in Transformer | SpringerLink
Skip to main content

Effective Lymph Nodes Detection in CT Scans Using Location Debiased Query Selection and Contrastive Query Representation in Transformer

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15100))

Included in the following conference series:

  • 170 Accesses

Abstract

Lymph node (LN) assessment is a critical yet very challenging task in the routine clinical workflow of radiology and oncology. Accurate LN analysis is essential for cancer diagnosis, staging and treatment planning. Finding scatteredly distributed, low-contrast clinically relevant LNs in 3D CT is difficult even for experienced physicians under high inter-observer variations. Previous automatic LN detection typically yields limited recall and high false positives (FPs) due to adjacent anatomies with similar image intensities, shapes or textures (vessels, muscles, esophagus, etc.). In this work, we propose a new LN DEtection TRansformer, named LN-DETR, with location debiased query selection and contrastive query learning to enhance the representation ability of LN queries, important to increase the detection sensitivity and reduce FPs or duplicates. We also enhance LN-DETR by adapting an efficient multi-scale 2.5D fusion scheme to incorporate the 3D context. Trained and tested on 3D CT scans of 1067 patients (with \(10,000+\) labeled LNs) via combining seven LN datasets from different body parts (neck, chest, and abdomen) and pathologies/cancers, our method significantly improves the performance of previous leading methods by >4\(\sim \)5% average recall at the same FP rates in both internal and external testing. We further evaluate on the universal lesion detection task using DeepLesion benchmark, and our method achieves the top performance of 88.46% averaged recall, compared with other leading reported results.

Q. Yu, Y. Wang—Equal contribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8465
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/CSCYQJ/ECCV24_LN_DETR.

References

  1. Ardila, D., et al.: End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography. Nat. Med. 25(6), 954–961 (2019)

    Article  Google Scholar 

  2. Barbu, A., Suehling, M., Xu, X., Liu, D., Zhou, S.K., Comaniciu, D.: Automatic detection and segmentation of lymph nodes from CT data. IEEE Trans. Med. Imaging 31(2), 240–250 (2011)

    Article  Google Scholar 

  3. Baumgartner, M., Jäger, P.F., Isensee, F., Maier-Hein, K.H.: nnDetection: a self-configuring method for medical object detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 530–539. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_51

    Chapter  Google Scholar 

  4. Bouget, D., Jørgensen, A., Kiss, G., Leira, H.O., Langø, T.: Semantic segmentation and detection of mediastinal lymph nodes and anatomical structures in CT data for lung cancer staging. Int. J. Comput. Assist. Radiol. Surg. 14, 977–986 (2019)

    Article  Google Scholar 

  5. Bouget, D., Pedersen, A., Vanel, J., Leira, H.O., Langø, T.: Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guiding. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 11(1), 44–58 (2023)

    Article  Google Scholar 

  6. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13

    Chapter  Google Scholar 

  7. Chang, J.M., Leung, J.W., Moy, L., Ha, S.M., Moon, W.K.: Axillary nodal evaluation in breast cancer: state of the art. Radiology 295(3), 500–515 (2020)

    Article  Google Scholar 

  8. Chao, C.-H., et al.: Lymph node gross tumor volume detection in oncology imaging via relationship learning using graph neural network. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 772–782. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_75

    Chapter  Google Scholar 

  9. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)

    Google Scholar 

  10. Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., Girdhar, R.: Masked-attention mask transformer for universal image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1290–1299 (2022)

    Google Scholar 

  11. Cheng, C.T., et al.: A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs. Nat. Commun. 12(1), 1066 (2021)

    Article  Google Scholar 

  12. Chilamkurthy, S., et al.: Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet 392(10162), 2388–2396 (2018)

    Article  Google Scholar 

  13. Detterbeck, F.C., Boffa, D.J., Kim, A.W., Tanoue, L.T.: The eighth edition lung cancer stage classification. Chest 151(1), 193–203 (2017)

    Google Scholar 

  14. Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

  15. El-Sherief, A.H., Lau, C.T., Wu, C.C., Drake, R.L., Abbott, G.F., Rice, T.W.: International association for the study of lung cancer (IASLC) lymph node map: radiologic review with CT illustration. Radiographics 34(6), 1680–1691 (2014)

    Article  Google Scholar 

  16. Feuerstein, M., Glocker, B., Kitasaka, T., Nakamura, Y., Iwano, S., Mori, K.: Mediastinal atlas creation from 3-d chest computed tomography images: application to automated detection and station mapping of lymph nodes. Med. Image Anal. 16(1), 63–74 (2012)

    Article  Google Scholar 

  17. Feulner, J., Zhou, S.K., Hammon, M., Hornegger, J., Comaniciu, D.: Lymph node detection and segmentation in chest CT data using discriminative learning and a spatial prior. Med. Image Anal. 17(2), 254–270 (2013)

    Article  Google Scholar 

  18. Guo, D., et al.: Thoracic lymph node segmentation in CT imaging via lymph node station stratification and size encoding. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. LNCS, vol. 13435, pp. 55–65. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16443-9_6

  19. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    Google Scholar 

  20. Jia, D., et al.: DETRs with hybrid matching. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19702–19712 (2023)

    Google Scholar 

  21. Jiang, B., Luo, R., Mao, J., Xiao, T., Jiang, Y.: Acquisition of localization confidence for accurate object detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 784–799 (2018)

    Google Scholar 

  22. Jin, D., et al.: Artificial intelligence in radiology. In: Artificial Intelligence in Medicine, pp. 265–289. Elsevier (2021)

    Google Scholar 

  23. Kann, B.H., et al.: Multi-institutional validation of deep learning for pretreatment identification of extranodal extension in head and neck squamous cell carcinoma. J. Clin. Oncol. 38(12), 1304–1311 (2020)

    Article  Google Scholar 

  24. Khosla, P., et al.: Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33, 18661–18673 (2020)

    Google Scholar 

  25. Kuhn, H.W.: The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2(1–2), 83–97 (1955)

    Article  MathSciNet  Google Scholar 

  26. Li, F., Zhang, H., Liu, S., Guo, J., Ni, L.M., Zhang, L.: Dn-DETR: accelerate DETR training by introducing query denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13619–13627 (2022)

    Google Scholar 

  27. Li, F., et al.: Mask DINO: towards a unified transformer-based framework for object detection and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3041–3050 (2023)

    Google Scholar 

  28. Li, H., Chen, L., Han, H., Chi, Y., Zhou, S.K.: Conditional training with bounding map for universal lesion detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 141–152. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_14

    Chapter  Google Scholar 

  29. Li, H., Chen, L., Han, H., Kevin Zhou, S.: SATr: slice attention with transformer for universal lesion detection. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. LNCS, vol. 13433, pp. 163–174. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16437-8_16

  30. Li, J., Dai, H., Shao, L., Ding, Y.: From voxel to point: iou-guided 3d object detection for point cloud with voxel-to-point decoder. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4622–4631 (2021)

    Google Scholar 

  31. Li, J., Chen, J., Tang, Y., Wang, C., Landman, B.A., Zhou, S.K.: Transforming medical imaging with transformers? A comparative review of key properties, current progresses, and future perspectives. Med. Image Anal. 102762 (2023)

    Google Scholar 

  32. Li, X., Wang, W., Wu, L., Chen, S., Hu, X., Li, J., Tang, J., Yang, J.: Generalized focal loss: learning qualified and distributed bounding boxes for dense object detection. Adv. Neural Inf. Process. Syst. 33, 21002–21012 (2020)

    Google Scholar 

  33. Li, Z., Zhang, S., Zhang, J., Huang, K., Wang, Y., Yu, Y.: MVP-Net: multi-view FPN with position-aware attention for deep universal lesion detection. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 13–21. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_2

    Chapter  Google Scholar 

  34. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

    Google Scholar 

  35. Liu, J., et al.: Mediastinal lymph node detection and station mapping on chest CT using spatial priors and random forest. Med. Phys. 43(7), 4362–4374 (2016)

    Article  Google Scholar 

  36. Liu, S., et al.: Dab-detr: dynamic anchor boxes are better queries for DETR. arXiv preprint arXiv:2201.12329 (2022)

  37. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  38. Mathai, T.S., et al.: Detection of lymph nodes in T2 MRI using neural network ensembles. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds.) MLMI 2021. LNCS, vol. 12966, pp. 682–691. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87589-3_70

    Chapter  Google Scholar 

  39. McLoud, T., et al.: Bronchogenic carcinoma: analysis of staging in the mediastinum with CT by correlative lymph node mapping and sampling. Radiology 182(2), 319–323 (1992)

    Article  Google Scholar 

  40. Meng, D., et al.: Conditional DETR for fast training convergence. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3651–3660 (2021)

    Google Scholar 

  41. Mitani, A., et al.: Detection of anaemia from retinal fundus images via deep learning. Nat. Biomed. Eng. 4(1), 18–27 (2020)

    Article  Google Scholar 

  42. Mountain, C.F., Dresler, C.M.: Regional lymph node classification for lung cancer staging. Chest 111(6), 1718–1723 (1997)

    Article  Google Scholar 

  43. Oda, H., et al.: Dense volumetric detection and segmentation of mediastinal lymph nodes in chest CT images. In: Medical Imaging 2018: Computer-Aided Diagnosis, vol. 10575, p. 1057502. SPIE (2018)

    Google Scholar 

  44. Oord, A.V.D., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

  45. Pu, Y., et al.: Rank-detr for high quality object detection. Adv. Neural Inf. Process. Syst. 36 (2024)

    Google Scholar 

  46. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)

    Google Scholar 

  47. Redmon, J., Farhadi, A.: Yolov3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)

  48. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. Adv. Neural Inf. Process. Syst. 28 (2015)

    Google Scholar 

  49. Rice, T.W., Ishwaran, H., Ferguson, M.K., Blackstone, E.H., Goldstraw, P.: Cancer of the esophagus and esophagogastric junction: an eighth edition staging primer. J. Thorac. Oncol. 12(1), 36–42 (2017)

    Google Scholar 

  50. Roth, H.R., et al.: Improving computer-aided detection using convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2015)

    Article  Google Scholar 

  51. Schwartz, L., et al.: Evaluation of lymph nodes with RECIST 1.1. Eur. J. Cancer 45(2), 261–267 (2009)

    Google Scholar 

  52. Shamshad, F., et al.: Transformers in medical imaging: a survey. Med. Image Anal. 102802 (2023)

    Google Scholar 

  53. Sheoran, M., Dani, M., Sharma, M., Vig, L.: DKMA-ULD: domain knowledge augmented multi-head attention based robust universal lesion detection. arXiv preprint arXiv:2203.06886 (2022)

  54. Takeuchi, H., et al.: Validation study of radio-guided sentinel lymph node navigation in esophageal cancer. Ann. Surg. 249(5), 757–763 (2009)

    Article  Google Scholar 

  55. Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627–9636 (2019)

    Google Scholar 

  56. Wang, S., et al.: Global-local attention network with multi-task uncertainty loss for abnormal lymph node detection in MR images. Med. Image Anal. 77, 102345 (2022)

    Article  Google Scholar 

  57. Wang, Y., Zhang, X., Yang, T., Sun, J.: Anchor detr: query design for transformer-based detector. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 2567–2575 (2022)

    Google Scholar 

  58. Wu, C., et al.: Integrating features from lymph node stations for metastatic lymph node detection. Comput. Med. Imaging Graph. 101, 102108 (2022)

    Article  Google Scholar 

  59. Yan, K., Bagheri, M., Summers, R.M.: 3D context enhanced region-based convolutional neural network for end-to-end lesion detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 511–519. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_58

    Chapter  Google Scholar 

  60. Yan, K., et al.: Learning from multiple datasets with heterogeneous and partial labels for universal lesion detection in CT. IEEE Trans. Med. Imaging 40(10), 2759–2770 (2021)

    Article  Google Scholar 

  61. Yan, K., et al.: Anatomy-aware lymph node detection in chest CT using implicit station stratification. arXiv preprint arXiv:2307.15271 (2023)

  62. Yan, K., et al.: MULAN: multitask universal lesion analysis network for joint lesion detection, tagging, and segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 194–202. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_22

    Chapter  Google Scholar 

  63. Yan, K., Wang, X., Lu, L., Summers, R.M.: Deeplesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5(3), 036501–036501 (2018)

    Article  Google Scholar 

  64. Yang, J., He, Y., Kuang, K., Lin, Z., Pfister, H., Ni, B.: Asymmetric 3D context fusion for universal lesion detection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 571–580. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_55

    Chapter  Google Scholar 

  65. Yao, Z., Ai, J., Li, B., Zhang, C.: Efficient DETR: improving end-to-end object detector with dense prior. arXiv preprint arXiv:2104.01318 (2021)

  66. Ye, M., et al.: Cascade-detr: delving into high-quality universal object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6704–6714 (2023)

    Google Scholar 

  67. Zhang, H., et al.: DINO: detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605 (2022)

  68. Zhang, H., et al.: MP-Former: mask-piloted transformer for image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18074–18083 (2023)

    Google Scholar 

  69. Zhang, H., Wang, Y., Dayoub, F., Sunderhauf, N.: Varifocalnet: an iou-aware dense object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8514–8523 (2021)

    Google Scholar 

  70. Zhang, S., Li, Z., Zhou, H.Y., Ma, J., Yu, Y.: Advancing 3d medical image analysis with variable dimension transform based supervised 3d pre-training. Neurocomputing 529, 11–22 (2023)

    Article  Google Scholar 

  71. Zhao, P., Li, H., Jin, R., Zhou, S.K.: DiffULD: diffusive universal lesion detection. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2023). https://api.semanticscholar.org/CorpusID:257771843

  72. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159 (2020)

  73. Zhu, Z., et al.: Lymph node gross tumor volume detection and segmentation via distance-based gating using 3D CT/PET imaging in radiotherapy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12267, pp. 753–762. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59728-3_73

    Chapter  Google Scholar 

  74. Zhu, Z., et al.: Detecting scatteredly-distributed, small, andcritically important objects in 3D oncologyimaging via decision stratification. arXiv preprint arXiv:2005.13705 (2020)

  75. Zlocha, M., Dou, Q., Glocker, B.: Improving RetinaNet for CT lesion detection with dense masks from weak RECIST labels. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11769, pp. 402–410. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32226-7_45

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Qinji Yu .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 5710 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yu, Q. et al. (2025). Effective Lymph Nodes Detection in CT Scans Using Location Debiased Query Selection and Contrastive Query Representation in Transformer. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15100. Springer, Cham. https://doi.org/10.1007/978-3-031-72946-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72946-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72945-4

  • Online ISBN: 978-3-031-72946-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics