Test–Retest Reproducibility Analysis of Lung CT Image Features | Journal of Imaging Informatics in Medicine Skip to main content

Advertisement

Springer Nature Link
Log in

Test–Retest Reproducibility Analysis of Lung CT Image Features

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Quantitative size, shape, and texture features derived from computed tomographic (CT) images may be useful as predictive, prognostic, or response biomarkers in non-small cell lung cancer (NSCLC). However, to be useful, such features must be reproducible, non-redundant, and have a large dynamic range. We developed a set of quantitative three-dimensional (3D) features to describe segmented tumors and evaluated their reproducibility to select features with high potential to have prognostic utility. Thirty-two patients with NSCLC were subjected to unenhanced thoracic CT scans acquired within 15 min of each other under an approved protocol. Primary lung cancer lesions were segmented using semi-automatic 3D region growing algorithms. Following segmentation, 219 quantitative 3D features were extracted from each lesion, corresponding to size, shape, and texture, including features in transformed spaces (laws, wavelets). The most informative features were selected using the concordance correlation coefficient across test–retest, the biological range and a feature independence measure. There were 66 (30.14 %) features with concordance correlation coefficient ≥ 0.90 across test–retest and acceptable dynamic range. Of these, 42 features were non-redundant after grouping features with R 2 Bet ≥ 0.95. These reproducible features were found to be predictive of radiological prognosis. The area under the curve (AUC) was 91 % for a size-based feature and 92 % for the texture features (runlength, laws). We tested the ability of image features to predict a radiological prognostic score on an independent NSCLC (39 adenocarcinoma) samples, the AUC for texture features (runlength emphasis, energy) was 0.84 while the conventional size-based features (volume, longest diameter) was 0.80. Test–retest and correlation analyses have identified non-redundant CT image features with both high intra-patient reproducibility and inter-patient biological range. Thus making the case that quantitative image features are informative and prognostic biomarkers for NSCLC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Nguyen T, Rangayyan R: Shape analysis of breast masses in mammograms via the fractal dimension. Conf Proc IEEE Eng Med Biol Soc 3:3210–3213, 2005

    PubMed  Google Scholar 

  2. Schuster DP: The opportunities and challenges of developing imaging biomarkers to study lung function and disease. Am J Respir Crit Care Med 176(3):224–230, 2007

    Article  PubMed  Google Scholar 

  3. Suzuki C, Jacobsson H, Hatschek T, et al: Radiologic measurements of tumor response to treatment: practical approaches and limitations. Radiographics 28(2):329–344, 2008

    Article  PubMed  Google Scholar 

  4. Tuma RS: Sometimes size doesn't matter: reevaluating RECIST and tumor response rate endpoints. J Natl Cancer Inst 98(18):1272–1274, 2006

    Article  PubMed  Google Scholar 

  5. Ganeshan B, Abaleke S, Young RC, et al: Texture analysis of non-small cell lung cancer on unenhanced computed tomography: initial evidence for a relationship with tumour glucose metabolism and stage. Cancer Imaging 10:137–143, 2010

    Article  PubMed Central  PubMed  Google Scholar 

  6. Way TW, Sahiner B, Chan HP, et al: Computer-aided diagnosis of pulmonary nodules on CT scans: improvement of classification performance with nodule surface features. Med Phys 36(7):3086–3098, 2009

    Article  PubMed Central  PubMed  Google Scholar 

  7. Samala R, Moreno W, You Y, et al: A novel approach to nodule feature optimization on thin section thoracic CT. Acad Radiol 16(4):418–427, 2009

    Article  PubMed  Google Scholar 

  8. Lee MC, Boroczky L, Sungur-Stasik K, et al: Computer-aided diagnosis of pulmonary nodules using a two-step approach for feature selection and classifier ensemble construction. Artif Intell Med 50(1):43–53, 2010

    Article  PubMed  Google Scholar 

  9. Zhu Y, Tan Y, Hua Y, et al: Feature selection and performance evaluation of support vector machine (SVM)-based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. J Digit Imaging 23(1):51–65, 2010

    Article  PubMed Central  PubMed  Google Scholar 

  10. Al-Kadi O, Watson D: Texture analysis of aggressive and nonaggressive lung tumor CE CT images. IEEE Trans Biomed Eng 55(7):1822–1830, 2008

    Article  PubMed  Google Scholar 

  11. Kido S, Kuriyama K, Higashiyama M, et al: Fractal analysis of internal and peripheral textures of small peripheral bronchogenic carcinomas in thin-section computed tomography: comparison of bronchioloalveolar cell carcinomas with nonbronchioloalveolar cell carcinomas. J Comput Assist Tomogr 27(1):56–61, 2003

    Article  PubMed  Google Scholar 

  12. Segal E, Sirlin CB, Ooi C, et al: Decoding global gene expression programs in liver cancer by noninvasive imaging. Nat Biotechnol 25(6):675–680, 2007

    Article  CAS  PubMed  Google Scholar 

  13. Buckler AJ, Mozley PD, Schwartz L, et al: Volumetric CT in lung cancer: an example for the qualification of imaging as a biomarker. Acad Radiol 17(1):107–115, 2010

    Article  PubMed  Google Scholar 

  14. America RSoN: Quantitative imaging biomarker alliance for volumetric CT image analysis: roadmap for a staged validation plan, 2010

  15. Zhao B, James LP, Moskowitz CS, et al: Evaluating variability in tumor measurements from same-day repeat CT scans of patients with non-small cell lung cancer. Radiology 252(1):263–272, 2009

    Article  PubMed Central  PubMed  Google Scholar 

  16. RIDER. The Reference Image Database to Evaluate Therapy Response. Available at: https://wiki.cancerimagingarchive.net/display/Public/RIDER+Collections;jsessionid=C78203F71E49C7EA3A43E0D213CE5555. Accessed 24 Jun 2014

  17. Gu Y, Kumar V, Hall LO, et al: Automated delineation of lung tumors from CT images using a single click ensemble segmentation approach. Pattern Recogn 46(3):692–702, 2013

    Article  Google Scholar 

  18. NBIA. National Biomedical Imaging Archive. Available at: https://imaging.nci.nih.gov/ncia. Accessed 30 June 2014

  19. Definiens. Definiens AG, Munchen, Germany. Available at: http://www.definiens.com/product-services/definiens-xd-product-suite.html. Accessed 30 June 2014

  20. Athelogou M, Schmidt G, Schaepe A, et al: Cognition network technology—a novel multimodal image analysis technique for automatic identification and quantification of biological image contents. In: Shorte SL, Frischknecht F Eds. Book cognition network technology—a novel multimodal image analysis technique for automatic identification and quantification of biological image contents. Springer-Verlag, New York City, 2007, pp 407–422

    Google Scholar 

  21. Baatz M, Zimmermann J, Blackmore CG: Automated analysis and detailed quantification of biomedical images using Definiens Congnition Network Technology. Comb Chem High Throughput Screen 12(9):908–916, 2009

    Article  CAS  PubMed  Google Scholar 

  22. Bendtsen C, Kietzmann M, Korn R, Mozley P, Schmidt G, Binnig G: X-ray computed tomography: semiautomated volumetric analysis of late-stage lung tumors as a basis for response assessments. Int J Biomed Imaging, vol 2011, 2011

  23. Basu S, Hall LO, Goldgof DB, et al: Developing a classifier model for lung tumors in ct-scan images. IEEE Intl Conf on Systems, Man and Cybernetics, (SMC 2011), Anchorage, Alaska, 2011

  24. Lin LI-K: A concordance correlation coefficient to evaluate reproducibility. Biometrics 45:13, 1989

    Article  Google Scholar 

  25. RGD Steel JT: Principles and procedures of statistics. McGraw-Hill, New York, 1960

    Google Scholar 

  26. Colin C, Frank AW, Gramaji H, et al: An R-square measured of goodness of fit for some common nonlinear regression models. J Econ 77(2):1790–1792, 1997

    Google Scholar 

  27. Aoki T, Tomoda Y, Watanabe H, et al: Peripheral lung adenocarcinoma: correlation of thin-section CT findings with histologic prognostic factors and survival. Radiology 220(3):803–809, 2001

    Article  CAS  PubMed  Google Scholar 

  28. Takashima S MY, Hasegawa M, Saito A, Haniuda M, Kadoya M. High-resolution CT features: prognostic significance in peripheral lung adenocarcinoma with bronchioloalveolar carcinoma components. Respiration: Int Rev Thorac Dis 70(1), 2003

  29. Subramanian J, Simon R: Gene exression-based signature in lung cancer: ready for clinical use? JNCI 102(7):464–474, 2010

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  30. Jain AK, Zongker D: Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal 19(2):153–158, 1997

    Article  Google Scholar 

  31. Pudil P, Novovičová J, Kittler J: Floating search methods in feature selection. Pattern Recogn Lett 15:1119–1125, 1994

    Article  Google Scholar 

  32. Saeys Y, Inza I: A review of feature selection techniques in bioinformatics. Bioinformatics 23:2507–2517, 2007

    Article  CAS  PubMed  Google Scholar 

  33. Landis JR, Koch G: The measurement of observer agreement for categorical data. Biometrics 33:159–174, 1977

    Article  CAS  PubMed  Google Scholar 

  34. Ganeshan B, Panayiotou E, Burnand K, et al: Tumour heterogeneity in non-small cell lung carcinoma assessed by CT texture analysis: a potential marker of survival. Eur Radiol 22(4):796–802, 2012

    Article  PubMed  Google Scholar 

  35. Yanagawa M, Tanaka Y, Kusumoto M, et al: Automated assessment of malignant degree of small peripheral adenocarcinomas using volumetric CT data: correlation with pathologic prognostic factors. Lung Cancer 70(3):286–294, 2010

    Article  PubMed  Google Scholar 

  36. John S: A direct approach to false discovery rate. J R Stat Soc B 64(3):479–498, 2002

    Article  Google Scholar 

  37. Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc 57(1):289–300, 1995

    Google Scholar 

  38. Zhao B, Oxnard G, Moskowitz CS, et al: A pilot study of volume measurement as a method of tumor response evaluation to aid biomarker development. Clin Cancer Res 16(18):4647–4653, 2010

    Article  CAS  PubMed Central  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Cancer Imaging and Metabolism, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA

    Yoganand Balagurunathan, Virendra Kumar, Yuhua Gu, Hua Wang, Ying Liu & Robert J. Gillies

  2. Biostatistics and Bioinformatics, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA

    Jongphil Kim & Steven Eschrich

  3. Department of Computer Science and Engineering, University of South Florida, Tampa, FL, USA

    Dmitry B. Goldgof, Lawrence O. Hall & Satrajit Basu

  4. Definiens AG, Bernhard-Wicki-Straße 5, 80636, Munchen, Germany

    Rene Korn

  5. Department of Radiology, Columbia University, New York, NY, USA

    Binsheng Zhao & Lawrence H. Schwartz

  6. Radiology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, FL, USA

    Robert A. Gatenby & Robert J. Gillies

  7. Department of Radiology, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China

    Hua Wang & Ying Liu

  8. Experimental Imaging Program, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Drive, SRB-2, Tampa, FL, 33612, USA

    Robert J. Gillies

Authors
  1. Yoganand Balagurunathan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Virendra Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhua Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jongphil Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dmitry B. Goldgof
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lawrence O. Hall
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Rene Korn
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Binsheng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Lawrence H. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Satrajit Basu
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Steven Eschrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Robert A. Gatenby
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Robert J. Gillies
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert J. Gillies.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 626 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balagurunathan, Y., Kumar, V., Gu, Y. et al. Test–Retest Reproducibility Analysis of Lung CT Image Features. J Digit Imaging 27, 805–823 (2014). https://doi.org/10.1007/s10278-014-9716-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-014-9716-x

Keywords

Search

Navigation