Abstract
Recent cancer survival prediction approaches have made great strides in analyzing H&E-stained gigapixel whole-slide images. However, methods targeting the immunohistochemistry (IHC) modality remain largely unexplored. We remedy this methodological gap and propose IHCSurv, a new framework that leverages IHC-specific priors to improve downstream survival prediction. We use these priors to guide our model to the most prognostic tissue regions and simultaneously enrich local features. To address drawbacks in recent approaches related to limited spatial context and cross-regional relation modeling, we propose a spatially-constrained spectral clustering algorithm that preserves spatial context alongside an efficient tissue region encoder that facilitates information transfer across tissue regions both within and between images. We evaluate our framework on a multi-stain IHC dataset of pancreatic cancer patients, where IHCSurv markedly outperforms existing state-of-the-art survival prediction methods. Our code is available on Github.
Y. Zhang and H. Chao—Contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Atabansi, C.C., Nie, J., Liu, H., Song, Q., Yan, L., Zhou, X.: A survey of Transformer applications for histopathological image analysis: New developments and future directions. BioMedical Engineering OnLine 22(1), 96 (2023)
Bankhead, P., Loughrey, M.B., Fernández, J.A., Dombrowski, Y., McArt, D.G., Dunne, P.D., McQuaid, S., Gray, R.T., Murray, L.J., Coleman, H.G., et al.: QuPath: Open source software for digital pathology image analysis. Scientific Reports 7(1), 1–7 (2017)
Bug, D., Feuerhake, F., Merhof, D.: Foreground extraction for histopathological whole slide imaging. In: Bildverarbeitung für die Medizin 2015: Algorithmen-Systeme-Anwendungen. pp. 419–424. Springer (2015)
Chen, R.J., Chen, C., Li, Y., Chen, T.Y., Trister, A.D., Krishnan, R.G., Mahmood, F.: Scaling vision Transformers to gigapixel images via hierarchical self-supervised learning. In: CVPR. pp. 16144–16155 (2022)
Chen, R.J., Lu, M.Y., Shaban, M., Chen, C., Chen, T.Y., Williamson, D.F., Mahmood, F.: Whole slide images are 2D point clouds: Context-aware survival prediction using patch-based graph convolutional networks. In: MICCAI. pp. 339–349. Springer (2021)
Di, D., Zhang, J., Lei, F., Tian, Q., Gao, Y.: Big-hypergraph factorization neural network for survival prediction from whole slide image. IEEE Transactions on Image Processing 31, 1149–1160 (2022)
Dwivedi, C., Nofallah, S., Pouryahya, M., Iyer, J., Leidal, K., et al.: Multi stain graph fusion for multimodal integration in pathology. In: CVPR. vol. 2021, pp. 1835–1845 (2021)
Foersch, S., Glasner, C., Woerl, A.C., Eckstein, M., Wagner, D.C., Schulz, S., Kellers, F., Fernandez, A., Tserea, K., Kloth, M., et al.: Multistain deep learning for prediction of prognosis and therapy response in colorectal cancer. Nature Medicine 29(2), 430–439 (2023)
Huang, Z., Chai, H., Wang, R., Wang, H., Yang, Y., Wu, H.: Integration of patch features through self-supervised learning and Transformer for survival analysis on whole slide images. In: MICCAI. pp. 561–570. Springer (2021)
Li, R., Yao, J., Zhu, X., Li, Y., Huang, J.: Graph CNN for survival analysis on whole slide pathological images. In: MICCAI. pp. 174–182. Springer (2018)
Lu, M.Y., Williamson, D.F., Chen, T.Y., Chen, R.J., Barbieri, M., Mahmood, F.: Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5(6), 555–570 (2021)
Mi, H., Sivagnanam, S., Betts, C.B., Liudahl, S.M., Jaffee, E.M., Coussens, L.M., Popel, A.S.: Quantitative spatial profiling of immune populations in pancreatic ductal adenocarcinoma reveals tumor microenvironment heterogeneity and prognostic biomarkers. Cancer Research 82(23), 4359–4372 (2022)
Muhammad, H., Xie, C., Sigel, C.S., Doukas, M., Alpert, L., Simpson, A.L., Fuchs, T.J.: EPIC-survival: End-to-end part inferred clustering for survival analysis, with prognostic stratification boosting. In: Medical Imaging with Deep Learning (2021)
Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. Computer Graphics and Applications 21(5), 34–41 (2001)
Shao, Z., Bian, H., Chen, Y., Wang, Y., Zhang, J., Ji, X., et al.: TransMIL: Transformer based correlated multiple instance learning for whole slide image classification. NeurIPS 34, 2136–2147 (2021)
Shao, Z., Chen, Y., Bian, H., Zhang, J., Liu, G., Zhang, Y.: HVTSurv: Hierarchical vision Transformer for patient-level survival prediction from whole slide image. In: AAAI. vol. 37, pp. 2209–2217 (2023)
Wood, R., Domingo, E., Sirinukunwattana, K., Lafarge, M.W., Koelzer, V.H., Maughan, T.S., Rittscher, J.: Joint prediction of response to therapy, molecular traits, and spatial organisation in colorectal cancer biopsies. In: MICCAI. pp. 758–767. Springer (2023)
Yan, R., Lv, Z., Yang, Z., Lin, S., Zheng, C., Zhang, F.: Sparse and hierarchical Transformer for survival analysis on whole slide images. IEEE Journal of Biomedical and Health Informatics (2023)
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., Huang, J.: Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis 65, 101789 (2020)
Zadeh, S.G., Schmid, M.: Bias in cross-entropy-based training of deep survival networks. TPAMI 43(9), 3126–3137 (2020)
Zhu, X., Yao, J., Huang, J.: Deep convolutional neural network for survival analysis with pathological images. In: BIBM. pp. 544–547. IEEE (2016)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Ethics declarations
Disclosure of Interests
The authors have no competing interests to declare that are relevant to the content of this article.
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y. et al. (2024). IHCSurv: Effective Immunohistochemistry Priors for Cancer Survival Analysis in Gigapixel Multi-stain Whole Slide Images. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15004. Springer, Cham. https://doi.org/10.1007/978-3-031-72083-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-72083-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72082-6
Online ISBN: 978-3-031-72083-3
eBook Packages: Computer ScienceComputer Science (R0)
