Abstract
Spatial transcriptomics (ST) has made it possible to link local spatial gene expression with the properties of tissue, which is very helpful to the research of histopathology and pathology. To obtain more ST data, we utilize deep learning methods to predict gene expression on tissue slide images. Considering the importance of the dependence of local tissue images on their neighborhoods, we propose the novel Multi-Neighborhood Network (MNN), composed of down-sampling module and vanilla Transformer blocks. Moreover, to satisfy the needs of architecture and address the computational and parameter challenges arising from it, we introduce dual-scale attention block and reconstructing attention block. To demonstrate the effectiveness of this network structure and the superiority of attention mechanisms, we conducted comparative experiments, where MNN achieved optimal PCC@M \((1\times 10^1)\) of 9.23 and 8.54 for the lung cancer and mouse brain datasets of 10x Genomics website, respectively, outperforming several state-of-the-art (SOTA) methods. This reveals the superiority of our method in terms of spatial gene prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, M., Zhang, B., Topatana, W., Cao, J., Zhu, H., Juengpanich, S., Mao, Q., Yu, H., Cai, X.: Classification and mutation prediction based on histopathology h &e images in liver cancer using deep learning. NPJ Precis. Oncol. 4(1), 14 (2020)
Chen, S., Xie, E., Ge, C., Chen, R., Liang, D., Luo, P.: CycleMLP: a MLP-like architecture for dense prediction. arxiv 2021. arXiv preprint arXiv:2107.10224
Chen, Z., et al.: DPT: deformable patch-based transformer for visual recognition. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 2899–2907 (2021)
Dawood, M., Branson, K., Rajpoot, N.M., Minhas, F.U.A.A.: All you need is color: image based spatial gene expression prediction using neural stain learning. In: Kamp, M., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021. Communications in Computer and Information Science, vol. 1525, pp. 437–450. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-93733-1_32
Dong, X., et al.: CSWIN transformer: a general vision transformer backbone with cross-shaped windows. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12124–12134 (2022)
Gerlinger, M., et al.: Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N. Engl. J. Med. 366(10), 883–892 (2012)
He, B., et al.: Integrating spatial gene expression and breast tumour morphology via deep learning. Nat. Biomed. Eng. 4(8), 827–834 (2020)
Ho, J., Kalchbrenner, N., Weissenborn, D., Salimans, T.: Axial attention in multidimensional transformers. arXiv preprint arXiv:1912.12180 (2019)
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: CCNet: CRISS-cross attention for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 603–612 (2019)
Jian, S., Kaiming, H., Shaoqing, R., Xiangyu, Z.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision & Pattern Recognition, pp. 770–778 (2016)
Lee, Y., Kim, J., Willette, J., Hwang, S.J.: Mpvit: Multi-path vision transformer for dense prediction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7287–7296 (2022)
Li, X., Wang, C.Y.: From bulk, single-cell to spatial RNA sequencing. Int. J. Oral Sci. 13(1), 36 (2021)
Liu, Z., et al.: SWIN transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Lu, W., Graham, S., Bilal, M., Rajpoot, N., Minhas, F.: Capturing cellular topology in multi-gigapixel pathology images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 260–261 (2020)
Ren, S., Zhou, D., He, S., Feng, J., Wang, X.: Shunted self-attention via multi-scale token aggregation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10853–10862 (2022)
Rodriques, S.G., et al.: Slide-SEQ: a scalable technology for measuring genome-wide expression at high spatial resolution. Science 363(6434), 1463–1467 (2019)
Ståhl, P.L., et al.: Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science 353(6294), 78–82 (2016)
Tay, Y., Dehghani, M., Bahri, D., Metzler, D.: Efficient transformers: a survey (2022)
Vaswani, A., et al.: Attention is all you need (2017). arXiv preprint arXiv:1706.03762 (2019)
Wang, W., et al.: Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578 (2021)
Xia, Z., Pan, X., Song, S., Li, L.E., Huang, G.: Vision transformer with deformable attention (2022)
Xu, Y., Zhang, Q., Zhang, J., Tao, D.: Vitae: vision transformer advanced by exploring intrinsic inductive bias. Adv. Neural. Inf. Process. Syst. 34, 28522–28535 (2021)
Yang, Y., Hossain, M.Z., Stone, E.A., Rahman, S.: Exemplar guided deep neural network for spatial transcriptomics analysis of gene expression prediction. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 5039–5048 (2023)
Zeng, W., et al.: Not all tokens are equal: human-centric visual analysis via token clustering transformer (2022)
Zhu, L., Wang, X., Ke, Z., Zhang, W., Lau, R.W.: Biformer: vision transformer with bi-level routing attention. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10323–10333 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tang, P., Zhang, Z., Chen, C., Sheng, Y. (2024). Spatial Gene Expression Prediction Using Multi-Neighborhood Network with Reconstructing Attention. In: Yang, DN., Xie, X., Tseng, V.S., Pei, J., Huang, JW., Lin, J.CW. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2024. Lecture Notes in Computer Science(), vol 14648. Springer, Singapore. https://doi.org/10.1007/978-981-97-2238-9_13
Download citation
DOI: https://doi.org/10.1007/978-981-97-2238-9_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2240-2
Online ISBN: 978-981-97-2238-9
eBook Packages: Computer ScienceComputer Science (R0)