Abstract
We propose an integrated deep-generative framework, that jointly models complementary information from resting-state functional MRI (rs-fMRI) connectivity and diffusion tensor imaging (DTI) tractography to extract predictive biomarkers of a disease. The generative part of our framework is a structurally-regularized Dynamic Dictionary Learning (sr-DDL) model that decomposes the dynamic rs-fMRI correlation matrices into a collection of shared basis networks and time varying patient-specific loadings. This matrix factorization is guided by the DTI tractography matrices to learn anatomically informed connectivity profiles. The deep part of our framework is an LSTM-ANN block, which models the temporal evolution of the patient sr-DDL loadings to predict multidimensional clinical severity. Our coupled optimization procedure collectively estimates the basis networks, the patient-specific dynamic loadings, and the neural network weights. We validate our framework on a multi-score prediction task in 57 patients diagnosed with Autism Spectrum Disorder (ASD). Our hybrid model outperforms state-of-the-art baselines in a five-fold cross validated setting and extracts interpretable multimodal neural signatures of brain dysfunction in ASD.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.: An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process 20(3), 681–695 (2010)
Aghdam, M.A., Sharifi, A., Pedram, M.M.: Combination of rs-fMRI and sMRI data to discriminate autism spectrum disorders in young children using deep belief network. J. Digit. Imaging 31(6), 895–903 (2018)
Assaf, Y., Pasternak, O.: Diffusion tensor imaging (DTI)-based white matter mapping in brain research: a review. J. Mol. Neurosci. 34(1), 51–61 (2008). https://doi.org/10.1007/s12031-007-0029-0
Banerjee, A., Jost, J.: On the spectrum of the normalized graph Laplacian. Linear Algebra Appl. 428(11–12), 3015–3022 (2008)
Behrens, T.E., Berg, H.J., Jbabdi, S., Rushworth, M.F., Woolrich, M.W.: Probabilistic diffusion tractography with multiple fibre orientations: what can we gain? Neuroimage 34(1), 144–155 (2007)
Behzadi, Y., et al.: A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37(1), 90–101 (2007)
Bennett, I.J., Rypma, B.: Advances in functional neuroanatomy: a review of combined DTI and fMRI studies in healthy younger and older adults. Neurosci. Biobehav. Rev. 37(7), 1201–1210 (2013)
Cabral, J., Kringelbach, M.L., Deco, G.: Functional connectivity dynamically evolves on multiple time-scales over a static structural connectome: models and mechanisms. NeuroImage 160, 84–96 (2017)
Cai, B., Zille, P., Stephen, J.M., Wilson, T.W., Calhoun, V.D., Wang, Y.P.: Estimation of dynamic sparse connectivity patterns from resting state fMRI. IEEE Trans. Med. Imaging 37(5), 1224–1234 (2017)
Damaraju, E., et al.: Dynamic functional connectivity analysis reveals transient states of dysconnectivity in schizophrenia. NeuroImage: Clin. 5, 298–308 (2014)
Dziuk, M., Larson, J.G., Apostu, A., Mahone, E.M., Denckla, M.B., Mostofsky, S.H.: Dyspraxia in autism: association with motor, social, and communicative deficits. Dev. Med. Child Neurol. 49(10), 734–739 (2007)
D’Souza, N.S., Nebel, M.B., Wymbs, N., Mostofsky, S., Venkataraman, A.: A generative-discriminative basis learning framework to predict clinical severity from resting state functional MRI data. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 163–171. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_19
D’Souza, N.S., Nebel, M.B., Wymbs, N., Mostofsky, S., Venkataraman, A.: A coupled manifold optimization framework to jointly model the functional connectomics and behavioral data spaces. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 605–616. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_47
D’Souza, N.S., Nebel, M.B., Wymbs, N., Mostofsky, S., Venkataraman, A.: Integrating neural networks and dictionary learning for multidimensional clinical characterizations from functional connectomics data. In: Shen, D. (ed.) MICCAI 2019. LNCS, vol. 11766, pp. 709–717. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_79
Everson, R.: Orthogonal, but not orthonormal, procrustes problems. Adv. Comput. Math. 3(4), 782–790 (1998)
Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M.: FSL. Neuroimage 62(2), 782–790 (2012)
Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage 146, 1038–1049 (2017)
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization (2015)
Lee, M.H., Smyser, C.D., Shimony, J.S.: Resting-state fMRI: a review of methods and clinical applications. Am. J. Neuroradiol. 34(10), 1866–1872 (2013)
Manton, J.H., Mahony, R., Hua, Y.: The geometry of weighted low-rank approximations. IEEE Trans. Sig. Process 51(2), 500–514 (2003)
Mostofsky, S.H., Dubey, P., Jerath, V.K., Jansiewicz, E.M., Goldberg, M.C., Denckla, M.B.: Developmental dyspraxia is not limited to imitation in children with autism spectrum disorders. J. Int. Neuropsychol. Soc. 12(3), 314–326 (2006)
Nebel, M.B., et al.: Intrinsic visual-motor synchrony correlates with social deficits in autism. Biol. Psychiatry 79(8), 633–641 (2016)
Payakachat, N., et al.: Autism spectrum disorders: a review of measures for clinical, health services and cost-effectiveness applications. Expert Rev. Pharmacoeconomics Outcomes Res. 12(4), 485–503 (2012)
Pouw, L.B., Rieffe, C., Stockmann, L., Gadow, K.D.: The link between emotion regulation, social functioning, and depression in boys with ASD. Res. Autism Spectrum Disord. 7(4), 549–556 (2013)
Price, T., Wee, C.-Y., Gao, W., Shen, D.: Multiple-network classification of childhood autism using functional connectivity dynamics. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8675, pp. 177–184. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10443-0_23
Rabany, L., et al.: Dynamic functional connectivity in schizophrenia and autism spectrum disorder: convergence, divergence and classification. NeuroImage Clin. 24, 101966 (2019)
Rashid, B., Damaraju, E., Pearlson, G.D., Calhoun, V.D.: Dynamic connectivity states estimated from resting fMRI identify differences among schizophrenia, bipolar disorder, and healthy control subjects. Front. Hum. Neurosci. 8, 897 (2014)
Schnabel, R.B., Toint, P.L.: Forcing sparsity by projecting with respect to a non-diagonally weighted Frobenius norm. Math. Program. 25(1), 125–129 (1983). https://doi.org/10.1007/BF02591723
Skudlarski, P., Jagannathan, K., Calhoun, V.D., Hampson, M., Skudlarska, B.A., Pearlson, G.: Measuring brain connectivity: diffusion tensor imaging validates resting state temporal correlations. Neuroimage 43(3), 554–561 (2008)
Sridharan, D., et al.: A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks. Proc. Nat. Acad. Sci. 105(34), 12569–12574 (2008)
Sui, J., et al.: Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by n-way MCCA+ JICA. Front. Hum. Neurosci. 7, 235 (2013)
Tzourio-Mazoyer, N., et al.: Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neuroimage 15(1), 273–289 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
D’Souza, N.S. et al. (2020). A Deep-Generative Hybrid Model to Integrate Multimodal and Dynamic Connectivity for Predicting Spectrum-Level Deficits in Autism. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12267. Springer, Cham. https://doi.org/10.1007/978-3-030-59728-3_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-59728-3_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59727-6
Online ISBN: 978-3-030-59728-3
eBook Packages: Computer ScienceComputer Science (R0)
