Spread Flows for Manifold Modelling

Mingtian Zhang, Yitong Sun, Chen Zhang, Steven Mcdonagh
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:11435-11456, 2023.

Abstract

Flow-based models typically define a latent space with dimensionality identical to the observational space. In many problems, however, the data does not populate the full ambient data space that they natively reside in, rather inhabiting a lower-dimensional manifold. In such scenarios, flow-based models are unable to represent data structures exactly as their densities will always have support off the data manifold, potentially resulting in degradation of model performance. To address this issue, we propose to learn a manifold prior for flow models that leverage the recently proposed spread divergence towards fixing the crucial problem; the KL divergence and maximum likelihood estimation are ill-defined for manifold learning. In addition to improving both sample quality and representation quality, an auxiliary benefit enabled by our approach is the ability to identify the intrinsic dimension of the manifold distribution.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-zhang23k, title = {Spread Flows for Manifold Modelling}, author = {Zhang, Mingtian and Sun, Yitong and Zhang, Chen and Mcdonagh, Steven}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {11435--11456}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/zhang23k/zhang23k.pdf}, url = {https://proceedings.mlr.press/v206/zhang23k.html}, abstract = {Flow-based models typically define a latent space with dimensionality identical to the observational space. In many problems, however, the data does not populate the full ambient data space that they natively reside in, rather inhabiting a lower-dimensional manifold. In such scenarios, flow-based models are unable to represent data structures exactly as their densities will always have support off the data manifold, potentially resulting in degradation of model performance. To address this issue, we propose to learn a manifold prior for flow models that leverage the recently proposed spread divergence towards fixing the crucial problem; the KL divergence and maximum likelihood estimation are ill-defined for manifold learning. In addition to improving both sample quality and representation quality, an auxiliary benefit enabled by our approach is the ability to identify the intrinsic dimension of the manifold distribution.} }
Endnote
%0 Conference Paper %T Spread Flows for Manifold Modelling %A Mingtian Zhang %A Yitong Sun %A Chen Zhang %A Steven Mcdonagh %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-zhang23k %I PMLR %P 11435--11456 %U https://proceedings.mlr.press/v206/zhang23k.html %V 206 %X Flow-based models typically define a latent space with dimensionality identical to the observational space. In many problems, however, the data does not populate the full ambient data space that they natively reside in, rather inhabiting a lower-dimensional manifold. In such scenarios, flow-based models are unable to represent data structures exactly as their densities will always have support off the data manifold, potentially resulting in degradation of model performance. To address this issue, we propose to learn a manifold prior for flow models that leverage the recently proposed spread divergence towards fixing the crucial problem; the KL divergence and maximum likelihood estimation are ill-defined for manifold learning. In addition to improving both sample quality and representation quality, an auxiliary benefit enabled by our approach is the ability to identify the intrinsic dimension of the manifold distribution.
APA
Zhang, M., Sun, Y., Zhang, C. & Mcdonagh, S.. (2023). Spread Flows for Manifold Modelling. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:11435-11456 Available from https://proceedings.mlr.press/v206/zhang23k.html.

Related Material