Non-linear ICA Based on Cramer-Wold Metric | SpringerLink
Skip to main content
Springer Nature Link
Log in

Non-linear ICA Based on Cramer-Wold Metric

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12534))

Included in the following conference series:

Abstract

Non-linear source separation is a challenging open problem with many applications. We extend a recently proposed Adversarial Non-linear ICA (ANICA) model and introduce Cramer-Wold ICA (CW-ICA). In contrast to ANICA, we use a simple, closed–form optimization target instead of a discriminator–based independence measure. Our results show that CW-ICA achieves comparable results to ANICA while foregoing the need for adversarial training.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Except for the trivial cases when either J or \(J'\) is emptyset.

  2. 2.

    In the computation we apply the equality \(\phi _D(0)=0\).

References

  1. Almeida, L.B.: MISEP-linear and nonlinear ICA based on mutual information. J. Mach. Learn. Res. 4(Dec), 1297–1318 (2003)

    MATH  Google Scholar 

  2. Almeida, L.B.: Linear and nonlinear ICA based on mutual information - the MISEP method. Signal Process. 84(2), 231–245 (2004)

    Article  Google Scholar 

  3. Bell, A.J., Sejnowski, T.J.: An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129–1159 (1995)

    Article  Google Scholar 

  4. Brakel, P., Bengio, Y.: Learning independent features with adversarial nets for non-linear ica. arXiv preprint arXiv:1710.05050 (2017)

  5. Burgess, C.P., et al.: Understanding disentangling in \(\beta \)-vae. CoRR abs/1804.03599 (2018)

    Google Scholar 

  6. Cardoso, J.F., Souloumiac, A.: Blind beamforming for non-gaussian signals. In: Radar and Signal Processing, IEE Proceedings F, vol. 140, pp. 362–370. IET (1993)

    Google Scholar 

  7. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 2172–2180. Curran Associates, Inc. (2016)

    Google Scholar 

  8. Dinh, L., Krueger, D., Bengio, Y.: Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516 (2014)

  9. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  10. Helwig, N.E.: ICA: Independent Component Analysis (2015). http://CRAN.R-project.org/package=ica, r package version 1.0-1

  11. Hirayama, J., Hyvärinen, A., Kawanabe, M.: SPLICE: fully tractable hierarchical extension of ICA with pooling. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1491–1500. PMLR, International Convention Centre, Sydney (2017)

    Google Scholar 

  12. Hyvärinen, A.: Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626–634 (1999)

    Article  Google Scholar 

  13. Hyvarinen, A., Morioka, H.: Unsupervised feature extraction by time-contrastive learning and nonlinear ica. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29, pp. 3765–3773. Curran Associates, Inc. (2016)

    Google Scholar 

  14. Hyvärinen, A., Pajunen, P.: Nonlinear independent component analysis: existence and uniqueness results. Neural Netw. 12(3), 429–439 (1999)

    Article  Google Scholar 

  15. Karvanen, J.: PearsonICA (2008). https://CRAN.R-project.org/package=PearsonICA, r package version 1.2-3

  16. Karvanen, J., Eriksson, J., Koivunen, V.: Pearson system based method for blind separation. In: Proceedings of Second International Workshop on Independent Component Analysis and Blind Signal Separation (ICA2000), Helsinki, Finland, pp. 585–590 (2000)

    Google Scholar 

  17. Kim, H., Mnih, A.: Disentangling by Factorising. ArXiv e-prints (2018)

    Google Scholar 

  18. Kingma, D., Welling, M.: Auto-encoding variational bayes. arXiv:1312.6114 (2014)

  19. Le, Q.V., Karpenko, A., Ngiam, J., Ng, A.Y.: ICA with reconstruction cost for efficient overcomplete feature learning. In: Advances in Neural Information Processing Systems, pp. 1017–1025 (2011)

    Google Scholar 

  20. Lee, T.W., Koehler, B.U., Orglmeister, R.: Blind source separation of nonlinear mixing models. In: Neural Networks for Signal Processing [1997] VII. Proceedings of the 1997 IEEE Workshop, pp. 406–415. IEEE (1997)

    Google Scholar 

  21. Matteson, D.S., Tsay, R.S.: Independent component analysis via distance covariance. J. Am. Stat. Assoc. 112, 1–16 (2017)

    Article  MathSciNet  Google Scholar 

  22. Spurek, P., Tabor, J., Rola, P., Ociepka, M.: ICA based on asymmetry. Pattern Recogn. 67, 230–244 (2017)

    Article  Google Scholar 

  23. Stuart, A., Kendall, M.G., et al.: The advanced theory of statistics. Charles Griffin (1968)

    Google Scholar 

  24. Székely, G.J., Rizzo, M.L., Bakirov, N.K., et al.: Measuring and testing dependence by correlation of distances. Ann. stat. 35(6), 2769–2794 (2007)

    Article  MathSciNet  Google Scholar 

  25. Tabor, J., Knop, S., Spurek, P., Podolak, I., Mazur, M., Jastrzebski, S.: Cramer-wold autoencoder. arXiv preprint arXiv:1805.09235 (2018)

  26. Tan, Y., Wang, J., Zurada, J.M.: Nonlinear blind source separation using a radial basis function network. IEEE Trans. Neural Netw. 12(1), 124–134 (2001)

    Article  Google Scholar 

  27. Tolstikhin, I., Bousquet, O., Gelly, S., Schoelkopf, B.: Wasserstein auto-encoders. arXiv:1711.01558 (2017)

  28. Zhang, K., Chan, L.: Minimal nonlinear distortion principle for nonlinear independent component analysis. J. Mach. Learn. Res. 9(Nov), 2455–2487 (2008)

    MathSciNet  MATH  Google Scholar 

  29. Zheng, C.H., Huang, D.S., Li, K., Irwin, G., Sun, Z.L.: MISEP method for postnonlinear blind source separation. Neural Comput. 19, 2557–2578 (2007)

    Article  Google Scholar 

Download references

Acknowledgements

The work of P. Spurek was supported by the National Centre of Science (Poland) Grant No. 2019/33/B/ST6/00894. The work of A. Nowak was supported by the Foundation for Polish Science Grant No. POIR.04.04.00-00-14DE/18-00 co-financed by the European Union under the European Regional Development Fund. The work of J. Tabor was supported by the National Centre of Science (Poland) Grant No. 2017/25/B/ST6/01271. The work of Ł. Maziarka was supported by the National Science Centre (Poland) grant no. 2018/31/B/ST6/00993.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Computer Science, Jagiellonian University, Krakow, Poland

    Przemysław Spurek, Aleksandra Nowak, Jacek Tabor & Łukasz Maziarka

  2. Center of Data Science/Department of Radiology, New York University, New York, USA

    Stanisław Jastrzębski

Authors
  1. Przemysław Spurek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aleksandra Nowak
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacek Tabor
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Łukasz Maziarka
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stanisław Jastrzębski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Łukasz Maziarka .

Editor information

Editors and Affiliations

  1. Department of AI, Ping An Life, Shenzhen, China

    Haiqin Yang

  2. Faculty of Information Technology, King Mongkut's Institute of Technology Ladkrabang, Bangkok, Thailand

    Kitsuchart Pasupa

  3. City University of Hong Kong, Kowloon, Hong Kong

    Andrew Chi-Sing Leung

  4. Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, Hong Kong

    James T. Kwok

  5. School of Information Technology, King Mongkut’s University of Technology Thonburi, Bangkok, Thailand

    Jonathan H. Chan

  6. The Chinese University of Hong Kong, New Territories, Hong Kong

    Irwin King

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Spurek, P., Nowak, A., Tabor, J., Maziarka, Ł., Jastrzębski, S. (2020). Non-linear ICA Based on Cramer-Wold Metric. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12534. Springer, Cham. https://doi.org/10.1007/978-3-030-63836-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-63836-8_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-63835-1

  • Online ISBN: 978-3-030-63836-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation