{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T16:01:19Z","timestamp":1726761679022},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,10,31]]},"abstract":"\n We propose to use deep neural networks for generating samples in Monte Carlo integration. Our work is based on non-linear independent components estimation (NICE), which we extend in numerous ways to improve performance and enable its application to integration problems. First, we introduce piecewise-polynomial coupling transforms that greatly increase the modeling power of individual coupling layers. Second, we propose to preprocess the inputs of neural networks using one-blob encoding, which stimulates localization of computation and improves inference. Third, we derive a gradient-descent-based optimization for the Kullback-Leibler and the \u03c7\n 2<\/jats:sup>\n divergence for the specific application of Monte Carlo integration with unnormalized stochastic estimates of the target distribution. Our approach enables fast and accurate inference and efficient sample generation independently of the dimensionality of the integration domain. We show its benefits on generating natural images and in two applications to light-transport simulation: first, we demonstrate learning of joint path-sampling densities in the primary sample space and importance sampling of multi-dimensional path prefixes thereof. Second, we use our technique to extract conditional directional densities driven by the product of incident illumination and the BSDF in the rendering equation, and we leverage the densities for path guiding. In all applications, our approach yields on-par or higher performance than competing techniques at equal sample count.\n <\/jats:p>","DOI":"10.1145\/3341156","type":"journal-article","created":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T13:13:05Z","timestamp":1570713185000},"page":"1-19","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":135,"title":["Neural Importance Sampling"],"prefix":"10.1145","volume":"38","author":[{"ORCID":"http:\/\/orcid.org\/0000-0001-7577-755X","authenticated-orcid":false,"given":"Thomas","family":"M\u00fcller","sequence":"first","affiliation":[{"name":"Disney Research 8 ETH Z\u00fcrich"}]},{"given":"Brian","family":"Mcwilliams","sequence":"additional","affiliation":[{"name":"Disney Research"}]},{"given":"Fabrice","family":"Rousselle","sequence":"additional","affiliation":[{"name":"Disney Research"}]},{"given":"Markus","family":"Gross","sequence":"additional","affiliation":[{"name":"Disney Research 8 ETH Z\u00fcrich"}]},{"given":"Jan","family":"Nov\u00e1k","sequence":"additional","affiliation":[{"name":"Disney Research"}]}],"member":"320","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean etal 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Retrieved from http:\/\/tensorflow.org\/. Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean et al. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Retrieved from http:\/\/tensorflow.org\/."},{"key":"e_1_2_2_2_1","volume-title":"Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas.","author":"Andrychowicz Marcin","year":"2016","unstructured":"Marcin Andrychowicz , Misha Denil , Sergio Gomez Colmenarejo , Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. 2016 . Learning to learn by gradient descent by gradient descent. arXiv:1606.04474 (June 2016). Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. 2016. Learning to learn by gradient descent by gradient descent. arXiv:1606.04474 (June 2016)."},{"key":"e_1_2_2_3_1","unstructured":"Benedikt Bitterli. 2016. Rendering resources. Retrieved from https:\/\/benedikt-bitterli.me\/resources\/. Benedikt Bitterli. 2016. Rendering resources. Retrieved from https:\/\/benedikt-bitterli.me\/resources\/."},{"key":"e_1_2_2_4_1","volume-title":"Neural ordinary differential equations. arXiv:1806.07366 (June","author":"Chen Tian Qi","year":"2018","unstructured":"Tian Qi Chen , Yulia Rubanova , Jesse Bettencourt , and David Duvenaud . 2018. Neural ordinary differential equations. arXiv:1806.07366 (June 2018 ). Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. 2018. Neural ordinary differential equations. arXiv:1806.07366 (June 2018)."},{"key":"e_1_2_2_5_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.)","volume":"70","author":"Chen Yutian","year":"2017","unstructured":"Yutian Chen , Matthew W. Hoffman , Sergio G\u00f3mez Colmenarejo , Misha Denil , Timothy P. Lillicrap , Matt Botvinick , and Nando de Freitas . 2017 . Learning to learn without gradient descent by gradient descent . In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.) , Vol. 70 . PMLR, International Convention Centre, Sydney, Australia, 748--756. Yutian Chen, Matthew W. Hoffman, Sergio G\u00f3mez Colmenarejo, Misha Denil, Timothy P. Lillicrap, Matt Botvinick, and Nando de Freitas. 2017. Learning to learn without gradient descent by gradient descent. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), Vol. 70. PMLR, International Convention Centre, Sydney, Australia, 748--756."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-91436-7_9"},{"key":"e_1_2_2_7_1","volume-title":"NICE: Non-linear independent components estimation. arXiv:1410.8516 (Oct.","author":"Dinh Laurent","year":"2014","unstructured":"Laurent Dinh , David Krueger , and Yoshua Bengio . 2014 . NICE: Non-linear independent components estimation. arXiv:1410.8516 (Oct. 2014). Laurent Dinh, David Krueger, and Yoshua Bengio. 2014. NICE: Non-linear independent components estimation. arXiv:1410.8516 (Oct. 2014)."},{"key":"e_1_2_2_8_1","volume-title":"Density estimation using real NVP. arXiv:1605.08803 (March","author":"Dinh Laurent","year":"2016","unstructured":"Laurent Dinh , Jascha Sohl-Dickstein , and Samy Bengio . 2016. Density estimation using real NVP. arXiv:1605.08803 (March 2016 ). Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. 2016. Density estimation using real NVP. arXiv:1605.08803 (March 2016)."},{"key":"e_1_2_2_9_1","volume-title":"International Conference on Machine Learning. 881--889","author":"Germain Mathieu","year":"2015","unstructured":"Mathieu Germain , Karol Gregor , Iain Murray , and Hugo Larochelle . 2015 . MADE: Masked autoencoder for distribution estimation . In International Conference on Machine Learning. 881--889 . Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle. 2015. MADE: Masked autoencoder for distribution estimation. In International Conference on Machine Learning. 881--889."},{"key":"e_1_2_2_10_1","volume-title":"Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15)","author":"Glorot Xavier","year":"2010","unstructured":"Xavier Glorot and Yoshua Bengio . 2010 . Understanding the difficulty of training deep feedforward neural networks . In Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15) . JMLR.org, 249--256. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15). JMLR.org, 249--256."},{"key":"e_1_2_2_11_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680."},{"key":"e_1_2_2_12_1","volume-title":"Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association.","author":"Guo Jerry Jinfeng","year":"2018","unstructured":"Jerry Jinfeng Guo , Pablo Bauszat , Jacco Bikker , and Elmar Eisemann . 2018 . Primary sample space path guiding . In Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association. Jerry Jinfeng Guo, Pablo Bauszat, Jacco Bikker, and Elmar Eisemann. 2018. Primary sample space path guiding. In Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601138"},{"key":"e_1_2_2_14_1","volume-title":"Harris","author":"Harris David Money","year":"2013","unstructured":"David Money Harris and Sarah L . Harris . 2013 . 3.4.2\u2014State encodings. In Digital Design and Computer Architecture (2nd Ed.). Morgan Kaufmann , Boston, 129--131. DOI:https:\/\/doi.org\/10.1016\/B978-0-12-394424-5.00002-1 10.1016\/B978-0-12-394424-5.00002-1 David Money Harris and Sarah L. Harris. 2013. 3.4.2\u2014State encodings. In Digital Design and Computer Architecture (2nd Ed.). Morgan Kaufmann, Boston, 129--131. DOI:https:\/\/doi.org\/10.1016\/B978-0-12-394424-5.00002-1"},{"key":"e_1_2_2_15_1","volume-title":"Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association.","author":"Herholz Sebastian","unstructured":"Sebastian Herholz , Oskar Elek , Jens Schindel , Jaroslav K\u0159iv\u00e1nek , and Hendrik P. A. Lensch . 2018. A unified manifold framework for efficient BRDF sampling based on parametric mixture models . In Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association. Sebastian Herholz, Oskar Elek, Jens Schindel, Jaroslav K\u0159iv\u00e1nek, and Hendrik P. A. Lensch. 2018. A unified manifold framework for efficient BRDF sampling based on parametric mixture models. In Eurographics Symposium on Rendering\u2014Experimental Ideas 8 Implementations, Wenzel Jakob and Toshiya Hachisuka (Eds.). The Eurographics Association."},{"key":"e_1_2_2_16_1","volume-title":"Product importance sampling for light transport path guiding. Computer Graphics Forum","author":"Herholz Sebastian","year":"2016","unstructured":"Sebastian Herholz , Oskar Elek , Ji\u0159\u00ed Vorba , Hendrik Lensch , and Jaroslav K\u0159iv\u00e1nek . 2016. Product importance sampling for light transport path guiding. Computer Graphics Forum ( 2016 ). DOI:https:\/\/doi.org\/10.1111\/cgf.12950 10.1111\/cgf.12950 Sebastian Herholz, Oskar Elek, Ji\u0159\u00ed Vorba, Hendrik Lensch, and Jaroslav K\u0159iv\u00e1nek. 2016. Product importance sampling for light transport path guiding. Computer Graphics Forum (2016). DOI:https:\/\/doi.org\/10.1111\/cgf.12950"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/584458.584476"},{"key":"e_1_2_2_18_1","volume-title":"Courville","author":"Huang Chin-Wei","year":"2018","unstructured":"Chin-Wei Huang , David Krueger , Alexandre Lacoste , and Aaron C . Courville . 2018 . Neural autoregressive flows. arXiv:1804.00779 (April 2018). Chin-Wei Huang, David Krueger, Alexandre Lacoste, and Aaron C. Courville. 2018. Neural autoregressive flows. arXiv:1804.00779 (April 2018)."},{"key":"e_1_2_2_19_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. Retrieved from http:\/\/www.mitsuba-renderer.org. Wenzel Jakob. 2010. Mitsuba renderer. Retrieved from http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_20_1","volume-title":"Rendering Techniques","author":"Jensen Henrik Wann","unstructured":"Henrik Wann Jensen . 1995. Importance driven path tracing using the photon map . In Rendering Techniques . Springer Vienna , Vienna , 326--335. DOI:https:\/\/doi.org\/10.1007\/978-3-7091-9430-0_31 10.1007\/978-3-7091-9430-0_31 Henrik Wann Jensen. 1995. Importance driven path tracing using the photon map. In Rendering Techniques. Springer Vienna, Vienna, 326--335. DOI:https:\/\/doi.org\/10.1007\/978-3-7091-9430-0_31"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/15886.15902"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.t01-1-00703"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.matcom.2019.01.010"},{"key":"e_1_2_2_24_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A method for stochastic optimization. arXiv:1412.6980 (June 2014). Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980 (June 2014)."},{"key":"e_1_2_2_25_1","volume-title":"Kingma and Prafulla Dhariwal","author":"Diederik","year":"2018","unstructured":"Diederik P. Kingma and Prafulla Dhariwal . 2018 . Glow : Generative flow with invertible 1x1 convolutions. arXiv:1807.03039 (July 2018). Diederik P. Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. arXiv:1807.03039 (July 2018)."},{"key":"e_1_2_2_26_1","unstructured":"Diederik P. Kingma Tim Salimans Rafal Jozefowicz Xi Chen Ilya Sutskever and Max Welling. 2016. Improved variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems. 4743--4751. Diederik P. Kingma Tim Salimans Rafal Jozefowicz Xi Chen Ilya Sutskever and Max Welling. 2016. Improved variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems. 4743--4751."},{"key":"e_1_2_2_27_1","volume-title":"6th Eurographics Workshop on Rendering). 11--20","author":"Eric","unstructured":"Eric P. Lafortune and Yves D. Willems. 1995. A 5D tree to reduce the variance of Monte Carlo ray tracing. In Rendering Techniques\u201995 (Proc. of the 6th Eurographics Workshop on Rendering). 11--20 . DOI:https:\/\/doi.org\/10.1007\/978-3-7091-9430-0_2 10.1007\/978-3-7091-9430-0_2 Eric P. Lafortune and Yves D. Willems. 1995. A 5D tree to reduce the variance of Monte Carlo ray tracing. In Rendering Techniques\u201995 (Proc. of the 6th Eurographics Workshop on Rendering). 11--20. DOI:https:\/\/doi.org\/10.1007\/978-3-7091-9430-0_2"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13227"},{"key":"e_1_2_2_30_1","volume-title":"Notes on optimal approximations for importance sampling. arXiv:1707.08358 (July","author":"Pantaleoni Jacopo","year":"2017","unstructured":"Jacopo Pantaleoni and Eric Heitz . 2017. Notes on optimal approximations for importance sampling. arXiv:1707.08358 (July 2017 ). Jacopo Pantaleoni and Eric Heitz. 2017. Notes on optimal approximations for importance sampling. arXiv:1707.08358 (July 2017)."},{"key":"e_1_2_2_31_1","unstructured":"George Papamakarios Iain Murray and Theo Pavlakou. 2017. Masked autoregressive flow for density estimation. In Advances in Neural Information Processing Systems. 2338--2347. George Papamakarios Iain Murray and Theo Pavlakou. 2017. Masked autoregressive flow for density estimation. In Advances in Neural Information Processing Systems. 2338--2347."},{"key":"e_1_2_2_32_1","volume-title":"Proceedings of the 3rd IEEE Symposium on Interactive Ray Tracing. 107--114","author":"Pegoraro Vincent","unstructured":"Vincent Pegoraro , Carson Brownlee , Peter S. Shirley , and Steven G. Parker . 2008a. Towards interactive global illumination effects via sequential Monte Carlo adaptation . In Proceedings of the 3rd IEEE Symposium on Interactive Ray Tracing. 107--114 . Vincent Pegoraro, Carson Brownlee, Peter S. Shirley, and Steven G. Parker. 2008a. Towards interactive global illumination effects via sequential Monte Carlo adaptation. In Proceedings of the 3rd IEEE Symposium on Interactive Ray Tracing. 107--114."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01247.x"},{"key":"e_1_2_2_34_1","volume-title":"International Conference on Machine Learning. 1530--1538","author":"Rezende Danilo","year":"2015","unstructured":"Danilo Rezende and Shakir Mohamed . 2015 . Variational inference with normalizing flows . In International Conference on Machine Learning. 1530--1538 . Danilo Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In International Conference on Machine Learning. 1530--1538."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024193"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/RT.2006.280224"},{"key":"e_1_2_2_37_1","volume-title":"Wavenet: A generative model for raw audio. arXiv:1609.03499 (Sept.","author":"van den Oord Aaron","year":"2016","unstructured":"Aaron van den Oord , Sander Dieleman , Heiga Zen , Karen Simonyan , Oriol Vinyals , Alex Graves , Nal Kalchbrenner , Andrew Senior , and Koray Kavukcuoglu . 2016 a. Wavenet: A generative model for raw audio. arXiv:1609.03499 (Sept. 2016). Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016a. Wavenet: A generative model for raw audio. arXiv:1609.03499 (Sept. 2016)."},{"key":"e_1_2_2_38_1","volume-title":"International Conference on Machine Learning. 1747--1756","author":"van den Oord Aaron","year":"2016","unstructured":"Aaron van den Oord , Nal Kalchbrenner , and Koray Kavukcuoglu . 2016 b. Pixel recurrent neural networks . In International Conference on Machine Learning. 1747--1756 . Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. 2016b. Pixel recurrent neural networks. In International Conference on Machine Learning. 1747--1756."},{"key":"e_1_2_2_40_1","volume-title":"EG Rendering Workshop.","author":"Veach Eric","unstructured":"Eric Veach and Leonidas J. Guibas . 1994. Bidirectional estimators for light transport . In EG Rendering Workshop. Eric Veach and Leonidas J. Guibas. 1994. Bidirectional estimators for light transport. In EG Rendering Workshop."},{"key":"e_1_2_2_41_1","volume-title":"Proc. SIGGRAPH. 419--428","author":"Veach Eric","year":"1838","unstructured":"Eric Veach and Leonidas J. Guibas . 1995. Optimally combining sampling techniques for Monte Carlo rendering . In Proc. SIGGRAPH. 419--428 . DOI:https:\/\/doi.org\/10.1145\/2 1838 0.218498 10.1145\/218380.218498 Eric Veach and Leonidas J. Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proc. SIGGRAPH. 419--428. DOI:https:\/\/doi.org\/10.1145\/218380.218498"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201340"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601203"},{"key":"e_1_2_2_44_1","volume-title":"Learning to importance sample in primary sample space. arXiv:1808.07840 (Sept","author":"Zheng Quan","year":"2018","unstructured":"Quan Zheng and Matthias Zwicker . 2018. Learning to importance sample in primary sample space. arXiv:1808.07840 (Sept . 2018 ). Quan Zheng and Matthias Zwicker. 2018. Learning to importance sample in primary sample space. arXiv:1808.07840 (Sept. 2018)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3341156","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T11:48:21Z","timestamp":1672573701000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3341156"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":43,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2019,10,31]]}},"alternative-id":["10.1145\/3341156"],"URL":"https:\/\/doi.org\/10.1145\/3341156","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"2018-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}