{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T02:01:37Z","timestamp":1725501697103},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"We present a modular convolutional architecture for denoising rendered images. We expand on the capabilities of kernel-predicting networks by combining them with a number of task-specific modules, and optimizing the assembly using an asymmetric loss. The source-aware encoder---the first module in the assembly---extracts low-level features and embeds them into a common feature space, enabling quick adaptation of a trained network to novel data. The spatial and temporal modules extract abstract, high-level features for kernel-based reconstruction, which is performed at three different spatial scales to reduce low-frequency artifacts. The complete network is trained using a class of asymmetric loss functions that are designed to preserve details and provide the user with a direct control over the variance-bias trade-off during inference. We also propose an error-predicting module for inferring reconstruction error maps that can be used to drive adaptive sampling. Finally, we present a theoretical analysis of convergence rates of kernel-predicting architectures, shedding light on why kernel prediction performs better than synthesizing the colors directly, complementing the empirical evidence presented in this and previous works. We demonstrate that our networks attain results that compare favorably to state-of-the-art methods in terms of detail preservation, low-frequency noise removal, and temporal stability on a variety of production and academic datasets.<\/jats:p>","DOI":"10.1145\/3197517.3201388","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-15","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":117,"title":["Denoising with kernel prediction and asymmetric loss functions"],"prefix":"10.1145","volume":"37","author":[{"given":"Thijs","family":"Vogels","sequence":"first","affiliation":[{"name":"Disney Research"}]},{"given":"Fabrice","family":"Rousselle","sequence":"additional","affiliation":[{"name":"Disney Research"}]},{"given":"Brian","family":"Mcwilliams","sequence":"additional","affiliation":[{"name":"Disney Research"}]},{"given":"Gerhard","family":"R\u00f6thlin","sequence":"additional","affiliation":[{"name":"Disney Research"}]},{"given":"Alex","family":"Harvill","sequence":"additional","affiliation":[{"name":"Pixar Animation Studios"}]},{"given":"David","family":"Adler","sequence":"additional","affiliation":[{"name":"Walt Disney Animation Studios"}]},{"given":"Mark","family":"Meyer","sequence":"additional","affiliation":[{"name":"Pixar Animation Studios"}]},{"given":"Jan","family":"Nov\u00e1k","sequence":"additional","affiliation":[{"name":"Disney Research"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Mart\u00edn Abadi Ashish Agarwal Paul Barham etal 2015. TensorFlow: Large-scale machine learning on heterogeneous systems http:\/\/tensorflow.org\/ Mart\u00edn Abadi Ashish Agarwal Paul Barham et al. 2015. TensorFlow: Large-scale machine learning on heterogeneous systems http:\/\/tensorflow.org\/"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.12"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_2_4_1","volume-title":"Proc. 34th International Conference on Machine Learning (Proc. Machine Learning Research)","volume":"70","author":"Balduzzi David","year":"2017","unstructured":"David Balduzzi , Brian McWilliams , and Tony Butler-Yeoman . 2017 . Neural Taylor approximations: Convergence and exploration in rectifier networks . In Proc. 34th International Conference on Machine Learning (Proc. Machine Learning Research) , Vol. 70 . PMLR, Sydney, Australia, 351--360. David Balduzzi, Brian McWilliams, and Tony Butler-Yeoman. 2017. Neural Taylor approximations: Convergence and exploration in rectifier networks. In Proc. 34th International Conference on Machine Learning (Proc. Machine Learning Research), Vol. 70. PMLR, Sydney, Australia, 351--360."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01996.x"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-6377(02)00231-6"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/3071773.3071785"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280924"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1137\/040616024"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-007-0052-1"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000050"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2354409.2354805"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073601"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.643267"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2532708"},{"key":"e_1_2_2_16_1","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","author":"Duchi John","year":"2011","unstructured":"John Duchi , Elad Hazan , and Yoram Singer . 2011 . Adaptive subgradient methods for online learning and stochastic optimization . Journal of Machine Learning Research 12 , Jul (2011), 2121 -- 2159 . John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121--2159.","journal-title":"Journal of Machine Learning Research 12"},{"key":"e_1_2_2_17_1","volume-title":"Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15)","author":"Glorot Xavier","year":"2010","unstructured":"Xavier Glorot and Yoshua Bengio . 2010 . Understanding the difficulty of training deep feedforward neural networks . In Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15) . JMLR.org, 249--256. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. 13th International Conference on Artificial Intelligence and Statistics (May 13--15). JMLR.org, 249--256."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360632"},{"key":"e_1_2_2_19_1","volume-title":"Proc. 33nd International Conference on Machine Learning (June 19--24)","author":"Hardt Moritz","year":"2016","unstructured":"Moritz Hardt , Ben Recht , and Yoram Singer . 2016 . Train faster, generalize better: Stability of stochastic gradient descent . In Proc. 33nd International Conference on Machine Learning (June 19--24) . JMLR.org, 1225--1234. Moritz Hardt, Ben Recht, and Yoram Singer. 2016. Train faster, generalize better: Stability of stochastic gradient descent. In Proc. 33nd International Conference on Machine Learning (June 19--24). JMLR.org, 1225--1234."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_21_1","volume-title":"Advances in Neural Information Processing Systems 21 (December 8--11). Curran Associates","author":"Jain Viren","unstructured":"Viren Jain and H. Sebastian Seung . 2008. Natural image denoising with convolutional networks . In Advances in Neural Information Processing Systems 21 (December 8--11). Curran Associates , Inc ., 769--776. Viren Jain and H. Sebastian Seung. 2008. Natural image denoising with convolutional networks. In Advances in Neural Information Processing Systems 21 (December 8--11). Curran Associates, Inc., 769--776."},{"key":"e_1_2_2_22_1","volume-title":"Tinne Tuytelaars, and Luc Van Gool.","author":"Jia Xu","year":"2016","unstructured":"Xu Jia , Bert De Brabandere , Tinne Tuytelaars, and Luc Van Gool. 2016 . Dynamic filter networks. In Advances in Neural Information Processing Systems 29 (December 5--10). Curran Associates, Inc ., 667--675. Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Luc Van Gool. 2016. Dynamic filter networks. In Advances in Neural Information Processing Systems 29 (December 5--10). Curran Associates, Inc., 667--675."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766977"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2776880.2792699"},{"key":"e_1_2_2_25_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A method for stochastic optimization. CoRR abs\/1412.6980 (2014). arXiv:1412.6980 Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR abs\/1412.6980 (2014). arXiv:1412.6980"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_2_2_27_1","volume-title":"Advances in Neural Information Processing Systems 29 (December 5--10). Curran Associates","author":"Krummenacher Gabriel","unstructured":"Gabriel Krummenacher , Brian McWilliams , Yannic Kilcher , Joachim M. Buhmann , and Nicolai Meinshausen . 2016. Scalable adaptive stochastic optimization using random projections . In Advances in Neural Information Processing Systems 29 (December 5--10). Curran Associates , Inc ., 1750--1758. Gabriel Krummenacher, Brian McWilliams, Yannic Kilcher, Joachim M. Buhmann, and Nicolai Meinshausen. 2016. Scalable adaptive stochastic optimization using random projections. In Advances in Neural Information Processing Systems 29 (December 5--10). Curran Associates, Inc., 1750--1758."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366213"},{"key":"e_1_2_2_30_1","volume-title":"A variance reduced stochastic Newton method. CoRR abs\/1503.08316","author":"Lucchi Aur\u00e9lien","year":"2015","unstructured":"Aur\u00e9lien Lucchi , Brian McWilliams , and Thomas Hofmann . 2015. A variance reduced stochastic Newton method. CoRR abs\/1503.08316 ( 2015 ). arXiv:1503.08316 Aur\u00e9lien Lucchi, Brian McWilliams, and Thomas Hofmann. 2015. A variance reduced stochastic Newton method. CoRR abs\/1503.08316 (2015). arXiv:1503.08316"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/318009.318015"},{"key":"e_1_2_2_32_1","volume-title":"Burst denoising with kernel prediction networks. CoRR abs\/1712.02327","author":"Mildenhall Ben","year":"2017","unstructured":"Ben Mildenhall , Jonathan T. Barron , Jiawen Chen , Dillon Sharlet , Ren Ng , and Robert Carroll . 2017. Burst denoising with kernel prediction networks. CoRR abs\/1712.02327 ( 2017 ). arXiv:1712.02327 Ben Mildenhall, Jonathan T. Barron, Jiawen Chen, Dillon Sharlet, Ren Ng, and Robert Carroll. 2017. Burst denoising with kernel prediction networks. CoRR abs\/1712.02327 (2017). arXiv:1712.02327"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/37401.37410"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641762"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12004"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925936"},{"key":"e_1_2_2_37_1","volume-title":"Machine learning - a probabilistic perspective","author":"Murphy Kevin P.","unstructured":"Kevin P. Murphy . 2012. Machine learning - a probabilistic perspective . MIT Press . Kevin P. Murphy. 2012. Machine learning - a probabilistic perspective. MIT Press."},{"key":"e_1_2_2_38_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition (July 21--26)","author":"Niklaus Simon","year":"2017","unstructured":"Simon Niklaus , Long Mai , and Feng Liu . 2017 . Video frame interpolation via adaptive convolution . In IEEE Conference on Computer Vision and Pattern Recognition (July 21--26) . IEEE Computer Society, 2270--2279. Simon Niklaus, Long Mai, and Feng Liu. 2017. Video frame interpolation via adaptive convolution. In IEEE Conference on Computer Vision and Pattern Recognition (July 21--26). IEEE Computer Society, 2270--2279."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618486"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311543"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2015.2388583"},{"key":"e_1_2_2_42_1","volume-title":"Medical Image Computing and Computer-Assisted Intervention - 18th International Conf. (October 5--9)","author":"Ronneberger Olaf","unstructured":"Olaf Ronneberger , Philipp Fischer , and Thomas Brox . 2015. U-Net: Convolutional networks for biomedical image segmentation . In Medical Image Computing and Computer-Assisted Intervention - 18th International Conf. (October 5--9) . Springer , 234--241. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention - 18th International Conf. (October 5--9). Springer, 234--241."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024193"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366214"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12219"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/192161.192189"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2167076.2167083"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000018"},{"key":"e_1_2_2_49_1","volume-title":"Kernel-predicting convolutional networks for denoising Monte Carlo renderings. Master's thesis. ETH Z\u00fcrich","author":"Vogels Thijs","unstructured":"Thijs Vogels . 2016. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. Master's thesis. ETH Z\u00fcrich , Z\u00fcrich, Switzerland . Thijs Vogels. 2016. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. Master's thesis. ETH Z\u00fcrich, Z\u00fcrich, Switzerland."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_51_1","volume-title":"Advances in Neural Information Processing Systems 30 (December 4--9). Curran Associates","author":"Wilson Ashia C.","unstructured":"Ashia C. Wilson , Rebecca Roelofs , Mitchell Stern , Nati Srebro , and Benjamin Recht . 2017. The marginal value of adaptive gradient methods in machine learning . In Advances in Neural Information Processing Systems 30 (December 4--9). Curran Associates , Inc ., 4151--4161. Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht. 2017. The marginal value of adaptive gradient methods in machine learning. In Advances in Neural Information Processing Systems 30 (December 4--9). Curran Associates, Inc., 4151--4161."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/2816723.2816781"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201388","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,31]],"date-time":"2022-12-31T20:34:38Z","timestamp":1672518878000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201388"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201388"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201388","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}