{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,20]],"date-time":"2024-10-20T14:10:20Z","timestamp":1729433420340,"version":"3.27.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"1","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2023,5,12]]},"abstract":"Neural rendering is an exciting topic injecting machine learning methodologies into the classical computer graphics rendering pipeline. Although recent works have achieved remarkable fidelity, discussion on how to enable it for interactive scenarios like video games seems to be lacking. Aside from an editable 3D model and UV-mapping, an interactive application will demand the neural rendering to handle animatable 3D content with interactive speed. This is currently a gap in neural rendering and our solution to this problem is a novel neural rendering pipeline involving a primitive named NeRFahedron. It localizes a NeRF field and as such effectively reduces the number of expensive network sampling operations to improve speed. Our pipeline involves tetrahedron rasterization, localized ray marching and near-surface particle sampling. The result is a method that can enable animatable content for neural rendering with interactive speed, which has been shown to be competitive in rendering animation. We will also showcase its ability to enable interactive applications via a real-time demo.<\/jats:p>","DOI":"10.1145\/3585512","type":"journal-article","created":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T17:05:34Z","timestamp":1684256734000},"page":"1-20","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["NeRFahedron"],"prefix":"10.1145","volume":"6","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-8377-0216","authenticated-orcid":false,"given":"Zackary P. T.","family":"Sin","sequence":"first","affiliation":[{"name":"The Hong Kong Polytechnic University, Hong Kong SAR"}]},{"ORCID":"http:\/\/orcid.org\/0000-0002-9671-896X","authenticated-orcid":false,"given":"Peter H. F.","family":"Ng","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University, Hong Kong SAR"}]},{"ORCID":"http:\/\/orcid.org\/0000-0001-7682-9032","authenticated-orcid":false,"given":"Hong Va","family":"Leong","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University, Hong Kong SAR"}]}],"member":"320","published-online":{"date-parts":[[2023,5,16]]},"reference":[{"key":"e_1_2_3_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00282"},{"key":"e_1_2_3_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00574"},{"key":"e_1_2_3_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13859"},{"key":"e_1_2_3_4_1","volume-title":"Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. arXiv preprint arXiv:2208.00277","author":"Chen Zhiqin","year":"2022","unstructured":"Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi. 2022. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. arXiv preprint arXiv:2208.00277 (2022)."},{"key":"e_1_2_3_5_1","doi-asserted-by":"crossref","unstructured":"Jonathan Cohen Amitabh Varshney Dinesh Manocha Greg Turk Hans Weber Pankaj Agarwal Frederick Brooks and William Wright. 1993. Simplification envelopes. In ACM SIGGRAPH. 119--128.","DOI":"10.1145\/237170.237220"},{"key":"e_1_2_3_6_1","volume-title":"Danish Conference on Pattern Recognition and Image Analysis (DSAGM)","author":"Erleben Kenny","year":"2004","unstructured":"Kenny Erleben and Henrik Dohlmann. 2004. The thin shell tetrahedral mesh. Danish Conference on Pattern Recognition and Image Analysis (DSAGM) (2004), 94--102."},{"key":"e_1_2_3_7_1","volume-title":"Conferece on Neural Information Processing Systems (NeurIPS)","volume":"27","author":"Goodfellow Ian J.","year":"2014","unstructured":"Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Conferece on Neural Information Processing Systems (NeurIPS), Vol. 27. 9 pages."},{"volume-title":"Image-to-Image Translation with Conditional Adversarial Nets. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1125--1134","author":"Isola Phillip","key":"e_1_2_3_8_1","unstructured":"Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Nets. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1125--1134."},{"key":"e_1_2_3_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13860"},{"key":"e_1_2_3_10_1","volume-title":"Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6498--6508","author":"Li Zhengqi","year":"2021","unstructured":"Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. 2021. Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6498--6508."},{"key":"e_1_2_3_11_1","first-page":"15651","article-title":"Neural Sparse Voxel Fields","volume":"33","author":"Liu Lingjie","year":"2020","unstructured":"Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. 2020. Neural Sparse Voxel Fields. Conferece on Neural Information Processing Systems (NeurIPS) 33 (2020), 15651--15663.","journal-title":"Conferece on Neural Information Processing Systems (NeurIPS)"},{"key":"e_1_2_3_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323020"},{"key":"e_1_2_3_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459863"},{"key":"e_1_2_3_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.468400"},{"key":"e_1_2_3_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3503250"},{"key":"e_1_2_3_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530127"},{"key":"e_1_2_3_17_1","doi-asserted-by":"crossref","unstructured":"Jes\u00fas R Nieto and Antonio Sus\u00edn. 2012. Cage based deformations: a survey. In Deformation Models: Tracking Animation and Applications. 75--99.","DOI":"10.1007\/978-94-007-5446-1_3"},{"key":"e_1_2_3_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00581"},{"key":"e_1_2_3_19_1","unstructured":"Yicong Peng Yichao Yan Shengqi Liu Yuhao Cheng Shanyan Guan Bowen Pan Guangtao Zhai and Xiaokang Yang. 2022. CageNeRF: Cage-based Neural Radiance Field for Generalized 3D Deformation and Animation. In Conferece on Neural Information Processing Systems (NeurIPS). 14 pages."},{"key":"e_1_2_3_20_1","doi-asserted-by":"crossref","unstructured":"Juan Pineda. 1988. A parallel algorithm for polygon rasterization. In ACM SIGGRAPH. 17--20.","DOI":"10.1145\/378456.378457"},{"key":"e_1_2_3_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073239"},{"key":"e_1_2_3_22_1","volume-title":"D-NeRF: Neural Radiance Fields for Dynamic Scenes. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10318--10327","author":"Pumarola Albert","year":"2020","unstructured":"Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. 2020. D-NeRF: Neural Radiance Fields for Dynamic Scenes. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10318--10327."},{"key":"e_1_2_3_23_1","volume-title":"ANR-Articulated Neural Rendering for Virtual Avatars. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3722--3731","author":"Raj Amit","year":"2021","unstructured":"Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, and Christoph Lassner. 2021. ANR-Articulated Neural Rendering for Virtual Avatars. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3722--3731."},{"key":"e_1_2_3_24_1","volume-title":"Article 871808","author":"Rasmuson Sverker","year":"2022","unstructured":"Sverker Rasmuson, Erik Sintorn, and Ulf Assarsson. 2022. PERF: performant, explicit radiance fields. Frontiers in Computer Science 4, Article 871808 (2022), 11 pages."},{"key":"e_1_2_3_25_1","volume-title":"KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs. arXiv preprint arXiv:2103.13744","author":"Reiser Christian","year":"2021","unstructured":"Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. 2021. KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs. arXiv preprint arXiv:2103.13744 (2021)."},{"key":"e_1_2_3_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00542"},{"key":"e_1_2_3_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58621-8_35"},{"key":"e_1_2_3_28_1","volume-title":"Structure-from-Motion Revisited. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4104--4113","author":"Sch\u00f6nberger Johannes Lutz","year":"2016","unstructured":"Johannes Lutz Sch\u00f6nberger and Jan-Michael Frahm. 2016. Structure-from-Motion Revisited. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4104--4113."},{"key":"e_1_2_3_29_1","volume-title":"Neural Proxy: Empowering Neural","volume":"6","author":"Sin Zackary P. T.","year":"2021","unstructured":"Zackary P. T. Sin, Peter H. F. Ng, and Hong Va Leong. 2021. Neural Proxy: Empowering Neural Volume Rendering for Animation. In Pacic Conference on Computer Graphics and Applications (PG). 6 pages."},{"key":"e_1_2_3_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00254"},{"key":"e_1_2_3_31_1","unstructured":"Shih-Yang Su Frank Yu Michael Zollh\u00f6fer and Helge Rhodin. 2021. A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape Appearance and Pose. In Conferece on Neural Information Processing Systems (NeurIPS). 14 pages."},{"key":"e_1_2_3_32_1","volume-title":"Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5459--5469","author":"Sun Cheng","year":"2022","unstructured":"Cheng Sun, Min Sun, and Hwann-Tzong Chen. 2022. Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5459--5469."},{"key":"e_1_2_3_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14022"},{"key":"e_1_2_3_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_2_3_35_1","volume-title":"Barycentric coordinates for convex sets. Advances in computational mathematics 27, 3","author":"Warren Joe","year":"2007","unstructured":"Joe Warren, Scott Schaefer, Anil N Hirani, and Mathieu Desbrun. 2007. Barycentric coordinates for convex sets. Advances in computational mathematics 27, 3 (2007), 319--338."},{"key":"e_1_2_3_36_1","volume-title":"Space-time Neural Irradiance Fields for Free-Viewpoint Video. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9421--9431","author":"Xian Wenqi","year":"2021","unstructured":"Wenqi Xian, Jia-Bin Huang, Johannes Kopf, and Changil Kim. 2021. Space-time Neural Irradiance Fields for Free-Viewpoint Video. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9421--9431."},{"key":"e_1_2_3_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00704"},{"key":"e_1_2_3_38_1","volume-title":"Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering. In International Conference on Computer Vision (ICCV). 13779--13788","author":"Yang Bangbang","year":"2021","unstructured":"Bangbang Yang, Yinda Zhang, Yinghao Xu, Yijin Li, Han Zhou, Hujun Bao, Guofeng Zhang, and Zhaopeng Cui. 2021. Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering. In International Conference on Computer Vision (ICCV). 13779--13788."},{"key":"e_1_2_3_39_1","volume-title":"STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering. arXiv preprint arXiv:2101.01602","author":"Yuan Wentao","year":"2021","unstructured":"Wentao Yuan, Zhaoyang Lv, Tanner Schmidt, and Steven Lovegrove. 2021. STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering. arXiv preprint arXiv:2101.01602 (2021)."},{"key":"e_1_2_3_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01781"},{"key":"e_1_2_3_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_3_42_1","volume-title":"Image GANs meet Differentiate Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. In International Conference on Learning Representation (ICLR). 25 pages.","author":"Zhang Yuxuan","year":"2021","unstructured":"Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, and Sanja Fidler. 2021. Image GANs meet Differentiate Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. In International Conference on Learning Representation (ICLR). 25 pages."}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3585512","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,20]],"date-time":"2024-10-20T13:49:09Z","timestamp":1729432149000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3585512"}},"subtitle":["A Primitive for Animatable Neural Rendering with Interactive Speed"],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":42,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,5,12]]}},"alternative-id":["10.1145\/3585512"],"URL":"https:\/\/doi.org\/10.1145\/3585512","relation":{},"ISSN":["2577-6193"],"issn-type":[{"type":"electronic","value":"2577-6193"}],"subject":[],"published":{"date-parts":[[2023,5,12]]},"assertion":[{"value":"2023-05-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}