{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T04:27:26Z","timestamp":1729139246828,"version":"3.27.0"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["804226"],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"Human visual sensitivity to spatial details declines towards the periphery. Novel image synthesis techniques, so-called foveated rendering, exploit this observation and reduce the spatial resolution of synthesized images for the periphery, avoiding the synthesis of high-spatial-frequency details that are costly to generate but not perceived by a viewer. However, contemporary techniques do not make a clear distinction between the range of spatial frequencies that must be reproduced and those that can be omitted. For a given eccentricity, there is a range of frequencies that are detectable but not resolvable. While the accurate reproduction of these frequencies is not required, an observer can detect their absence if completely omitted. We use this observation to improve the performance of existing foveated rendering techniques. We demonstrate that this specific range of frequencies can be efficiently replaced with procedural noise whose parameters are carefully tuned to image content and human perception. Consequently, these frequencies do not have to be synthesized during rendering, allowing more aggressive foveation, and they can be replaced by noise generated in a less expensive post-processing step, leading to improved performance of the rendering system. Our main contribution is a perceptually-inspired technique for deriving the parameters of the noise required for the enhancement and its calibration. The method operates on rendering output and runs at rates exceeding 200 FPS at 4K resolution, making it suitable for integration with real-time foveated rendering systems for VR and AR devices. We validate our results and compare them to the existing contrast enhancement technique in user experiments.<\/jats:p>","DOI":"10.1145\/3528223.3530101","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-14","update-policy":"http:\/\/dx.doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Noise-based enhancement for foveated rendering"],"prefix":"10.1145","volume":"41","author":[{"given":"Taimoor","family":"Tariq","sequence":"first","affiliation":[{"name":"Universit\u00e0 della Svizzera italiana, Switzerland"}]},{"given":"Cara","family":"Tursun","sequence":"additional","affiliation":[{"name":"Universit\u00e0 della Svizzera italiana, Switzerland and University of Groningen, Netherlands"}]},{"given":"Piotr","family":"Didyk","sequence":"additional","affiliation":[{"name":"Universit\u00e0 della Svizzera italiana, Switzerland"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3127589"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(02)00013-5"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1991.sp018781"},{"key":"e_1_2_2_4_1","volume-title":"Beitrage zur kenntnisse der indirecten sehens [Translation: Contributions of knowledge to indirect vision]. Graefes Archiv fur Ophthalmologie 3","author":"Aubert H. R.","year":"1857","unstructured":"H. R. Aubert and C. F. R. Foerster. 1857. Beitrage zur kenntnisse der indirecten sehens [Translation: Contributions of knowledge to indirect vision]. Graefes Archiv fur Ophthalmologie 3 (1857)."},{"key":"e_1_2_2_5_1","volume-title":"Possible principles underlying the transformation of sensory messages. Sensory communication 1, 01","author":"Barlow Horace B.","year":"1961","unstructured":"Horace B. Barlow. 1961. Possible principles underlying the transformation of sensory messages. Sensory communication 1, 01 (1961)."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR46266.2020.00108"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01747.x"},{"volume-title":"Proc. of Flight Simulation Technologies Conference.","author":"Browder G.","key":"e_1_2_2_8_1","unstructured":"G. Browder and W. Chambers. 1988. Eye-enslaved area-of-interest display systems. In Proc. of Flight Simulation Technologies Conference."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/cne.902920402"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(82)90113-4"},{"volume-title":"International Conference on Learning Representations. Openreview.net","author":"Deza Arturo","key":"e_1_2_2_11_1","unstructured":"Arturo Deza, Aditya Jonnalagadda, and Miguel P. Eckstein. 2019. Towards metamerism via foveated style transfer. In International Conference on Learning Representations. Openreview.net, New Orleans, USA."},{"key":"e_1_2_2_12_1","unstructured":"Piotr Didyk Tobias Ritschel Elmar Eisemann Karol Myszkowski and Hans-Peter Seidel. 2010. Adaptive image-space stereo view synthesis. In VMV. 299--306."},{"key":"e_1_2_2_13_1","volume-title":"Ueber st\u00e4bchensehsch\u00e4rfe und zapfensehsch\u00e4rfe. Albrecht von Graefes Archiv f\u00fcr Ophthalmologie 45, 2","author":"Fick A. Eugen","year":"1898","unstructured":"A. Eugen Fick. 1898. Ueber st\u00e4bchensehsch\u00e4rfe und zapfensehsch\u00e4rfe. Albrecht von Graefes Archiv f\u00fcr Ophthalmologie 45, 2 (1898), 336--356."},{"key":"e_1_2_2_14_1","volume-title":"Simoncelli","author":"Freeman Jeremy","year":"2011","unstructured":"Jeremy Freeman and Eero P. Simoncelli. 2011. Metamers of the ventral stream. Nature neuroscience 14, 9 (2011), 1195--1201."},{"volume-title":"Visual Science and Engineering: Models and Applications","author":"Glenn William E.","key":"e_1_2_2_15_1","unstructured":"William E. Glenn. 1994. Real-time display systems, present and future. In Visual Science and Engineering: Models and Applications. Marcel Dekker, 387--413."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1970.sp009065"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366183"},{"key":"e_1_2_2_18_1","volume-title":"What is visible across the visual field? Neuroscience of Consciousness","author":"Haun Andrew M.","year":"2021","unstructured":"Andrew M. Haun. 2021. What is visible across the visual field? Neuroscience of Consciousness 2021, 1 (06 2021)."},{"key":"e_1_2_2_19_1","volume-title":"Concerning the limits of visual acuity. Ber. d. math.-phys. Kl. d. K. Sachs. Gesellsch. d. Wissensch. zu Leipzig","author":"Hering E.","year":"1899","unstructured":"E. Hering. 1899. Concerning the limits of visual acuity. Ber. d. math.-phys. Kl. d. K. Sachs. Gesellsch. d. Wissensch. zu Leipzig (1899)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(89)90058-8"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/jsid.730"},{"key":"e_1_2_2_22_1","volume-title":"Hoyer","author":"Hyv\u00e4rinen Aapo","year":"2009","unstructured":"Aapo Hyv\u00e4rinen, Jarmo Hurri, and Patrick O. Hoyer. 2009. Natural image statistics: A probabilistic approach to early computational vision. Vol. 39. Springer Science & Business Media."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356557"},{"key":"e_1_2_2_24_1","volume-title":"Geisler","author":"Kortum Philip","year":"1996","unstructured":"Philip Kortum and Wilson S. Geisler. 1996. Implementation of a foveated image coding system for image bandwidth reduction. In Human Vision and Electronic Imaging, Vol. 2657. International Society for Optics and Photonics, 350--360."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01827.x"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531360"},{"key":"e_1_2_2_27_1","first-page":"8","article-title":"Contrast discrimination in peripheral vision","volume":"4","author":"Legge Gordon E.","year":"1987","unstructured":"Gordon E. Legge and Daniel Kersten. 1987. Contrast discrimination in peripheral vision. J. Opt. Soc. Am. A 4, 8 (Aug 1987), 1594--1598.","journal-title":"J. Opt. Soc. Am. A"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2326-1951.1976.tb01231.x"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1038\/320360a0"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/91394.91449"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964935"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459831"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3203199"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(75)90190-X"},{"key":"e_1_2_2_35_1","volume-title":"An integrative view of foveated rendering. Computers & Graphics","author":"Mohanto Bipul","year":"2021","unstructured":"Bipul Mohanto, A. B. M. Tariqul Islam, Enrico Gobbetti, and Oliver Staadt. 2021. An integrative view of foveated rendering. Computers & Graphics (2021)."},{"key":"e_1_2_2_36_1","volume-title":"Topography of the layer of the rods and cones in the human retina. Acta ophthalmol 13, 6","author":"\u00d8sterberg G. A.","year":"1935","unstructured":"G. A. \u00d8sterberg. 1935. Topography of the layer of the rods and cones in the human retina. Acta ophthalmol 13, 6 (1935)."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980246"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.7.002032"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026553619983"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-vision-082114-035733"},{"key":"e_1_2_2_41_1","volume-title":"Rossi and Austin Roorda","author":"Ethan","year":"2010","unstructured":"Ethan A. Rossi and Austin Roorda. 2010. The relationship between visual resolution and cone spacing in the human fovea. Nature neuroscience 13, 2 (2010), 156--157."},{"key":"e_1_2_2_42_1","volume-title":"An estimation and application of the human cortical magnification factor. Experimental brain research 37, 3","author":"Rovamo Jyrki","year":"1979","unstructured":"Jyrki Rovamo and Veijo Virsu. 1979. An estimation and application of the human cortical magnification factor. Experimental brain research 37, 3 (1979), 495--510."},{"key":"e_1_2_2_43_1","volume-title":"Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision. Nature 271, 5640","author":"Rovamo Jyrki","year":"1978","unstructured":"Jyrki Rovamo, Veijo Virsu, and Risto N\u00e4s\u00e4nen. 1978. Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision. Nature 271, 5640 (1978), 54--56."},{"key":"e_1_2_2_44_1","volume-title":"The statistics of natural images. Network: computation in neural systems 5, 4","author":"Ruderman Daniel L.","year":"1994","unstructured":"Daniel L. Ruderman. 1994. The statistics of natural images. Network: computation in neural systems 5, 4 (1994), 517--548."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1167\/17.12.12"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1068\/p2991"},{"key":"e_1_2_2_47_1","volume-title":"Olshausen","author":"Simoncelli Eero P.","year":"2001","unstructured":"Eero P. Simoncelli and Bruno A. Olshausen. 2001. Natural image statistics and neural representation. Annual review of neuroscience 24, 1 (2001), 1193--1216."},{"volume-title":"Comput Graph Forum","author":"Stengel Michael","key":"e_1_2_2_48_1","unstructured":"Michael Stengel, Steve Grogorick, Martin Eisemann, and Marcus Magnor. 2016. Adaptive image-space sampling for gaze-contingent real-time rendering. In Comput Graph Forum, Vol. 35. Wiley Online Library, 129--139."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1167\/11.5.13"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931002.2931011"},{"key":"e_1_2_2_51_1","volume-title":"Acuity perimetry and the sampling theory of visual resolution. Optometry and vision science: official publication of the American Academy of Optometry 75, 6","author":"Thibos L. N.","year":"1998","unstructured":"L. N. Thibos. 1998. Acuity perimetry and the sampling theory of visual resolution. Optometry and vision science: official publication of the American Academy of Optometry 75, 6 (1998), 399--406."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.4.001524"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(95)00109-D"},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","unstructured":"L. N. Thibos and D. J. Walsh. 1985. Detection of high frequency gratings in the periphery. J. Opt. Soc. Am. A 2 (1985).","DOI":"10.1364\/OAM.1985.THH4"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(87)90134-9"},{"key":"e_1_2_2_56_1","unstructured":"H. M. Tong and R. A. Fisher. 1984. Progress report on an eye-slaved area-of-interest visual display. Technical Report. Singer Co Silver Spring Md Link Simul. Systems Div."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.238733"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322985"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.4.001568"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459943"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_62_1","volume-title":"The field of view, the field of resolution, and the field of contrast sensitivity. Electronic Imaging","author":"Watson Andrew B.","year":"2018","unstructured":"Andrew B. Watson. 2018. The field of view, the field of resolution, and the field of contrast sensitivity. Electronic Imaging 2018, 14 (2018)."},{"key":"e_1_2_2_63_1","volume-title":"Ahumada","author":"Watson Andrew B.","year":"2016","unstructured":"Andrew B. Watson and Albert J. Ahumada. 2016. The pyramid of visibility. Electronic Imaging 2016, 16 (2016)."},{"key":"e_1_2_2_64_1","first-page":"172","article-title":"Uber die indirekte sehscharfe","volume":"7","author":"Wertheim T. H.","year":"1894","unstructured":"T. H. Wertheim. 1894. Uber die indirekte sehscharfe. Zeitschrift fur Psychologie 7 (1894), 172--187.","journal-title":"Zeitschrift fur Psychologie"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(85)90113-0"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.2.001087"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530101","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T17:44:31Z","timestamp":1729100671000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":66,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530101"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530101","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"type":"print","value":"0730-0301"},{"type":"electronic","value":"1557-7368"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}