Deep hybrid order-independent transparency | The Visual Computer
Skip to main content
Springer Nature Link
Log in

Deep hybrid order-independent transparency

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Correctly compositing transparent fragments is an important and long-standing open problem in real-time computer graphics. Multifragment rendering is considered a key solution to providing high-quality order-independent transparency at interactive frame rates. To achieve that, practical implementations severely constrain the overall memory budget by adopting bounded fragment configurations such as the k-buffer. Relying on an iterative trial-and-error procedure, however, where the value of k is manually configured per case scenario, can inevitably result in bad memory utilization and view-dependent artifacts. To this end, we introduce a novel intelligent k-buffer approach that performs a non-uniform per pixel fragment allocation guided by a deep learning prediction mechanism. A hybrid scheme is further employed to facilitate the approximate blending of non-significant (remaining) fragments and thus contribute to a better overall final color estimation. An experimental evaluation substantiates that our method outperforms previous approaches when evaluating transparency in various high-depth-complexity scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. Shader source code available at: https://github.com/gtsopus/dhoit.

References

  1. Andersson, P., Nilsson, J., Akenine-Möller, T., Oskarsson, M., Åström, K., Fairchild, M.D.: FLIP: a difference evaluator for alternating images. In: Proceedings of the ACM on computer graphics and interactive techniques 3(2), 15:1-15:23 (2020). https://doi.org/10.1145/3406183

  2. Bavoil, L., Callahan, S.P., Lefohn, A., Comba, J.a.L.D., Silva, C.T.: Multi-fragment effects on the gpu using the k-buffer. In: Proceedings of the 2007 symposium on interactive 3D graphics and games, I3D ’07, . Association for computing machinery, New York, NY, USA. pp 97–104 (2007). https://doi.org/10.1145/1230100.1230117

  3. Bavoil, L., Myers, K.: Order independent transparency with dual depth peeling (2008)

  4. Carpenter, L.: The A-buffer, an antialiased hidden surface method. SIGGRAPH Comput. Graph. 18(3), 103–108 (1984). https://doi.org/10.1145/964965.808585

    Article  Google Scholar 

  5. Catmull, E.E.: A subdivision algorithm for computer display of curved surfaces. Ph.D. thesis, The University of Utah (1974)

  6. Lavoué, G., Larabi, M.C., Váša, L.: On the efficiency of image metrics for evaluating the visual quality of 3d models. IEEE Trans. Vis. Comput. Gr. 22(8), 1987–1999 (2016). https://doi.org/10.1109/TVCG.2015.2480079

    Article  Google Scholar 

  7. Liu, F., Huang, M.C., Liu, X.H., Wu, E.H.: Freepipe: A programmable parallel rendering architecture for efficient multi-fragment effects. In: Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, I3D ’10, Association for Computing Machinery, New York, NY, USA. pp 75–82(2010). https://doi.org/10.1145/1730804.1730817

  8. Maule, M., Comba, J.a., Torchelsen, R., Bastos, R.: Hybrid transparency. In: Proceedings of the ACM SIGGRAPH symposium on interactive 3D graphics and games, I3D ’13, Association for computing machinery, New York, NY, USA. pp 103–118 (2013). https://doi.org/10.1145/2448196.2448212

  9. Maule, M., Comba, J.L., Torchelsen, R.P., Bastos, R.: A survey of raster-based transparency techniques. Comput. Gr. 35(6), 1023–1034 (2011). https://doi.org/10.1016/j.cag.2011.07.006

    Article  Google Scholar 

  10. McGuire, M.: Computer graphics archive (2017). https://casual-effects.com/data

  11. Nalbach, O., Arabadzhiyska, E., Mehta, D., Seidel, H.P., Ritschel, T.: Deep shading: convolutional neural networks for screen space shading. Comput. Gr. Forum 36(4), 65–78 (2017). https://doi.org/10.1111/cgf.13225

    Article  Google Scholar 

  12. Porter, T., Duff, T.: Compositing digital images. In: Proceedings of the 11th annual conference on computer graphics and interactive techniques, SIGGRAPH ’84, Association for computing machinery, New York, NY, USA p. 253–259(1984). https://doi.org/10.1145/800031.808606

  13. Rossignac, J., Fudos, I., Vasilakis, A.: Direct rendering of boolean combinations of self-trimmed surfaces. Comput. Aided Des. 45(2), 288–300 (2013). https://doi.org/10.1016/j.cad.2012.10.012

    Article  MathSciNet  Google Scholar 

  14. Salvi, M., Montgomery, J., Lefohn, A.: Adaptive transparency. In: Eurographics/ ACM SIGGRAPH symposium on high performance graphics. ACM (2011). https://doi.org/10.1145/2018323.2018342

  15. Salvi, M., Vaidyanathan, K.: Multi-layer alpha blending. In: Proceedings of the 18th meeting of the ACM SIGGRAPH symposium on interactive 3D graphics and games, I3D ’14, Association for computing machinery, New York, NY, USA. pp. 151–158 (2014). https://doi.org/10.1145/2556700.2556705

  16. Tewari, A., Fried, O., Thies, J., Sitzmann, V., Lombardi, S., Sunkavalli, K., Martin-Brualla, R., Simon, T., Saragih, J., NieSSner, M., Pandey, R., Fanello, S., Wetzstein, G., Zhu, J.Y., Theobalt, C., Agrawala, M., Shechtman, E., Goldman, D.B., Zollhöfer, M.: State of the art on neural rendering. Comput. Gr. Forum 39(2), 701–727 (2020). https://doi.org/10.1111/cgf.14022

    Article  Google Scholar 

  17. Thomas, M.M., Forbes, A.G.: Deep illumination: Approximating dynamic global illumination with generative adversarial network. CoRR. (2017)

  18. Vardis, K., Vasilakis, A.A., Papaioannou, G.: A multiview and multilayer approach for interactive ray tracing. In: Proceedings of the 20th ACM SIGGRAPH symposium on interactive 3D graphics and games, I3D ’16, Association for computing machinery, New York, NY, USA. pp. 171–178 (2016). https://doi.org/10.1145/2856400.2856401

  19. Vasilakis, A., Fudos, I.: S-buffer: sparsity-aware multi-fragment rendering. In: Eurographics (Short Papers), The Eurographics Association, Cagliari, Sardinia pp. 101–104 (2012). https://doi.org/10.2312/conf/EG2012/short/101-104

  20. Vasilakis, A.A., Fudos, I.: \(k^+\)-buffer: fragment synchronized k-buffer. In: Proceedings of the 18th meeting of the ACM SIGGRAPH symposium on interactive 3D graphics and games, I3D ’14, Association for computing machinery, New York, NY, USA. pp. 143–150 (2014). https://doi.org/10.1145/2556700.2556702

  21. Vasilakis, A.A., Papaioannou, G., Fudos, I.: \(k^+\)-buffer: an efficient, memory-friendly and dynamic \(k\)-buffer framework. IEEE Trans. Vis. Comput. Graphics 21(6), 688–700 (2015). https://doi.org/10.1109/TVCG.2015.2417581

    Article  Google Scholar 

  22. Vasilakis, A.A., Vardis, K., Papaioannou, G.: A survey of multifragment rendering. Comput. Gr. Forum 39(2), 623–642 (2020). https://doi.org/10.1111/cgf.14019

    Article  Google Scholar 

  23. Vasilakis, A.A., Vardis, K., Papaioannou, G., Moustakas, K.: Variable k-buffer using importance maps. In: Proceedings of the European association for computer graphics: Short Papers, EG ’17, Eurographics Association, Goslar, DEU. pp. 21–24 (2017). https://doi.org/10.2312/egsh.20171005

  24. Wyman, C.: Exploring and expanding the continuum of oit algorithms. In: Proceedings of high performance graphics, HPG ’16, Eurographics Association, Goslar, DEU. pp. 1–11(2016). https://doi.org/10.2312/hpg.20161187

  25. Yang, J.C., Hensley, J., Grün, H., Thibieroz, N.: Real-time concurrent linked list construction on the gpu. In: Proceedings of the 21st Eurographics conference on rendering, EGSR’10, Eurographics Association, Goslar, DEU. pp. 1297–1304 (2010). https://doi.org/10.1111/j.1467-8659.2010.01725.x

  26. Zhang, D., Xian, C., Luo, G., Xiong, Y., Han, C.: Deepao: efficient screen space ambient occlusion generation via deep network. IEEE Access 8, 64434–64441 (2020). https://doi.org/10.1109/ACCESS.2020.2984771

    Article  Google Scholar 

Download references

Acknowledgements

Lost Empire, Vokselia Spawn, Rungholt and Crytek Sponza were downloaded from Morgan McGuire’s Computer Graphics Archive [10].

Funding

This research was supported by project “Dioni: Computing Infrastructure for Big-Data Processing and Analysis” (MIS No. 5047222) co-funded by European Union (ERDF) and Greece through Operational Program “Competitiveness, Entrepreneurship and Innovation,” NSRF 2014-2020. This work has been co-financed by the European Union (European Regional Development Fund- ERDF) and Greek national funds through the Interreg Greece Albania 2014-2020 Program (project VirtuaLand).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Ioannina, Ioannina, Greece

    Grigoris Tsopouridis, Ioannis Fudos & Andreas-Alexandros Vasilakis

  2. Department of Informatics, Athens University of Economics and Business, Athens, Greece

    Andreas-Alexandros Vasilakis

Authors
  1. Grigoris Tsopouridis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioannis Fudos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas-Alexandros Vasilakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ioannis Fudos.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (mp4 146800 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsopouridis, G., Fudos, I. & Vasilakis, AA. Deep hybrid order-independent transparency. Vis Comput 38, 3289–3300 (2022). https://doi.org/10.1007/s00371-022-02562-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02562-7

Keywords

Search

Navigation