Image-Adaptive 3D Lookup Tables for Real-Time Image Enhancement with Bilateral Grids | SpringerLink
Skip to main content
Springer Nature Link
Log in

Image-Adaptive 3D Lookup Tables for Real-Time Image Enhancement with Bilateral Grids

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15107))

Included in the following conference series:

  • 293 Accesses

Abstract

Image enhancement and restoration methods using adaptive 3D lookup tables (3D LUTs) have shown promising results with real-time inferencing. These methods directly transform input pixel values into enhanced ones by using interpolation operations with predicted 3D LUT values. However, it is still challenging to deal with locally different properties of images since most 3D LUT methods are simple color-to-color transforms. Although including spatial information in this transform can be a good solution, it can significantly increase the number of parameters and inference time. To address this issue, we propose an efficient spatial-aware image enhancement model that combines bilateral grids and 3D LUTs. Specifically, we transform bilateral grids into a spatial feature domain to incorporate spatial information in our 3D LUT model. To reduce inference time and save parameters, we use slicing operations in our network architecture instead of the long decoding path of the U-Net architecture used in most existing studies. Our model achieves state-of-the-art performance without increasing parameters and further reduces inference time, as demonstrated by extensive results. Codes are available at https://github.com/WontaeaeKim/LUTwithBGrid

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8465
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/gejinchen/HDRnet-PyTorch.

References

  1. A Sharif, S., Naqvi, R.A., Biswas, M., Kim, S.: A two-stage deep network for high dynamic range image reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 550–559 (2021)

    Google Scholar 

  2. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  3. Akyüz, A.O., et al.: Deep joint deinterlacing and denoising for single shot dual-iso hdr reconstruction. IEEE Trans. Image Process. 29, 7511–7524 (2020)

    Article  Google Scholar 

  4. Bychkovsky, V., Paris, S., Chan, E., Durand, F.: Learning photographic global tonal adjustment with a database of input/output image pairs. In: CVPR 2011, pp. 97–104. IEEE (2011)

    Google Scholar 

  5. Chen, C., Chen, Q., Xu, J., Koltun, V.: Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3291–3300 (2018)

    Google Scholar 

  6. Chen, J., Adams, A., Wadhwa, N., Hasinoff, S.W.: Bilateral guided upsampling. ACM Trans. Graph. (TOG) 35(6), 1–8 (2016)

    Article  Google Scholar 

  7. Chen, J., Paris, S., Durand, F.: Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. (TOG) 26(3), 103–es (2007)

    Google Scholar 

  8. Chen, X., Liu, Y., Zhang, Z., Qiao, Y., Dong, C.: Hdrunet: single image hdr reconstruction with denoising and dequantization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 354–363 (2021)

    Google Scholar 

  9. Chen, Y.S., Wang, Y.C., Kao, M.H., Chuang, Y.Y.: Deep photo enhancer: unpaired learning for image enhancement from photographs with gans. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6306–6314 (2018)

    Google Scholar 

  10. Deng, Y., Loy, C.C., Tang, X.: Aesthetic-driven image enhancement by adversarial learning. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 870–878 (2018)

    Google Scholar 

  11. Gharbi, M., Chen, J., Barron, J.T., Hasinoff, S.W., Durand, F.: Deep bilateral learning for real-time image enhancement. ACM Trans. Graph. (TOG) 36(4), 1–12 (2017)

    Article  Google Scholar 

  12. Guo, C., Fan, L., Zhang, Q., Liu, H., Liu, K., Jiang, X.: Redistributing the precision and content in 3d-lut-based inverse tone-mapping for hdr/wcg display. In: Proceedings of the 20th ACM SIGGRAPH European Conference on Visual Media Production, pp. 1–10 (2023)

    Google Scholar 

  13. Hashimoto, N., Takamaeda-Yamazaki, S.: An fpga-based fully pipelined bilateral grid for real-time image denoising. In: 2021 31st International Conference on Field-Programmable Logic and Applications (FPL), pp. 167–173. IEEE (2021)

    Google Scholar 

  14. He, J., Liu, Y., Qiao, Yu., Dong, C.: Conditional sequential modulation for efficient global image retouching. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 679–695. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_40

    Chapter  Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  16. Hu, Y., He, H., Xu, C., Wang, B., Lin, S.: Exposure: a white-box photo post-processing framework. ACM Trans. Graph. (TOG) 37(2), 1–17 (2018)

    Article  Google Scholar 

  17. Huang, X., Zhang, Q., Feng, Y., Li, H., Wang, X., Wang, Q.: Hdr-nerf: high dynamic range neural radiance fields. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18398–18408 (2022)

    Google Scholar 

  18. Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1501–1510 (2017)

    Google Scholar 

  19. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., Van Gool, L.: Dslr-quality photos on mobile devices with deep convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3277–3285 (2017)

    Google Scholar 

  20. Jiang, Y., et al.: Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)

    Article  Google Scholar 

  21. Jiang, Y., et al.: Ssh: A self-supervised framework for image harmonization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4832–4841 (2021)

    Google Scholar 

  22. Karaimer, H.C., Brown, M.S.: A software platform for manipulating the camera imaging pipeline. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I, pp. 429–444. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_26

    Chapter  Google Scholar 

  23. Kim, H.-U., Koh, Y.J., Kim, C.-S.: PieNet: personalized image enhancement network. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX, pp. 374–390. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-58577-8_23

    Chapter  Google Scholar 

  24. g Kim, S.Y., Oh, J., Kim, M.: Jsi-gan: gan-based joint super-resolution and inverse tone-mapping with pixel-wise task-specific filters for uhd hdr video. In: Proceedings of the AAAI Conference on Artificial Intelligencem, pp. 11287–11295 (2020)

    Google Scholar 

  25. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  26. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  27. Le, P.H., Le, Q., Nguyen, R., Hua, B.S.: Single-image hdr reconstruction by multi-exposure generation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4063–4072 (2023)

    Google Scholar 

  28. Liang, J., Zeng, H., Cui, M., Xie, X., Zhang, L.: Ppr10k: A large-scale portrait photo retouching dataset with human-region mask and group-level consistency. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 653–661 (2021)

    Google Scholar 

  29. Liu, C., Yang, H., Fu, J., Qian, X.: 4d lut: learnable context-aware 4d lookup table for image enhancement. IEEE Trans. Image Process. 32, 4742–4756 (2023)

    Article  Google Scholar 

  30. Liu, Y.L., et al: Single-image hdr reconstruction by learning to reverse the camera pipeline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1651–1660 (2020)

    Google Scholar 

  31. Moran, S., Marza, P., McDonagh, S., Parisot, S., Slabaugh, G.: Deeplpf: deep local parametric filters for image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12826–12835 (2020)

    Google Scholar 

  32. Park, J., Lee, J.Y., Yoo, D., Kweon, I.S.: Distort-and-recover: color enhancement using deep reinforcement learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5928–5936 (2018)

    Google Scholar 

  33. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III, pp. 234–241. Springer International Publishing, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  34. Song, Y., Qian, H., Du, X.: Starenhancer: learning real-time and style-aware image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4126–4135 (2021)

    Google Scholar 

  35. Vandenberg, J.D., Andriani, S.: A survey on 3d-lut performance in 10-bit and 12-bit hdr bt. 2100 pq. In: SMPTE 2018, pp. 1–19. SMPTE (2018)

    Google Scholar 

  36. Wang, D., Zheng, Z., Ding, W., Jia, X.: Lgabl: Uhd multi-exposure image fusion via local and global aware bilateral learning. IEEE Transactions on Emerging Topics in Computational Intelligence (2023)

    Google Scholar 

  37. Wang, R., Zhang, Q., Fu, C.W., Shen, X., Zheng, W.S., Jia, J.: Underexposed photo enhancement using deep illumination estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6849–6857 (2019)

    Google Scholar 

  38. Wang, T., et al.: Real-time image enhancer via learnable spatial-aware 3d lookup tables. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2471–2480 (2021)

    Google Scholar 

  39. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  40. Wei, C., Wang, W., Yang, W., Liu, J.: Deep retinex decomposition for low-light enhancement. arXiv preprint arXiv:1808.04560 (2018)

  41. Xu, B., Xu, Y., Yang, X., Jia, W., Guo, Y.: Bilateral grid learning for stereo matching networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12497–12506 (2021)

    Google Scholar 

  42. Xu, Q., Wang, L., Wang, Y., Sheng, W., Deng, X.: Deep bilateral learning for stereo image super-resolution. IEEE Signal Process. Lett. 28, 613–617 (2021)

    Article  Google Scholar 

  43. Yan, Z., Zhang, H., Wang, B., Paris, S., Yu, Y.: Automatic photo adjustment using deep neural networks. ACM Trans. Graph. (TOG) 35(2), 1–15 (2016)

    Article  Google Scholar 

  44. Yang, C., Jin, M., Jia, X., Xu, Y., Chen, Y.: Adaint: learning adaptive intervals for 3d lookup tables on real-time image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17522–17531 (2022)

    Google Scholar 

  45. Yang, C., Jin, M., Xu, Y., Zhang, R., Chen, Y., Liu, H.: Seplut: separable image-adaptive lookup tables for real-time image enhancement. In: European Conference on Computer Vision (ECCV), pp. 201–217. Springer (2022). https://doi.org/10.1007/978-3-031-19797-0_12

  46. Zeng, H., Cai, J., Li, L., Cao, Z., Zhang, L.: Learning image-adaptive 3d lookup tables for high performance photo enhancement in real-time. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 44(4), 2058–2073 (2022)

    Google Scholar 

  47. Zhang, F., Zeng, H., Zhang, T., Zhang, L.: Clut-net: learning adaptively compressed representations of 3dluts for lightweight image enhancement. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 6493–6501 (2022)

    Google Scholar 

  48. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)

    Google Scholar 

  49. Zhang, Y., Zhang, J., Guo, X.: Kindling the darkness: a practical low-light image enhancer. In: Proceedings of the 27th ACM International Conference On Multimedia, pp. 1632–1640 (2019)

    Google Scholar 

  50. Zhang, Z., Jiang, Y., Jiang, J., Wang, X., Luo, P., Gu, J.: Star: A structure-aware lightweight transformer for real-time image enhancement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4106–4115 (2021)

    Google Scholar 

  51. Zheng, Z., Ren, W., Cao, X., Wang, T., Jia, X.: Ultra-high-definition image hdr reconstruction via collaborative bilateral learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4449–4458 (2021)

    Google Scholar 

  52. Zhou, Y., Li, C., Liang, J., Xu, T., Liu, X., Xu, J.: 4k-resolution photo exposure correction at 125 fps with\(\sim \) parameters. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1587–1597 (2024)

    Google Scholar 

Download references

Acknowledgements

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) [NO.RS-2021-II211343, Artificial Intelligence Graduate School Program (Seoul National University)] and in part by the BK21 FOUR program of the Education and Research Program for Future ICT Pioneers, Seoul National University in 2024.

Author information

Authors and Affiliations

  1. IPAI, Seoul National University, Seoul, Korea

    Wontae Kim & Nam Ik Cho

  2. Department of ECE, INMC, Seoul National University, Seoul, Korea

    Nam Ik Cho

  3. LG Electronics, Seoul, Korea

    Wontae Kim

Authors
  1. Wontae Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nam Ik Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wontae Kim .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 24998 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kim, W., Cho, N.I. (2025). Image-Adaptive 3D Lookup Tables for Real-Time Image Enhancement with Bilateral Grids. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15107. Springer, Cham. https://doi.org/10.1007/978-3-031-72967-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72967-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72966-9

  • Online ISBN: 978-3-031-72967-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics