Segmentation-Guided Layer-Wise Image Vectorization with Gradient Fills | SpringerLink
Skip to main content

Segmentation-Guided Layer-Wise Image Vectorization with Gradient Fills

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Abstract

The widespread use of vector graphics creates a significant demand for vectorization methods. While recent learning-based techniques have shown their capability to create vector images of clear topology, filling these primitives with gradients remains a challenge. In this paper, we propose a segmentation-guided vectorization framework to convert raster images into concise vector graphics with radial gradient fills. With the guidance of an embedded gradient-aware segmentation subroutine, our approach progressively appends gradient-filled Bézier paths to the output, where primitive parameters are initiated with our newly designed initialization technique and are optimized to minimize our novel loss function. We build our method on a differentiable renderer with traditional segmentation algorithms to develop it as a model-free tool for raster-to-vector conversion. It is tested on various inputs to demonstrate its feasibility, independent of datasets, to synthesize vector graphics with improved visual quality and layer-wise topology compared to prior work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 8465
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 10581
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Baksteen, S.D., Hettinga, G.J., Echevarria, J., Kosinka, J.: Mesh colours for gradient meshes. STAG: Smart Tools and Applications in Graphics (2021)

    Google Scholar 

  2. Beucher, S., Meyer, F.: The morphological approach to segmentation: the watershed transformation. Math. Morphol. Image Process. 34(1993), 49 (1993)

    Google Scholar 

  3. Carlier, A., Danelljan, M., Alahi, A., Timofte, R.: DeepSVG: a hierarchical generative network for vector graphics animation. In: Advances in Neural Information Processing Systems, vol. 33, pp. 16351–16361 (2020)

    Google Scholar 

  4. Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)

    Article  Google Scholar 

  5. Du, Z.J., Kang, L.F., Tan, J., Gingold, Y., Xu, K.: Image vectorization and editing via linear gradient layer decomposition. ACM Trans. Graph. (TOG) 42(4), 1–13 (2023)

    Article  Google Scholar 

  6. Egiazarian, V., et al.: Deep vectorization of technical drawings. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 582–598. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_35

    Chapter  Google Scholar 

  7. Favreau, J.D., Lafarge, F., Bousseau, A.: Photo2clipart: image abstraction and vectorization using layered linear gradients. ACM Trans. Graph. (TOG) 36(6), 1–11 (2017)

    Article  Google Scholar 

  8. Frans, K., Soros, L., Witkowski, O.: CLIPDraw: exploring text-to-drawing synthesis through language-image encoders. In: Advances in Neural Information Processing Systems, vol. 35, pp. 5207–5218 (2022)

    Google Scholar 

  9. Noto emoji. https://github.com/googlefonts/noto-emoji. Accessed 19 Sept 2023

  10. Ha, D., Eck, D.: A neural representation of sketch drawings. In: International Conference on Learning Representations (2018)

    Google Scholar 

  11. Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 4, 532–550 (1987)

    Article  Google Scholar 

  12. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  13. Vector illustrations library. https://www.iconfont.cn/illustrations/index. Accessed 02 Nov 2023

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)

    Google Scholar 

  15. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (2014)

    Google Scholar 

  16. Li, T.M., Lukáč, M., Gharbi, M., Ragan-Kelley, J.: Differentiable vector graphics rasterization for editing and learning. ACM Trans. Graph. (TOG) 39(6), 1–15 (2020)

    Article  Google Scholar 

  17. Liao, Z., Hoppe, H., Forsyth, D., Yu, Y.: A subdivision-based representation for vector image editing. IEEE Trans. Visual Comput. Graph. 18(11), 1858–1867 (2012)

    Article  Google Scholar 

  18. Liu, Y.T., Zhang, Z., Guo, Y.C., Fisher, M., Wang, Z., Zhang, S.H.: DualVector: unsupervised vector font synthesis with dual-part representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14193–14202 (2023)

    Google Scholar 

  19. Lopes, R.G., Ha, D., Eck, D., Shlens, J.: A learned representation for scalable vector graphics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, Korea (South), pp. 7929–7938. IEEE (2019)

    Google Scholar 

  20. Ma, X., et al.: Towards layer-wise image vectorization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16314–16323 (2022)

    Google Scholar 

  21. Fluent emoji. https://github.com/microsoft/fluentui-emoji. Accessed 27 Oct 2023

  22. Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., Salesin, D.: Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. (TOG) 27(3), 1–8 (2008)

    Article  Google Scholar 

  23. Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  24. Reddy, P., Gharbi, M., Lukac, M., Mitra, N.J.: Im2vec: synthesizing vector graphics without vector supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7342–7351 (2021)

    Google Scholar 

  25. Reddy, P., Zhang, Z., Wang, Z., Fisher, M., Jin, H., Mitra, N.: A multi-implicit neural representation for fonts. In: Advances in Neural Information Processing Systems, vol. 34, pp. 12637–12647 (2021)

    Google Scholar 

  26. Richardt, C., Lopez-Moreno, J., Bousseau, A., Agrawala, M., Drettakis, G.: Vectorising bitmaps into semi-transparent gradient layers. Comput. Graph. Forum (Proc. EGSR) 33(4), 11–19 (2014)

    Article  Google Scholar 

  27. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695 (2022)

    Google Scholar 

  28. Su, H., et al.: MARVEL: raster gray-level manga vectorization via primitive-wise deep reinforcement learning. IEEE Trans. Circuits Syst. Video Technol. (2023)

    Google Scholar 

  29. Sun, J., Liang, L., Wen, F., Shum, H.Y.: Image vectorization using optimized gradient meshes. ACM Trans. Graph. (TOG) 26(3), 11–es (2007)

    Google Scholar 

  30. Tian, X., Günther, T.: A survey of smooth vector graphics: recent advances in representation, creation, rasterization and image vectorization. IEEE Trans. Vis. Comput. Graph. (2022)

    Google Scholar 

  31. Van der Walt, S., et al.: scikit-image: image processing in python. PeerJ 2, e453 (2014)

    Article  Google Scholar 

  32. Wang, Y., Lian, Z.: DeepVecFont: synthesizing high-quality vector fonts via dual-modality learning. ACM Trans. Graph. (TOG) 40(6), 1–15 (2021)

    Google Scholar 

  33. Xia, T., Liao, B., Yu, Y.: Patch-based image vectorization with automatic curvilinear feature alignment. ACM Trans. Graph. (TOG) 28(5), 1–10 (2009)

    Article  Google Scholar 

  34. Xie, G., Sun, X., Tong, X., Nowrouzezahrai, D.: Hierarchical diffusion curves for accurate automatic image vectorization. ACM Trans. Graph. (TOG) 33(6), 1–11 (2014)

    Article  Google Scholar 

  35. Yang, M., Chao, H., Zhang, C., Guo, J., Yuan, L., Sun, J.: Effective clipart image vectorization through direct optimization of bezigons. IEEE Trans. Visual Comput. Graph. 22(2), 1063–1075 (2015)

    Article  Google Scholar 

  36. Zhu, H., Cao, J., Xiao, Y., Chen, Z., Zhong, Z., Zhang, Y.J.: TCB-spline-based image vectorization. ACM Trans. Graph. (TOG) 41(3), 1–17 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the NSFC under Grant 62072271.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Hui Zhang or Bin Wang .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 20109 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhou, H., Zhang, H., Wang, B. (2025). Segmentation-Guided Layer-Wise Image Vectorization with Gradient Fills. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15068. Springer, Cham. https://doi.org/10.1007/978-3-031-72684-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72684-2_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72683-5

  • Online ISBN: 978-3-031-72684-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics