Abstract
The widespread use of vector graphics creates a significant demand for vectorization methods. While recent learning-based techniques have shown their capability to create vector images of clear topology, filling these primitives with gradients remains a challenge. In this paper, we propose a segmentation-guided vectorization framework to convert raster images into concise vector graphics with radial gradient fills. With the guidance of an embedded gradient-aware segmentation subroutine, our approach progressively appends gradient-filled Bézier paths to the output, where primitive parameters are initiated with our newly designed initialization technique and are optimized to minimize our novel loss function. We build our method on a differentiable renderer with traditional segmentation algorithms to develop it as a model-free tool for raster-to-vector conversion. It is tested on various inputs to demonstrate its feasibility, independent of datasets, to synthesize vector graphics with improved visual quality and layer-wise topology compared to prior work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Baksteen, S.D., Hettinga, G.J., Echevarria, J., Kosinka, J.: Mesh colours for gradient meshes. STAG: Smart Tools and Applications in Graphics (2021)
Beucher, S., Meyer, F.: The morphological approach to segmentation: the watershed transformation. Math. Morphol. Image Process. 34(1993), 49 (1993)
Carlier, A., Danelljan, M., Alahi, A., Timofte, R.: DeepSVG: a hierarchical generative network for vector graphics animation. In: Advances in Neural Information Processing Systems, vol. 33, pp. 16351–16361 (2020)
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)
Du, Z.J., Kang, L.F., Tan, J., Gingold, Y., Xu, K.: Image vectorization and editing via linear gradient layer decomposition. ACM Trans. Graph. (TOG) 42(4), 1–13 (2023)
Egiazarian, V., et al.: Deep vectorization of technical drawings. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12358, pp. 582–598. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58601-0_35
Favreau, J.D., Lafarge, F., Bousseau, A.: Photo2clipart: image abstraction and vectorization using layered linear gradients. ACM Trans. Graph. (TOG) 36(6), 1–11 (2017)
Frans, K., Soros, L., Witkowski, O.: CLIPDraw: exploring text-to-drawing synthesis through language-image encoders. In: Advances in Neural Information Processing Systems, vol. 35, pp. 5207–5218 (2022)
Noto emoji. https://github.com/googlefonts/noto-emoji. Accessed 19 Sept 2023
Ha, D., Eck, D.: A neural representation of sketch drawings. In: International Conference on Learning Representations (2018)
Haralick, R.M., Sternberg, S.R., Zhuang, X.: Image analysis using mathematical morphology. IEEE Trans. Pattern Anal. Mach. Intell. 4, 532–550 (1987)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Vector illustrations library. https://www.iconfont.cn/illustrations/index. Accessed 02 Nov 2023
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: International Conference on Learning Representations (2015)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. In: International Conference on Learning Representations (2014)
Li, T.M., Lukáč, M., Gharbi, M., Ragan-Kelley, J.: Differentiable vector graphics rasterization for editing and learning. ACM Trans. Graph. (TOG) 39(6), 1–15 (2020)
Liao, Z., Hoppe, H., Forsyth, D., Yu, Y.: A subdivision-based representation for vector image editing. IEEE Trans. Visual Comput. Graph. 18(11), 1858–1867 (2012)
Liu, Y.T., Zhang, Z., Guo, Y.C., Fisher, M., Wang, Z., Zhang, S.H.: DualVector: unsupervised vector font synthesis with dual-part representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14193–14202 (2023)
Lopes, R.G., Ha, D., Eck, D., Shlens, J.: A learned representation for scalable vector graphics. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, Korea (South), pp. 7929–7938. IEEE (2019)
Ma, X., et al.: Towards layer-wise image vectorization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16314–16323 (2022)
Fluent emoji. https://github.com/microsoft/fluentui-emoji. Accessed 27 Oct 2023
Orzan, A., Bousseau, A., Winnemöller, H., Barla, P., Thollot, J., Salesin, D.: Diffusion curves: a vector representation for smooth-shaded images. ACM Trans. Graph. (TOG) 27(3), 1–8 (2008)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Reddy, P., Gharbi, M., Lukac, M., Mitra, N.J.: Im2vec: synthesizing vector graphics without vector supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7342–7351 (2021)
Reddy, P., Zhang, Z., Wang, Z., Fisher, M., Jin, H., Mitra, N.: A multi-implicit neural representation for fonts. In: Advances in Neural Information Processing Systems, vol. 34, pp. 12637–12647 (2021)
Richardt, C., Lopez-Moreno, J., Bousseau, A., Agrawala, M., Drettakis, G.: Vectorising bitmaps into semi-transparent gradient layers. Comput. Graph. Forum (Proc. EGSR) 33(4), 11–19 (2014)
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695 (2022)
Su, H., et al.: MARVEL: raster gray-level manga vectorization via primitive-wise deep reinforcement learning. IEEE Trans. Circuits Syst. Video Technol. (2023)
Sun, J., Liang, L., Wen, F., Shum, H.Y.: Image vectorization using optimized gradient meshes. ACM Trans. Graph. (TOG) 26(3), 11–es (2007)
Tian, X., Günther, T.: A survey of smooth vector graphics: recent advances in representation, creation, rasterization and image vectorization. IEEE Trans. Vis. Comput. Graph. (2022)
Van der Walt, S., et al.: scikit-image: image processing in python. PeerJ 2, e453 (2014)
Wang, Y., Lian, Z.: DeepVecFont: synthesizing high-quality vector fonts via dual-modality learning. ACM Trans. Graph. (TOG) 40(6), 1–15 (2021)
Xia, T., Liao, B., Yu, Y.: Patch-based image vectorization with automatic curvilinear feature alignment. ACM Trans. Graph. (TOG) 28(5), 1–10 (2009)
Xie, G., Sun, X., Tong, X., Nowrouzezahrai, D.: Hierarchical diffusion curves for accurate automatic image vectorization. ACM Trans. Graph. (TOG) 33(6), 1–11 (2014)
Yang, M., Chao, H., Zhang, C., Guo, J., Yuan, L., Sun, J.: Effective clipart image vectorization through direct optimization of bezigons. IEEE Trans. Visual Comput. Graph. 22(2), 1063–1075 (2015)
Zhu, H., Cao, J., Xiao, Y., Chen, Z., Zhong, Z., Zhang, Y.J.: TCB-spline-based image vectorization. ACM Trans. Graph. (TOG) 41(3), 1–17 (2022)
Acknowledgements
This work was supported by the NSFC under Grant 62072271.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhou, H., Zhang, H., Wang, B. (2025). Segmentation-Guided Layer-Wise Image Vectorization with Gradient Fills. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15068. Springer, Cham. https://doi.org/10.1007/978-3-031-72684-2_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-72684-2_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72683-5
Online ISBN: 978-3-031-72684-2
eBook Packages: Computer ScienceComputer Science (R0)