Toward accurate and realistic garment texture transfer with attention to details | Neural Computing and Applications Skip to main content
Springer Nature Link
Log in

Toward accurate and realistic garment texture transfer with attention to details

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The categories and styles of garment are constantly diversifying. For designers, it is a pressing issue to evaluate how different fabrics will look on them in a timely manner for users. In this paper, we present a novel garment texture transfer framework from a single person image. Based on the garment model constructed in the person image, we render the texture to the model surface using parallax mapping. To determine the relative positions of the garments when exporting the images, we calculate the contour center moments of the garment mask and the center of mass coordinates of the 3D model and use their consistency to perform position calibration. Finally, we align the rendered garment image with the figure image to obtain the final transfer effect. Experiments demonstrated that our method is robust to different character pose with different garments and background. Qualitative experimental results show that our method accurately and realistically relocates the texture of the garment in the image of the person while preserving the original folds of the garment. Quantitative comparisons with other methods show that our method is optimal in several metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Cheng Z-Q, Chen Y, Martin RR, Lai Y-K, Wang A (2013) Supermatching: feature matching using supersymmetric geometric constraints. IEEE Trans Vis Comput Graph 19(11):1885–1894

    Article  Google Scholar 

  2. Wang X, Kuzmichev V E, Luo Y, Li Y (2010) A garment simulating method based on scanning technology In: 2010 international conference on computer and communication technologies in agriculture engineering, vol. 1, pp. 412–415

  3. Wang CCL, Wang Y, Yuen MMF (2003) Feature based 3D garment design through 2D sketches. Comput Des 35(7):659–672

    Google Scholar 

  4. Turquin E, Cani M-P, Hughes J F (2007) Sketching garments for virtual characters In: ACM SIGGRAPH 2007 courses, pp. 28--es

  5. Power J, Apeagyei PR, Jefferson AM (2011) Integrating 3D scanning data\textile parameters into virtual clothing. Hometrica Consulting, Switzerland

    Book  Google Scholar 

  6. Decaudin P, Julius D, Wither J, Boissieux L, Sheffer A, Cani M-P (2006) Virtual garments: a fully geometric approach for clothing design. Comput Gr Forum 25(3):625–634

    Article  Google Scholar 

  7. Fang J-J, Tien C-H (2013) Trimmed NURBS surface applications in computerized 3D fashion design for garment industry. Int J Cloth Sci Technol 25(1):24–42

    Article  Google Scholar 

  8. Liu S, Sheng J, He L, Chen F (2008) Simulation of 3D garment based on six pieces of Bezier curved surfaces In: 2008 international conference on computer science and software engineering, vol. 2, pp. 1082–1085

  9. Gatys L A, Ecker A S, Bethge M (2016) Image style transfer using convolutional neural networks In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2414–2423

  10. Guan X et al (2021) Automatic embroidery texture synthesis for garment design and online display. Vis Comput 37:2553–2565

    Article  Google Scholar 

  11. He W, Wang J, Wang L, Pan R, Gao W (2023) A semantic segmentation algorithm for fashion images based on modified mask RCNN. Multimed Tools Appl 82(18):1–18

    Article  Google Scholar 

  12. He W, Zhang N, Song B, Pan R (2023) Garment reconstruction from a single-view image based on pixel-aligned implicit function. Multimed Tools Appl 82:1–19

    Article  Google Scholar 

  13. Boisse P, Gasser A, Hivet G (2001) Analyses of fabric tensile behaviour: determination of the biaxial tension–strain surfaces and their use in forming simulations. Compos Part A Appl Sci Manuf 32(10):1395–1414

    Article  Google Scholar 

  14. Bridson R, Marino S, Fedkiw R (2005) Simulation of clothing with folds and wrinkles In: ACM SIGGRAPH 2005 courses, pp. 3--es

  15. Volino P, Magnenat-Thalmann N, Faure F (2009) A simple approach to nonlinear tensile stiffness for accurate cloth simulation. ACM Trans Graph 28(4):1–16

    Article  Google Scholar 

  16. Wang H, O’Brien JF, Ramamoorthi R (2011) Data-driven elastic models for cloth: modeling and measurement. ACM Trans Graph 30(4):1–12

    Google Scholar 

  17. Narain R, Samii A, O’brien JF (2012) Adaptive anisotropic remeshing for cloth simulation ACM Trans. Graph 31(6):1–10

    Google Scholar 

  18. Thomassey S, Bruniaux P (2013) A template of ease allowance for garments based on a 3D reverse methodology. Int J Ind Ergon 43(5):406–416

    Article  Google Scholar 

  19. Jeong Y, Hong K, Kim S-J (2006) 3D pattern construction and its application to tight-fitting garments for comfortable pressure sensation. Fibers Polym 7:195–202

    Article  Google Scholar 

  20. Meng Y, Mok PY, Jin X (2010) Interactive virtual try-on clothing design systems. Comput Des 42(4):310–321

    Google Scholar 

  21. Meng Y, Mok PY, Jin X (2012) Computer aided clothing pattern design with 3D editing and pattern alteration. Comput Des 44(8):721–734

    Google Scholar 

  22. Bouman K L, Xiao B, Battaglia P, Freeman W T (2013) Estimating the material properties of fabric from video In: Proceedings of the IEEE international conference on computer vision, pp. 1984–1991

  23. Chen X, Tao X, Zeng X, Koehl L, Boulenguez-Phippen J (2015) Control and optimization of human perception on virtual garment products by learning from experimental data. Knowledge-Based Syst 87:92–101

    Article  Google Scholar 

  24. Raj A, Sangkloy P, Chang H, Hays J, Ceylan D, Lu J (2018) SwapNet Image based garment transfer. Lect Notes Comput Sci including Subser Lect Notes Artif Intell Lect Notes Bioinf 11216:679–695. https://doi.org/10.1007/978-3-030-01258-8_41

    Article  Google Scholar 

  25. Bhatnagar B L, Tiwari G, Theobalt C, Pons-Moll G (2019) Multi-garment net: learning to dress 3d people from images In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 5420–5430

  26. Han X, Wu Z, Wu Z, Yu R, Davis L S (2018) Viton: an image-based virtual try-on network In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7543–7552

  27. Wang B, Zheng H, Liang X, Chen Y, Lin L, Yang M (2018) Toward characteristic-preserving image-based virtual try-on network In: Proceedings of the European conference on computer vision (ECCV), pp. 589–604

  28. Dong H et al. (2019) Towards multi-pose guided virtual try-on network In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 9026–9035

  29. Wu Z, Lin G, Tao Q, Cai J (2019) M2e-try on net: fashion from model to everyone In: Proceedings of the 27th ACM international conference on multimedia, pp. 293–301

  30. Zeng W, Zhao M, Gao Y, Zhang Z (2020) TileGAN: category-oriented attention-based high-quality tiled clothes generation from dressed person. Neural Comput Appl 32(23):17587–17600. https://doi.org/10.1007/S00521-020-04928-1

    Article  Google Scholar 

  31. Zheng H, Chen L, Xu C, Luo J (2019) Unsupervised pose flow learning for pose guided synthesis. arXiv Prepr. arXiv:1909.13819

  32. Zhao F, Liao S, Zhang K, Shao L (2020) Human parsing based texture transfer from single image to 3D human via cross-view consistency. Adv Neural Inf Process Syst 33:14326–14337

    Google Scholar 

  33. Cui A, McKee D, Lazebnik S (2021) Dressing in order: recurrent person image generation for pose transfer, virtual try-on and outfit editing In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 14638–14647

  34. Liu M, Zhao Y, Zhang Z, Zhang H, Yan S (2021) Arbitrary virtual ry-on network: characteristics preservation and trade-off between body and clothing arXiv e-prints

  35. Ulyanov D, Lebedev A, Vedaldi V, Lempitsky V (2016) Texture networks: feed-forward synthesis of textures and stylized images, arXiv Prepr. arXiv1603.03417

  36. Huang X, Belongie S (2019) Arbitrary style transfer in real-time with adaptive instance normalization, In: 5th Int Conf Learn Represent ICLR 2017—Work. Track Proc

  37. Sun K, Zhang J, Zhang P, Yuan K, Li G (2023) TsrNet: a two-stage unsupervised approach for clothing region-specific textures style transfer. J Vis Commun Image Represent 91:103778

    Article  Google Scholar 

  38. Zhu J Y, Park T, Isola P, Efros A A (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks? In: Proc IEEE Int Conf Comput Vis, vol. 2017, pp. 2242–2251 https://doi.org/10.1109/ICCV.2017.244

  39. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans image Process 13(4):600–612

    Article  Google Scholar 

  40. Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training gans. Adv Neural Inf Process Syst, vol. 29

  41. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Adv Neural Inf Process Syst, vol. 30

  42. Zhu Z, Huang T, Shi B, Yu M, Wang B, Bai X (2019) Progressive pose attention transfer for person image generation In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2347–2356

  43. Men Y, Mao Y, Y. Jiang Y, Ma W-Y, Lian Z (2020) Controllable person image synthesis with attribute-decomposed gan In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5084–5093

  44. Zhang J, Li K, Lai Y-K, Yang J (2021) Pise: person image synthesis and editing with decoupled gan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 7982–7990

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61976105 and No. 62202202) and Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. KYCX22_2342). This paper was also supported by the National Scholarship Fund of China (CSC202206790028). In addition, thanks to volunteer Dr. Cheng Yin for his participation.

Author information

Authors and Affiliations

  1. Key Laboratory of Eco-Textiles, Ministry of Education, Jiangnan University, No. 1800, Lihu Avenue, Wuxi, 214122, Jiangsu, China

    Wentao He, Bingpeng Song, Ning Zhang, Jun xiang & Ruru Pan

  2. Vision and Affective Science Integrated Laboratory, Kochi University of Technology, Kochi, Japan

    Wentao He

Authors
  1. Wentao He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bingpeng Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ning Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruru Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruru Pan.

Ethics declarations

Conflict of interest

The author declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, W., Song, B., Zhang, N. et al. Toward accurate and realistic garment texture transfer with attention to details. Neural Comput & Applic 36, 12991–13007 (2024). https://doi.org/10.1007/s00521-024-09653-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09653-7

Keywords

Search

Navigation