Abstract
This paper leverages the effectiveness of variational autoencoders in removing hair in dermoscopy images and presents an enhanced approach that combines variational autoencoders with generative adversarial learning and disentangled representation referred to as GAD-VAE to produce high-quality dermoscopy images that are free from hair. The proposed GAD-VAE employs disentangled representation to design a two-branch network for the generator. This network learns latent representations that distinguish hair-related features from non-hair-related features. Each branch is dedicated to a specific task. The first branch concentrates on generating hair-free images using variational autoencoders, while the second branch models hair characteristics. Notably, this model exhibits contextual awareness, explicitly capturing hair properties, thereby benefiting the second branch. The model also incorporates two discriminators that assess the realism of the generated hair-free images and hair images. Experimental results demonstrate the potential of the proposed model in effectively removing some types of hair compared to variational autoencoders and some of the state-of-the-art methods. Additionally, it generates hair that can be utilized to develop hair segmentation methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availibility Statement
The dataset used in this work is available on Kaggle: https://www.kaggle.com/datasets/bardoudalal/gad-vae-hairremovaldataset
References
Attia M, Hossny M, Zhou H, Nahavandi S, Asadi H, Yazdabadi A (2019) Digital hair segmentation using hybrid convolutional and recurrent neural networks architecture. Comput Methods Progr Biomed 177:17–30
Attia M, Hossny M, Zhou H, Nahavandi S, Asadi H, Yazdabadi A (2020) Realistic hair simulator for skin lesion images: a novel benchemarking tool. Artif Intell Med 108:101933
Bardou D, Bouaziz H, Lv L, Zhang T (2022) Hair removal in dermoscopy images using variational autoencoders. Skin Res Technol 28(3):445–454
Bataille V (2009) Early detection of melanoma improves survival. Practitioner 253(1722):29–33
Bengio Y, Yao L, Alain G, Vincent P (2013) Generalized denoising auto-encoders as generative models. Adv Neural Inf Process Syst 26
Ciudad-Blanco C, Avilés-Izquierdo J, Lázaro-Ochaita P, Suárez-Fernández R (2014) Dermoscopic findings for the early detection of melanoma: an analysis of 200 cases. Actas Dermo-Sifiliográficas (English Edition) 105(7):683–693
Ganokratanaa T, Aramvith S, Sebe N (2020) Unsupervised anomaly detection and localization based on deep spatiotemporal translation network. IEEE Access 8:50312–50329
Gewirtzman A, Braun R (2003) Computerized digital dermoscopy. J Cosmet Dermatol 2(1):14–20
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27
Gui J, Sun Z, Wen Y, Tao D, Ye J (2021) A review on generative adversarial networks: Algorithms, theory, and applications. IEEE Trans Knowl Data Eng 35(4):3313–3332
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134
Jin X, Chen Z, Li W (2020) Ai-gan: asynchronous interactive generative adversarial network for single image rain removal. Pattern Recognit 100:107143
Karimkhani C, Green AC, Nijsten T, Weinstock MA, Dellavalle RP, Naghavi M, Fitzmaurice C (2017) The global burden of melanoma: results from the global burden of disease study 2015. Br J Dermatol 177(1):134–140
Kim D, Hong B-W (2021) Unsupervised feature elimination via generative adversarial networks: application to hair removal in melanoma classification. IEEE Access 9:42610–42620
Kim T, Cha M, Kim H, Lee JK, Kim J (2017) Learning to discover cross-domain relations with generative adversarial networks. In: International conference on machine learning. PMLR, pp 1857–1865
Kingma DP, Welling M et al (2019) An introduction to variational autoencoders. Found Trends Mach Learn 12(4):307–392
Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: International conference on machine learning. PMLR, pp 1558–1566
Lee Y, You W (2023) Ebat: enhanced bidirectional and autoregressive transformers for removing hairs in hairy dermoscopic images. IEEE Access 11:14225–14235
Li W, Raj ANJ, Tjahjadi T, Zhuang Z (2021) Digital hair removal by deep learning for skin lesion segmentation. Pattern Recognit 117:107994
Malvehy J, Puig S (2002) Follow-up of melanocytic skin lesions with digital total-body photography and digital dermoscopy: a two-step method. Clin Dermatol 20(3):297–304
Masci J, Meier U, Cireşan D, Schmidhuber J (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: Artificial neural networks and machine learning—ICANN 2011: 21st international conference on artificial neural networks, Espoo, Finland, June 14–17, 2011, Proceedings, Part I 21. Springer, pp 52–59
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434
Rumelhart DE, Hinton GE, Williams RJ et al (1985) Learning internal representations by error propagation. Institute for Cognitive Science, University of California, San Diego La
Shao M, Zhang Y, Fan Y, Zuo W, Meng D (2021) Iit-gat: Instance-level image transformation via unsupervised generative attention networks with disentangled representations. Knowl Based Syst 225:107122
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Stewart SL, Hayes NS, Moore AR, Robert Bailey I, Brown PM, Wanliss E (2018) Combating cancer through public health practice in the united states: an in-depth look at the national comprehensive cancer control program. In: Public health-emerging and re-emerging issues. IntechOpen
Talavera-Martinez L, Bibiloni P, Gonzalez-Hidalgo M (2020) Hair segmentation and removal in dermoscopic images using deep learning. IEEE Access 9:2694–2704
Tromme I, Devleesschauwer B, Beutels P, Richez P, Praet N, Sacré L, Marot L, Van Eeckhout P, Theate I, Baurain J-F et al (2014) Selective use of sequential digital dermoscopy imaging allows a cost reduction in the melanoma detection process: a belgian study of patients with a single or a small number of atypical nevi. PLoS One 9(10):109339
Tschandl P, Rosendahl C, Kittler H (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5(1):1–9
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Wang J, Gai S, Huang X, Zhang H (2021) From coarse to fine: a two stage conditional generative adversarial network for single image rain removal. Dig Signal Process 111:102985
Wan C, Probst T, Van Gool L, Yao A (2017) Crossing nets: combining gans and vaes with a shared latent space for hand pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 680–689
Xian Y, Sharma S, Schiele B, Akata Z (2019) f-vaegan-d2: A feature generating framework for any-shot learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10275–10284
Yi Z, Zhang H, Tan P, Gong M (2017) Dualgan: unsupervised dual learning for image-to-image translation. In: Proceedings of the IEEE international conference on computer vision, pp 2849–2857
Yue H, Cheng Y, Liu F, Yang J (2021) Unsupervised moiré pattern removal for recaptured screen images. Neurocomputing 456:352–363
Zhao H, Gallo O, Frosio I, Kautz J (2015) Loss functions for neural networks for image processing. arXiv:1511.08861
Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232
Zhu Y, Deng C, Cao H, Wang H (2020) Object and background disentanglement for unsupervised cross-domain person re-identification. Neurocomputing 403:88–97
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Ethical approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Yes.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bardou, D., Lv, L., Medjadba, Y. et al. GAD-VAE: generative adversarial disentanglement with variational autoencoders for hair removal in dermoscopy images. Netw Model Anal Health Inform Bioinforma 13, 32 (2024). https://doi.org/10.1007/s13721-024-00461-6
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s13721-024-00461-6