Abstract
Although there are many studies on NPR (Non-Photorealistic Rendering) image synthesis using GANs (Generative Adversarial Networks), it is still difficult to create high-quality comic portraits of a real person. Moreover, there are few studies focused on the painting styles of comic authors, which is the style that makes a comic visually unique. It is expected that for comic readers, the synthesized comic portraits can be more attractive and meaningful if the portraits are presented in the user-preferred comic styles. Therefore, this paper proposes a styled comic portraits synthesis system based on CycleGAN and PIX2PIX. By integrating Deep Learning and NPR techniques, the proposed system aims to transform user’s real pictures into comic portraits with features preserved and defined painting style presented. CNN (Convolutional Neural Networks) is trained to classify the painting styles of comic authors. After that, two sets of GANs are trained with classified and augmented dataset, which is generated by mapping comic characters’ 2D texture onto perturbed and deformed 3D facial models. The experiment results show that the proposed system can successfully create clear and vivid comic portraits, which has a great potential to serve as a useful tool for social network and comic industry.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Conference on Computer Vision and Pattern Recognition (2017)
Zhu, J., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: International Conference on Computer Vision (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Isola, P., Zhu, J., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Conference on Computer Vision and Pattern Recognition (2016)
Zhang, H., et al.: StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. In: International Conference on Computer Vision (2017)
Zhang, H., et al.: StackGAN++: realistic image synthesis with stacked generative adversarial networks. Inst. Electr. Electron. Eng. Trans. Pattern Anal. Mach. Intell. 41, 1947-1962 (2018)
Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. In: International Conference on Learning Representations (2016)
Chen, L., Chen, Y., Lin, S., Liu, T., Hsieh, W.: Synthesizing non-photorealistic rendering effects of volumetric strokes. J. Inf. Sci. Eng. 28, 521–535 (2012)
Sun, C., Chen, L., Takama, Y.: Synthesizing NPR styled street view animation based on deep learning. In: International Conference on Technologies and Applications of Artificial Intelligence (2019)
Chen, S., Su, W., Gao, L., Xia, S., Fu, H.: Deep generation of face images from sketches. In: Special Interest Group on Computer Graphics and Interactive Techniques (2020)
Karras, T., Laine, S., Ail, T.: A style-based generator architecture for generative adversarial networks. In: Conference on Computer Vision and Pattern Recognition (2019)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Conference on Computer Vision and Pattern Recognition (2015)
Demir, U., Unal, G.: Patch-based image inpainting with generative adversarial networks. arXiv preprint, arXiv:1803.07422 (2018)
Goodfellow, I.J., et al.: Image augmentation using radial transform for training deep neural network. In: Institute of Electrical and Electronics Engineers International Conference on Acoustics, Speech, and Signal Processing (2014)
Perez, L., Wang, J.: The Effectiveness of data augmentation in image classification using deep learning. arXiv preprint, arXiv:1712.04621 (2017)
Bloice, M.D., Stocker, C., Holzinger, A.: Augmentor: an image augmentation library for machine learning. J. Open Sour. Softw. (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, YC., Chen, LH., Shibata, H., Takama, Y. (2022). Styled Comic Portrait Synthesis Based on GAN. In: Takama, Y., et al. Advances in Artificial Intelligence. JSAI 2021. Advances in Intelligent Systems and Computing, vol 1423. Springer, Cham. https://doi.org/10.1007/978-3-030-96451-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-96451-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96450-4
Online ISBN: 978-3-030-96451-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)