Abstract
High-fidelity kinship face synthesis is a challenging task due to the limited amount of kinship data available for training and low-quality images. In addition, it is also hard to trace the genetic traits between parents and children from those low-quality training images. To address these issues, we leverage the pre-trained state-of-the-art face synthesis model, StyleGAN2, for kinship face synthesis. To handle large age, gender and other attribute variations between the parents and their children, we conduct a thorough study of its rich latent spaces and different encoder architectures for an optimized encoder design to repurpose StyleGAN2 for kinship face synthesis. The obtained latent representation from our developed encoder pipeline with stage-wise training strikes a better balance of editability and synthesis fidelity for identity preserving and attribute manipulations than other compared approaches. With extensive subjective, quantitative, and qualitative evaluations, the proposed approach consistently achieves better performance in terms of facial attribute heredity and image generation fidelity than other compared state-of-the-art methods. This demonstrates the effectiveness of the proposed approach which can yield promising and satisfactory kinship face synthesis using only a single and straightforward encoder architecture.
This work was supported by the National Science and Technology Council under Grant 108-2628-E-001-003-MY3, 111-2628-E-001 -002 -MY3, 111-3114-E-194-001 -, 110-2221-E-001 -009 -MY2, 110-2634-F-002-051-, 111-2221-E-001-002-, and the Academia Sinica under Thematic Research Grant AS-TP-110-M07-2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ertugrul, I.Ö., Dibeklioglu, H.: What will your future child look like? modeling and synthesis of hereditary patterns of facial dynamics. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2017)
Ozkan, S., Ozkan, A.: KinshipGAN: synthesizing of kinship faces from family photos by regularizing a deep face network. In: IEEE International Conference on Image Processing (ICIP) (2018)
Gao, P., Robinson, J., Zhu, J., Xia, C., Shao, M., Xia, S.: DNA-Net: age and gender aware kin face synthesizer. In: IEEE International Conference on Multimedia and Expo (ICME) (2021)
Lin, C.H., Chen, H.C., Cheng, L.C., Hsu, S.C., Chen, J.C., Wang, C.Y.: StyleDNA: a high-fidelity age and gender aware kinship face synthesizer. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2021)
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of styleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Zhang, Y., Li, L., Liu, Z., Wu, B., Fan, Y., Li, Z.: Controllable descendant face synthesis. arXiv preprint arXiv:2002.11376 (2020)
Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., Cohen-Or, D.: Designing an encoder for styleGAN image manipulation. ACM Trans. Graph. (TOG) 40, 1–14 (2021)
Robinson, J.P., Shao, M., Wu, Y., Liu, H., Gillis, T., Fu, Y.: Visual kinship recognition of families in the wild. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2624–2637 (2018)
Qin, X., Tan, X., Chen, S.: Tri-subjects kinship verification: understanding the core of a family. In: 2015 14th IAPR International Conference on Machine Vision Applications (MVA), pp. 580–583 (2015)
Chong, M.J., Chu, W.S., Kumar, A., Forsyth, D.: Retrieve in style: unsupervised facial feature transfer and retrieval. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021)
Liu, S., et al.: Face aging with contextual generative adversarial nets. In: Proceedings of the 25th ACM International Conference on Multimedia (2017)
Tang, H., Bai, S., Sebe, N.: Dual attention GANs for semantic image synthesis. In: Proceedings of the 28th ACM International Conference on Multimedia (2020)
Brock, A., Donahue, J., Simonyan, K.: Large scale GAN training for high fidelity natural image synthesis. In: International Conference on Learning Representations (ICLR) (2019)
Zhao, J., et al.: Dual-agent GANs for photorealistic and identity preserving profile face synthesis. In: Advances in Neural Information Processing Systems 30 (2017)
Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019)
Abdal, R., Qin, Y., Wonka, P.: Image2styleGAN: how to embed images into the stylegan latent space? In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
Zhang, L., Bai, X., Gao, Y.: SalS-GAN: spatially-adaptive latent space in styleGAN for real image embedding. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 5176–5184 (2021)
Sainburg, T., Thielk, M., Theilman, B., Migliori, B., Gentner, T.: Generative adversarial interpolative autoencoding: adversarial training on latent space interpolations encourage convex latent distributions. arXiv preprint arXiv:1807.06650 (2018)
Mukherjee, S., Asnani, H., Lin, E., Kannan, S.: ClusterGAN: latent space clustering in generative adversarial networks. Proceed. AAAI Conf. Artif. Intell. 33, 4610–4617 (2019)
Bojanowski, P., Joulin, A., Lopez-Paz, D., Szlam, A.: Optimizing the latent space of generative networks. arXiv preprint arXiv:1707.05776 (2017)
Robinson, J.P., Shao, M., Fu, Y.: Survey on the analysis and modeling of visual kinship: a decade in the making. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)
Ertuğrul, I.Ö., Jeni, L.A., Dibeklioğlu, H.: Modeling and synthesis of kinship patterns of facial expressions. Image Vis. Comput. 79, 133–143 (2018)
Sinha, R., Vatsa, M., Singh, R.: FamilyGAN: generating kin face images using generative adversarial networks. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12537, pp. 297–311. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-67070-2_18
Ghatas, F.S., Hemayed, E.E.: GANKIN: generating kin faces using disentangled GAN. SN Appl. Sci. 2, 166 (2020)
Zaman, I., Crandall, D.: Genetic-GAN: synthesizing images between two domains by genetic crossover. In: European Conference on Computer Vision Workshops (ECCVW) (2020)
Cui, X., Zhou, W., Hu, Y., Wang, W., Li, H.: Heredity-aware child face image generation with latent space disentanglement. arXiv preprint arXiv:2108.11080 (2021)
Alaluf, Y., Patashnik, O., Cohen-Or, D.: Only a matter of style: age transformation using a style-based regression model. ACM Trans. Graph. (TOG) 40, 1–12 (2021)
Kafri, O., Patashnik, O., Alaluf, Y., Cohen-Or, D.: StyleFusion: a generative model for disentangling spatial segments. arXiv preprint arXiv:2107.07437 (2021)
Or-El, R., Sengupta, S., Fried, O., Shechtman, E., Kemelmacher-Shlizerman, I.: Lifespan age transformation synthesis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12351, pp. 739–755. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58539-6_44
Wang, X., Li, Y., Zhang, H., Shan, Y.: Towards real-world blind face restoration with generative facial prior. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Chen, S., Liu, Y., Gao, X., Han, Z.: MobileFaceNets: Efficient CNNs for accurate real-time face verification on mobile devices. CoRR abs/1804.07573 (2018)
Richardson, E., et al.: Encoding in style: a styleGAN encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cheng, LC. et al. (2023). KinStyle: A Strong Baseline Photorealistic Kinship Face Synthesis with an Optimized StyleGAN Encoder. In: Wang, L., Gall, J., Chin, TJ., Sato, I., Chellappa, R. (eds) Computer Vision – ACCV 2022. ACCV 2022. Lecture Notes in Computer Science, vol 13844. Springer, Cham. https://doi.org/10.1007/978-3-031-26316-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-26316-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26315-6
Online ISBN: 978-3-031-26316-3
eBook Packages: Computer ScienceComputer Science (R0)