Abstract
Hand posture drawing is one of the greatest challenges for most people who are not experts in figure painting. The lack of professional knowledge and guidance limits their freedom of creation. This paper aims to explore the methods of using artificial intelligence technologies to assist people in drawing hand postures. We investigated the causes of frustrations in drawing hands and inferred the design requirements through a user study. Then we proposed a scene-fit hand posture drawing aid called “Give Me A Hand”. The aid gives creators visual references that almost fully fit their ideation of hand postures and overall scenes in the form of 3D hand models. Unlike most existing studies on using artificial intelligence (AI) in the field of art creation, we pay more attention to the creative experience of humans during the intelligent collaboration between creators and AI. In the validation stage, we conducted a repeated-measures designed experiment to verify the effectiveness of the aid. The results of validation also provide us more inspirations on the relationship between human and AI in the artistic creation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Intelligent cognitive assistants (2016)
Bidgoli, A., De Guevara, M.L., Hsiung, C., Oh, J., Kang, E.: Artistic style in robotic painting; a machine learning approach to learning brushstroke from human artists. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 412–418. IEEE (2020)
Boukhayma, A., Bem, R., Torr, P.H.: 3D hand shape and pose from images in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10843–10852 (2019)
Cao, N., Yan, X., Shi, Y., Chen, C.: AI-sketcher: a deep generative model for producing high-quality sketches. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 2564–2571 (2019)
Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2019)
Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)
Cohen, D.J.: Look little, look often: the influence of gaze frequency on drawing accuracy. Percept. Psychophysics 67(6), 997–1009 (2005)
Cohen, D.J., Bennett, S.: Why can’t most people draw what they see? J. Exp. Psychol. Hum. Percept. Perform. 23(3), 609 (1997)
Cohen, H.: How to draw three people in a botanical garden. In: AAAI, vol. 89, pp. 846–855 (1988)
Davis, N., Hsiao, C.-P., Popova, Y., Magerko, B.: An enactive model of creativity for computational collaboration and co-creation. In: Zagalo, N., Branco, P. (eds.) Creativity in the Digital Age. SSCC, pp. 109–133. Springer, London (2015). https://doi.org/10.1007/978-1-4471-6681-8_7
Davis, N., Hsiao, C.P., Yashraj Singh, K., Li, L., Magerko, B.: Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent. In: Proceedings of the 21st International Conference on Intelligent User Interfaces, pp. 196–207 (2016)
Davis, N.M., Hsiao, C.P., Singh, K.Y., Magerko, B.: Co-creative drawing agent with object recognition. In: Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference (2016)
Fox, M.T., Brathwaite, A.C., Sidani, S.: Evaluating the effectiveness of interventions: exploration of two statistical methods. Can. J. Nurs. Res. Arch. 20–30 (2004)
Frans, K.: Outline colorization through tandem adversarial networks. arXiv preprint arXiv:1704.08834 (2017)
Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)
Geer, T.: What we illustrate when we draw: normative visual processing in beginner drawings, and the capacity to observe detail. Thinking Drawing Pract. Knowl. 45 (2011)
Giorgi, A.: The theory, practice, and evaluation of the phenomenological method as a qualitative research procedure. J. Phenomenol. Psychol. 28(2), 235–260 (1997)
Goodfellow, I.: Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016)
Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017)
Huang, S., Chen, Y., Yuan, T., Qi, S., Zhu, Y., Zhu, S.C.: Perspectivenet: 3D object detection from a single RGB image via perspective points. arXiv preprint arXiv:1912.07744 (2019)
Huang, Z., Heng, W., Zhou, S.: Learning to paint with model-based deep reinforcement learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8709–8718 (2019)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Knaack, P.: Phenomenological research. West. J. Nurs. Res. 6(1), 107–114 (1984)
Kozbelt, A.: Artists as experts in visual cognition. Vis. Cogn. 8(6), 705–723 (2001)
Lin, Y., Guo, J., Chen, Y., Yao, C., Ying, F.: It is your turn: collaborative ideation with a co-creative robot through sketch. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2020)
Manovich, L.: Defining AI arts: Three proposals. AI and dialog of cultures exhibition catalog. Hermitage Museum, Saint-Petersburg (2019)
Oh, C., Song, J., Choi, J., Kim, S., Lee, S., Suh, B.: I lead, you help but only with enough details: understanding user experience of co-creation with artificial intelligence. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)
Ostrofsky, J., Kozbelt, A.: A multi-stage attention hypothesis of drawing ability. In: Thinking through Drawing: Practice into Knowledge. Proceedings of an Interdisciplinary Symposium on Drawing, Cognition and Education, pp. 61–66. Columbia University, New York, USA Teachers College (2011)
Park, T., Liu, M.Y., Wang, T.C., Zhu, J.Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2337–2346 (2019)
Romero, J., Tzionas, D., Black, M.J.: Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610 (2022)
Shaw, D., Barnes, N., et al.: Perspective rectangle detection. In: Proceedings of the Workshop of the Application of Computer Vision, in Conjunction with ECCV 2006, pp. 119–127 (2006)
Shi, Y., Cao, N., Ma, X., Chen, S., Liu, P.: EmoG: supporting the sketching of emotional expressions for storyboarding. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2020)
Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1145–1153 (2017)
Solso, R.L.: Cognition and the Visual Arts. MIT Press, Cambridge (1996)
Tan, J., Echevarria, J., Gingold, Y.: Palette-based image decomposition, harmonization, and color transfer. arXiv preprint arXiv:1804.01225 (2018)
Tchalenko, J.: Segmentation and accuracy in copying and drawing: experts and beginners. Vis. Res. 49(8), 791–800 (2009)
Wei, S.E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4724–4732 (2016)
Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., He, X.: AttnGAN: fine-grained text to image generation with attentional generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1316–1324 (2018)
You, S., You, N., Pan, M.: PI-REC: Progressive image reconstruction network with edge and color domain. arXiv preprint arXiv:1903.10146 (2019)
Zeng, Z., Sun, X., Liao, X.: Artificial intelligence augments design creativity: a typeface family design experiment. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11584, pp. 400–411. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23541-3_29
Zhang, Q., Xiao, C., Sun, H., Tang, F.: Palette-based image recoloring using color decomposition optimization. IEEE Trans. Image Process. 26(4), 1952–1964 (2017)
Zhou, Z., Farhat, F., Wang, J.Z.: Detecting dominant vanishing points in natural scenes with application to composition-sensitive image retrieval. IEEE Trans. Multimedia 19(12), 2651–2665 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, X., Qin, J., Xu, W., Peng, X. (2022). Give Me a Hand: A Scene-Fit Hand Posture Drawing Aid. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-05643-7_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05642-0
Online ISBN: 978-3-031-05643-7
eBook Packages: Computer ScienceComputer Science (R0)