Give Me a Hand: A Scene-Fit Hand Posture Drawing Aid | SpringerLink
Skip to main content

Give Me a Hand: A Scene-Fit Hand Posture Drawing Aid

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13336))

Included in the following conference series:

Abstract

Hand posture drawing is one of the greatest challenges for most people who are not experts in figure painting. The lack of professional knowledge and guidance limits their freedom of creation. This paper aims to explore the methods of using artificial intelligence technologies to assist people in drawing hand postures. We investigated the causes of frustrations in drawing hands and inferred the design requirements through a user study. Then we proposed a scene-fit hand posture drawing aid called “Give Me A Hand”. The aid gives creators visual references that almost fully fit their ideation of hand postures and overall scenes in the form of 3D hand models. Unlike most existing studies on using artificial intelligence (AI) in the field of art creation, we pay more attention to the creative experience of humans during the intelligent collaboration between creators and AI. In the validation stage, we conducted a repeated-measures designed experiment to verify the effectiveness of the aid. The results of validation also provide us more inspirations on the relationship between human and AI in the artistic creation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
JPY 3498
Price includes VAT (Japan)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
JPY 11439
Price includes VAT (Japan)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
JPY 14299
Price includes VAT (Japan)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Intelligent cognitive assistants (2016)

    Google Scholar 

  2. Bidgoli, A., De Guevara, M.L., Hsiung, C., Oh, J., Kang, E.: Artistic style in robotic painting; a machine learning approach to learning brushstroke from human artists. In: 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 412–418. IEEE (2020)

    Google Scholar 

  3. Boukhayma, A., Bem, R., Torr, P.H.: 3D hand shape and pose from images in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10843–10852 (2019)

    Google Scholar 

  4. Cao, N., Yan, X., Shi, Y., Chen, C.: AI-sketcher: a deep generative model for producing high-quality sketches. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 2564–2571 (2019)

    Google Scholar 

  5. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2019)

    Article  Google Scholar 

  6. Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  7. Cohen, D.J.: Look little, look often: the influence of gaze frequency on drawing accuracy. Percept. Psychophysics 67(6), 997–1009 (2005)

    Article  Google Scholar 

  8. Cohen, D.J., Bennett, S.: Why can’t most people draw what they see? J. Exp. Psychol. Hum. Percept. Perform. 23(3), 609 (1997)

    Article  Google Scholar 

  9. Cohen, H.: How to draw three people in a botanical garden. In: AAAI, vol. 89, pp. 846–855 (1988)

    Google Scholar 

  10. Davis, N., Hsiao, C.-P., Popova, Y., Magerko, B.: An enactive model of creativity for computational collaboration and co-creation. In: Zagalo, N., Branco, P. (eds.) Creativity in the Digital Age. SSCC, pp. 109–133. Springer, London (2015). https://doi.org/10.1007/978-1-4471-6681-8_7

    Chapter  Google Scholar 

  11. Davis, N., Hsiao, C.P., Yashraj Singh, K., Li, L., Magerko, B.: Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent. In: Proceedings of the 21st International Conference on Intelligent User Interfaces, pp. 196–207 (2016)

    Google Scholar 

  12. Davis, N.M., Hsiao, C.P., Singh, K.Y., Magerko, B.: Co-creative drawing agent with object recognition. In: Twelfth Artificial Intelligence and Interactive Digital Entertainment Conference (2016)

    Google Scholar 

  13. Fox, M.T., Brathwaite, A.C., Sidani, S.: Evaluating the effectiveness of interventions: exploration of two statistical methods. Can. J. Nurs. Res. Arch. 20–30 (2004)

    Google Scholar 

  14. Frans, K.: Outline colorization through tandem adversarial networks. arXiv preprint arXiv:1704.08834 (2017)

  15. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414–2423 (2016)

    Google Scholar 

  16. Geer, T.: What we illustrate when we draw: normative visual processing in beginner drawings, and the capacity to observe detail. Thinking Drawing Pract. Knowl. 45 (2011)

    Google Scholar 

  17. Giorgi, A.: The theory, practice, and evaluation of the phenomenological method as a qualitative research procedure. J. Phenomenol. Psychol. 28(2), 235–260 (1997)

    Article  Google Scholar 

  18. Goodfellow, I.: Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016)

  19. Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017)

  20. Huang, S., Chen, Y., Yuan, T., Qi, S., Zhu, Y., Zhu, S.C.: Perspectivenet: 3D object detection from a single RGB image via perspective points. arXiv preprint arXiv:1912.07744 (2019)

  21. Huang, Z., Heng, W., Zhou, S.: Learning to paint with model-based deep reinforcement learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8709–8718 (2019)

    Google Scholar 

  22. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  23. Knaack, P.: Phenomenological research. West. J. Nurs. Res. 6(1), 107–114 (1984)

    Article  Google Scholar 

  24. Kozbelt, A.: Artists as experts in visual cognition. Vis. Cogn. 8(6), 705–723 (2001)

    Article  Google Scholar 

  25. Lin, Y., Guo, J., Chen, Y., Yao, C., Ying, F.: It is your turn: collaborative ideation with a co-creative robot through sketch. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2020)

    Google Scholar 

  26. Manovich, L.: Defining AI arts: Three proposals. AI and dialog of cultures exhibition catalog. Hermitage Museum, Saint-Petersburg (2019)

    Google Scholar 

  27. Oh, C., Song, J., Choi, J., Kim, S., Lee, S., Suh, B.: I lead, you help but only with enough details: understanding user experience of co-creation with artificial intelligence. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2018)

    Google Scholar 

  28. Ostrofsky, J., Kozbelt, A.: A multi-stage attention hypothesis of drawing ability. In: Thinking through Drawing: Practice into Knowledge. Proceedings of an Interdisciplinary Symposium on Drawing, Cognition and Education, pp. 61–66. Columbia University, New York, USA Teachers College (2011)

    Google Scholar 

  29. Park, T., Liu, M.Y., Wang, T.C., Zhu, J.Y.: Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2337–2346 (2019)

    Google Scholar 

  30. Romero, J., Tzionas, D., Black, M.J.: Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610 (2022)

  31. Shaw, D., Barnes, N., et al.: Perspective rectangle detection. In: Proceedings of the Workshop of the Application of Computer Vision, in Conjunction with ECCV 2006, pp. 119–127 (2006)

    Google Scholar 

  32. Shi, Y., Cao, N., Ma, X., Chen, S., Liu, P.: EmoG: supporting the sketching of emotional expressions for storyboarding. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12 (2020)

    Google Scholar 

  33. Simon, T., Joo, H., Matthews, I., Sheikh, Y.: Hand keypoint detection in single images using multiview bootstrapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1145–1153 (2017)

    Google Scholar 

  34. Solso, R.L.: Cognition and the Visual Arts. MIT Press, Cambridge (1996)

    Google Scholar 

  35. Tan, J., Echevarria, J., Gingold, Y.: Palette-based image decomposition, harmonization, and color transfer. arXiv preprint arXiv:1804.01225 (2018)

  36. Tchalenko, J.: Segmentation and accuracy in copying and drawing: experts and beginners. Vis. Res. 49(8), 791–800 (2009)

    Article  Google Scholar 

  37. Wei, S.E., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4724–4732 (2016)

    Google Scholar 

  38. Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., He, X.: AttnGAN: fine-grained text to image generation with attentional generative adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1316–1324 (2018)

    Google Scholar 

  39. You, S., You, N., Pan, M.: PI-REC: Progressive image reconstruction network with edge and color domain. arXiv preprint arXiv:1903.10146 (2019)

  40. Zeng, Z., Sun, X., Liao, X.: Artificial intelligence augments design creativity: a typeface family design experiment. In: Marcus, A., Wang, W. (eds.) HCII 2019. LNCS, vol. 11584, pp. 400–411. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23541-3_29

    Chapter  Google Scholar 

  41. Zhang, Q., Xiao, C., Sun, H., Tang, F.: Palette-based image recoloring using color decomposition optimization. IEEE Trans. Image Process. 26(4), 1952–1964 (2017)

    Article  MathSciNet  Google Scholar 

  42. Zhou, Z., Farhat, F., Wang, J.Z.: Detecting dominant vanishing points in natural scenes with application to composition-sensitive image retrieval. IEEE Trans. Multimedia 19(12), 2651–2665 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Juexiao Qin .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sun, X., Qin, J., Xu, W., Peng, X. (2022). Give Me a Hand: A Scene-Fit Hand Posture Drawing Aid. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05643-7_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05642-0

  • Online ISBN: 978-3-031-05643-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics