Abstract
The aim of gaze following is to estimate the gaze direction, which is useful for the understanding of human behaviour in various applications. However, it is still an open problem that has not been fully studied. In this paper, we present a novel framework for gaze following problem, where both the front/side face case and the back face case are taken into account. For the front/side face case, head pose estimation is applied to estimate the gaze, and then object detection is used to further refine the gaze direction by selecting the object that intersects with the gaze in a certain range. For the back face case, a deep neural network with the human pose information is proposed for gaze estimation. Experiments are carried out to demonstrate the superiority of the proposed method, as compared with the state-of-the-art method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Fathi, A., Hodgins, J.K., Rehg, J.M.: Social interactions: a first-person perspective. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1226–1233 (2012)
Gkioxari, G., Girshick, R.B., Dollár, P., He, K.: Detecting and recognizing human-object interactions. CoRR abs/1704.07333 (2017). http://arxiv.org/abs/1704.07333
Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010)
Krafka, K., et al.: Eye tracking for everyone. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2176–2184 (2016)
Marin-Jimenez, M.J., Zisserman, A., Eichner, M., Ferrari, V.: Detecting people looking at each other in videos. Int. J. Comput. Vis. 106(3), 282–296 (2014)
Park, H.S., Jain, E., Sheikh, Y.: Predicting primary gaze behavior using social saliency fields. In: Proceedings of IEEE International Conference on Computer Vision, pp. 3503–3510 (2013)
Park, H.S., Shi, J.: Social saliency prediction. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4777–4785 (2015)
Parks, D., Borji, A., Itti, L.: Augmented saliency model using automatic 3D head pose detection and learned gaze following in natural scenes. Vis. Res. 116, 113–126 (2015)
Recasens, A., Vondrick, C., Khosla, A., Torralba, A.: Following gaze in video. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1444–1452 (2017)
Recasens, A., Khosla, A., Vondrick, C., Torralba, A.: Where are they looking? In: Proceedings of Advances in Neural Information Processing Systems, pp. 199–207 (2015)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)
Ruiz, N., Chong, E., Rehg, J.M.: Fine-grained head pose estimation without keypoints. CoRR abs/1710.00925 (2017). http://arxiv.org/abs/1710.00925
Santini, T., Fuhl, W., Kasneci, E.: CalibMe: fast and unsupervised eye tracker calibration for gaze-based pervasive human-computer interaction. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 2594–2605. ACM (2017)
Shepherd, S.V.: Following gaze: gaze-following behavior as a window into social cognition. Front. Integr. Neurosci. 4, 1–13 (2010)
Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., Li, S.Z.: \(\text{S}^3\)FD: single shot scale-invariant face detector. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 192–201 (2017)
Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4511–4520 (2015)
Acknowledgement
This work is partly supported by the Fundamental Research Funds for the Central Universities (No. 3072019CFJ0602).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Guan, J., Yin, L., Sun, J., Qi, S., Wang, X., Liao, Q. (2020). Enhanced Gaze Following via Object Detection and Human Pose Estimation. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11962. Springer, Cham. https://doi.org/10.1007/978-3-030-37734-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-37734-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37733-5
Online ISBN: 978-3-030-37734-2
eBook Packages: Computer ScienceComputer Science (R0)