Abstract
Facial expression recognition is becoming a hot topic due to its wide applications in computer vision research fields. Traditional methods adopt hand-crafted features combined with classifiers to achieve the recognition goal. However, the accuracy of these methods often relies heavily on the extracted features and the classifier’s parameters, and thus cannot get good result with unseen data. Recently, deep learning, which simulates the mechanism of human brain to interpret data, has shown remarkable results in visual object recognition. In this paper, we present a novel convolutional neural network which consists of local binary patterns and improved Inception-ResNet layers for automatic facial expression recognition. We apply the proposed method to three expression datasets, i.e., the Extended Cohn-kanade Dataset (CK+), the Japanese Female Expression Database (JAFFE), and the FER2013 Dataset. The experimental results demonstrate the feasibility and effectiveness of our proposed network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)
Shimodaira, H.: Improving predictive inference under covariate shift by weighting the log likelihood function. J. Stat. Plan. Inference 90(2), 227–244 (2000)
Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision. IEEE, pp. 1–10 (2016)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 1–9 (2015)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, pp. 770–778 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Szegedy, C., Ioffe, S., Vanhoucke, V., et al.: Inception-v4, Inception-ResNet and the impact of residual connections on learning. arXiv:1602.07261 (2016)
Hasani, B., Mahoor, M.H.: Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields. In: IEEE International Conference on Automatic Face & Gesture Recognition. IEEE, pp. 790–795 (2017)
Hasani, B., Mahoor, M.H.: Facial expression recognition using enhanced deep 3D convolutional neural networks. In: Computer Vision and Pattern Recognition Workshops, pp. 2278–2288 (2017)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems. Curran Associates Inc., pp. 1097–1105 (2012)
Ojala, T., Harwood, I.: A comparative study of texture measures with classification based on feature distributions. Pattern Recognit. 29(1), 51–59 (1996)
Ali, A., Hussain, S., Haroon, F., et al.: Face recognition with local binary patterns. Bahria Univ. J. Inf. Commun. Technol. 5(1), 46–50 (2014)
Ojala, T., Pietikinen, M., Menp, T.: Multiresolution grayscale and rotation invariant texture classification with local binary patterns. IEEE PAMI 24(7), 971–987 (2002)
Lucey, P., Cohn, J.F., Kanade, Y., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101 (2010)
Tian, Y., Brown, L., Hampapur, A., Pankanti, S., Senior, A., Bolle, R.: Real world real-time automatic recognition of facial expression. In: Proceedings of IEEE Workshop on (2003)
Lyons, M., Akamatsu, S., Kamachi, M., et al.: Coding facial expressions with gabor wavelets. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200–205 (1998)
Song, Z., Ni, B., Guo, D., Sim, T., Yan, S.: Learning universal multi-view age estimator using video context. In: Proceedings of IEEE International Conference Computer Vision, pp. 241–248 (2011)
Lucey, P., Cohn, J., Lucey, S., Matthews, I., Sridharan, S., Prkachin, K.: Automatically detecting pain using facial actions. In: IEEE International Conference on Affective Computing and Intelligent Interaction, pp. 1–8 (2009)
Lecun, Y., Bottou, L., Bengio, Y.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(2), 1097–1105 (2012)
Happy, S.L., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)
Rodriguez, P., Cucurull, G., Gonzàlez, J.: Deep pain: exploiting long short-term memory networks for facial expression classification. IEEE Trans. Cybern. 1–11 (2017). https://doi.org/10.1109/TCYB.2017.2662199
Mehrabian, A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)
Picard, R.W.: Affective computing. MIT Press, vol. 1, no. 1, pp. 71–73 (1997)
Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. Pers. Soc. Psychol. 17(2), 124–129 (1971)
Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)
Suwa, M., Sugie, N., Fujimora, K.: A preliminary note on pattern recognition of human emotional expression. In: Proceedings of the Fourth International Joint Conference on Pattern Recognition, Kyoto, Japan, pp. 408–410 (1978)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)
Lecun, Y.: Generalization and network design strategies. In: Connectionism in Perspective (1989)
Lv, Y., Feng, Z., Xu, C.: Facial expression recognition via deep learning. In: International Conference on Smart Computing. IEEE, pp. 347–355 (2015)
Tang, Y.: Deep learning using linear support vector machines. Eprint Arxiv (2013)
The FER2013 dataset. https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data
Zheng, L., Shen, L., Tian, L.: Person re-identification meets image search. Comput. Sci. 1 (2015). https://doi.org/10.1109/TPAMI.2015.2505297
Acknowledgments
This work is supported by National Natural Science Foundation of China under Grant 61463032 and 61703198, Natural Science Foundation for Distinguished Young Scholars of Jiangxi Province under Grant 2018ACB21014, Open Fund of State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences under Grant 20180109.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, J., Mi, Y., Yu, J., Ju, Z. (2019). A Novel Convolutional Neural Network for Facial Expression Recognition. In: Sun, F., Liu, H., Hu, D. (eds) Cognitive Systems and Signal Processing. ICCSIP 2018. Communications in Computer and Information Science, vol 1006. Springer, Singapore. https://doi.org/10.1007/978-981-13-7986-4_28
Download citation
DOI: https://doi.org/10.1007/978-981-13-7986-4_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-7985-7
Online ISBN: 978-981-13-7986-4
eBook Packages: Computer ScienceComputer Science (R0)