Feast: face and emotion analysis system for smart tablets | Multimedia Tools and Applications Skip to main content
Springer Nature Link
Log in

Feast: face and emotion analysis system for smart tablets

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face and emotion recognition is still an open and very challenging problem. This paper presents a system FEAST which is an intelligent control system of Smart Tablets. It involves manipulating user sessions to adapt the working environment to his emotional state. First, a face detection followed by face and emotion recognition is performed, then a profile change is made basing on the obtained results. The face detection is based on skin color and geometric moments and face recognition is done by merging two features spaces, namely, Zernike moments and EAR-LBP. A feature selection technique reducing the parameter space size is applied. The same parameters are used for the emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
¥17,985 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Japan)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Alastair JG, Darren G, Robert MF, Jon O (2008) Emotion rating from short blog texts. In Proceedings of CHI Florence, Italy

  2. Aouatif A, Rziza M, Driss A (2008) SVM-based face recognition using genetic search for frequency-feature subset selection. Image Signal Process Lect Notes Comput Sci 5099:321–328

    Article  Google Scholar 

  3. Ayadi ME, Kamel MS, Karray F (2011) Survey on speech emotion recognition: features, classification schemes, and databases. Pattern Recogn 44:572–587

    Article  MATH  Google Scholar 

  4. Black MJ, Yacoob Y (1995) Tracking and recognizing rigid and non-rigid facial motions using local parametric model of image motion. Proc Int Conf Comput Vision IEEE Comput Soc 374–381

  5. Burgoon JK, Jensen ML, Meservy TO, Kruse J, Nunamaker JF (2005) Augmenting human identification of emotional states in video. Intelligence Analysis Conference, McClean

    Google Scholar 

  6. Camurri A, Lagerlof I, Volpe G (2003) Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int J Hum Comput Stud 59(1–2):213225

    Google Scholar 

  7. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  8. Carlo S, Rada (2008) Learning to identify emotions in text. In SAC 08, Fortaleza

  9. Cristian XR, Rainer L (2012) A survey on visual adult image recognition. Multimed Tools Appl

  10. Crowley K, Sliney A, Pitt I, Murphy D (2010) Evaluating a brain-computer interface to categorize human emotional response, advanced learning technologies (ICALT). IEEE Int Conf 276–278

  11. Ekman P, Friesen WV (1978) Facial action coding system: a technique for measure-ment of facial movement. Consulting Psychologists Press, Palo Alto

    Google Scholar 

  12. Erik H, Boon KL (2001) Face Detection. Surv Comput Vision Image Underst 83:236–274

    Article  MATH  Google Scholar 

  13. Frank YS, Chao-Fa C (2004) Automatic extraction of head and face boundaries and facial features. Inf Sci 158:117–130

    Article  Google Scholar 

  14. Gunes H, Piccardi M (2006) A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior. Proc ICPR Int Conf Pattern Recogn 1:1148–1153

    Google Scholar 

  15. Gyu-Tae P, Bien Z (2000) Neural network-based fuzzy observer with application to facial analysis. Pattern Recogn Lett 21(1):93105

    Google Scholar 

  16. Haapalainen E, Kim S, Forlizzi JF, Dey AK (2010) Psycho-physiological measures for assessing cognitive load. Proc ACM Int Conf Ubiquit Comput ACM 301–310

  17. Han CC, Liao HYM, Chung Yu K, Chen LH (1997) Fast face detection via morphology-based pre-processing. In: Lecture notes in computer science 1311. Springer, Berlin, p 469476

    Google Scholar 

  18. Hong P, Siyu X, Lizuo J, Liangzheng X (2011) Efficient face recognition fusing Dy-namic morphological quotient image with local binary pattern. IWANN Part II LNCS 6692:228–235

    Google Scholar 

  19. Hua Y, Yihua C, Hang S, Yawen F (2012) The large-scale crowd analysis based on sparse spatial-temporal local binary pattern. Mutimed Tools Appl

  20. Jae-Kyung K, Won-Sung S, YangSun L (2012) Advanced knowledge sharing strategies based on learning style similarity for smart education. Comput Appl Biotechnol Multimed Ubiquit City 353:141–148

    Article  Google Scholar 

  21. James. J, Anurag R, Jestin J (2012) Architecture for secure tablet integration in automotive network. Proceedings of the FISITA

  22. Javad H, Karim F, Madjid A (2003) An efficient human face recognition system using pseudo Zernike moment invariant and radial basis function neural network. Int J Pattern Recogn 17:41–62

    Article  Google Scholar 

  23. Jen-Da S, Shyi-Ming C (2007) Feature subset selection based on fuzzy entropy measures for handling classification problems. Appl Intell 28:69–82

    Google Scholar 

  24. Jingru WM, Guangzheng WY (1999) Knowledge-based edge detection and feature extraction of human-face organs. Pattern Recogn Artif Intell 12(3):340346

    Google Scholar 

  25. Kapur A, Kapur A, Babul NV, Tzanetakis G, Driessen PF (2005) Gesture-based af-fective computing on motion capture data. In: ACII 17

  26. Karin S, Loannis P (1997) A fully automated approach to facial feature detection and tracking. Int Work Audio and Video Biom Person Authentification (AVBPA)

  27. Lai JH, Yuen PC, FENG GC (2001) Face recognition using holistic fourier invariant features. Pattern Recogn 34(1):95109

    Article  Google Scholar 

  28. Lanitis A, Taylor CJ, Cootes TF (1995) An automatic face identification system using flexible appearance models. Image Vision Comput 13(5):393–401

    Article  Google Scholar 

  29. Le Hoai B, Nguyen AT (2005) Using rough Set in feature selection and reduction in face recognition problem. Adv Knowl Discov Data Min Lect Notes Comput Sci 3518:226–233

    Article  Google Scholar 

  30. Le HT, Nguyen DTN, Tran SH (2011) A facial expression classification system integrating canny principal component analysis and artificial neural network. Int J Mach Learn Comput 1(4)

  31. Leonardis A, Bischof H (2000) Robust recognition using Eigen images. Comput Vision Image Underst 78(1):99118

    Article  Google Scholar 

  32. Mase K (1991) Recognition of facial expression from optical flow. IEICE Trans 74(10):34743483

    Google Scholar 

  33. Michal K (2010) Energy-based blob analysis for improving precision of skin segmentation. Multimed Tools Appl 49:463–481

    Article  Google Scholar 

  34. Nefian AV, Monson H, Hayes III (1998) Face detection and recognition using hidden Markov models. ICIP 1:141–145

    Google Scholar 

  35. New T, Foo S, DeSilva L (2003) Speech emotion recognition using hidden Markov models. Speech Comm 41:603623

    Google Scholar 

  36. Niemann H (1990) Pattern analysis and understanding. Springer, Verlag

    Book  MATH  Google Scholar 

  37. Nikolaidis A, Pitas I (2000) Facial feature extraction and pose determination. Pattern Recogn 33(5):17831791

    Google Scholar 

  38. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distribution. Pattern Recogn 29(1):51–59

    Article  Google Scholar 

  39. Osuna E, Freund R, Girosi F (1997)Training support vector machines, an application to face detection. Proc IEEE Conf Comput Vision Pattern Recogn 130–136

  40. Pei-zhi C, Shui-li CA (2010) New face recognition algorithm based on DCT and LBP

  41. Pfurtscheller G, Neuper C (2001) Motor imagery and direct brain-computer communi-cation. Proc IEEE 89:1123–1134

    Article  Google Scholar 

  42. Pitas I, Tsekeridou S (1998) Facial feature extraction in frontal views using biometric analogies. In: Proc of EUSIPCO 315–318

  43. Pitas I, Venetsanopoulos AN (1990) Non linear digital filters: Principles and Applica-tions. Kluwer Academic publisher

  44. Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):2338

    Article  Google Scholar 

  45. Ryu YS, Oh SY (2001) Automatic extraction of Eye and mouth fields from a face Im-age using Eigen features and multilayer perceptirons. Pattern Recogn 34(8):24592466

    Google Scholar 

  46. Satyanadh G, Vijayan A (2007) Selection for improved face recognition in multisensor images. Signals Comm Technol 109–120

  47. Schneiderman H, Kenade T (1998) Probabilistic modeling of local appearance and spatial relationships for object recognition. Proc. IEEE Conf. Comput Vision Pattern Recogn 4551

  48. Shenghua B, Shengliang X, Li Z, Rong Yan, Zhong S, Dingyi H, Yong Y (2012) Mining social emotions from affective text. In: IEEE Transactions on Knowledge and Data Engineering 24(9)

  49. Shizhi C, YingLi T, Qingshan L, Dimitris NM (2011) Recognizing expressions from face and body gesture by temporal normalized motion and appearance features. Comput Vision Pattern Recogn Work (CVPRW)

  50. Sung KK (1996) Learning and example selection for object and pattern detection. PhD thesis, Massachusetts Institute of Technology

  51. Taskeed J, Md. Hasanul K, Oksam C (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5)

  52. Tefas A, Kotropoulos C, Pitas I (2001) Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication. IEEE Trans Pattern Anal Mach Intell 23(7):735746

    Article  Google Scholar 

  53. Tian YL, Kanade T, Cohn J (2000) Recognizing lower face action units for facial expression analysis. Proc IEEE Int Conf Autom Face Gesture Recogn 484–490

  54. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):7186

    Article  Google Scholar 

  55. Venugopal KR, Patnaik LM (2012) Automatic facial expression. Recog AR-LBP ICIP CCIS 292:244–252

    Google Scholar 

  56. Wan-zeng K, Shan-an Z (2007) Multi-face detection based on down sampling and modi-fied subtractive clustering for color images. J Zhejiang Univ Sci A 8(1):7278

    Google Scholar 

  57. Xin-Mao H, Yi C (2013) Using homography relationship for auto-calibration in mobile smart-project device system. Multimed Tools Appl. doi:10.1007/s11042-013-1783-3

    Google Scholar 

  58. Yacoob Y, Davis L (1994) Computing spatio-temporal representations of human faces. Computer Vision and Pattern Recognition. Proceedings CVPR ’94, IEEE Computer Society Conference 70–75

  59. Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):3458

    Google Scholar 

  60. Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: A survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58

    Article  Google Scholar 

  61. Yazhou L, Hongxun Y, Wen G, Debin Z (2005) Illumination invariant feature selection for face recognition, advances in multimedia information processing—PCM. Lect Notes Comput Sci 3768:946–957

    Article  Google Scholar 

  62. Yilmaz A, G¨okmen M (2001) Eigenhill vs. Eigenface and Eigenedge. Pattern Recogn 34(1):181–184

    Article  Google Scholar 

  63. Yu-Tzu L, Ruei-Yan L, Yu-Chih L, Greg C (2012) Real-time eye-gaze estimation using a low-resolution webcam. Multimedia Tools Appl 65(3):543–568

    Google Scholar 

  64. Zhiping S, Xi L, Qingyong L, Qing H (2012) Extracting discriminative features for CBIR. Multimed Tools Appl 61:263–279

    Article  Google Scholar 

  65. Zhou N, Lipo W (2009) Class-dependant feature selection for face recognition. AdvNeuro-Inf Process Lect Notes Comput Sci 5507:551–558

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. REGIM: Research Groups on Intelligent Machines, University of Sfax, National Engineering School of Sfax (ENIS), BP 1173, Sfax, 3038, Tunisia

    Mohamed Neji, Ali Wali & Adel M. Alimi

  2. Department of Electronics, Faculty of Engineering, Badji-Mokhtar Annaba University, P.O. Box 12, Annaba, 23000, Algeria

    Messaoud Ramdani

  3. Faculty of Science and Technique, Department of Informatique, Mohamed Cherif Messaadia University of Souk Ahras, 41000, Souk Ahras, Algeria

    Abderrahim Benmohamed

  4. Department of informatique, Faculty of Engineering, Badji Mokhtar-Annaba University, P.O. Box 12, 23000, Annaba, Algeria

    Abderrahim Benmohamed

Authors
  1. Abderrahim Benmohamed
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mohamed Neji
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Messaoud Ramdani
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ali Wali
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Adel M. Alimi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Abderrahim Benmohamed or Ali Wali.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Benmohamed, A., Neji, M., Ramdani, M. et al. Feast: face and emotion analysis system for smart tablets. Multimed Tools Appl 74, 9297–9322 (2015). https://doi.org/10.1007/s11042-014-2082-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2082-3

Keywords