Micro-Expression Recognition by Aggregating Local Spatio-Temporal Patterns

Abstract

Micro-expression is an extremely quick facial expression that reveals people’s hidden emotions, which has become one of the most important clues for lies as well as many other applications. Current methods mostly focus on the micro-expression recognition based on the simplified environment. This paper aims at developing a discriminative feature descriptor that are less sensitive to variants in pose, illumination, etc., and thus better implement the recognition task. Our novelty lies in the use of local statistical features from interest regions in which AUs (Action Units) indicate micro-expressions and the combination of these features for the recognition. To this end, we first use a face alignment algorithm to locate the face landmarks in each video frame. The positioned face is then divided to several specific regions (facial cubes) based on the location of the feature points. In the following, the movement tendency and intensity in each region are extracted using optical flow orientation histogram and Local Binary Patterns from Three Orthogonal Planes (LBP-TOP) feature respectively. The two kinds of features are concatenated region-by-region to generate the proposed local statistical descriptor. We evaluate the local descriptor using state-of-the-art classifiers in the experiments. It is observed that the proposed local statistical descriptor, which is located by the facial spatial distribution, can capture more detailed and representative information than the global features, and the fusion of different local features can inspire more characteristics of micro-expressions than the single feature, leading to better experimental results.