Real-time facial expression recognition using local appearance-based descriptors

Abstract

In human-computer interaction the automatic face sensing and recognition of facial expressions is still a challenging task of affective computing, psychology and biomedical applications. The main goal of this paper is to increment a recognition rate of approaches for unobtrusive face sensing and automatic interpretation of emotions. The proposed approach explores local scale invariant feature transform descriptors for extraction of face key points used for face detection, recognition and then for encoding facial deformations in terms of Ekman’s Facial Action Coding System (FACS). Real-time face tracking and recognition is provided by quadratic discriminant analysis and Bayesian approaches as classification tools. Based on detected fiducial points, the accurate automatic recognizing six prototypical human facial expressions as well as detecting affective states in real-time scenes is provided by fuzzy inference system based on the proposed reasoning model. Carried out experiments demonstrate that Ekman’s FACS traditionally used in affective computing may be extended to interpretation of non-prototypical compound emotions using Plutchik psychological model of emotional responses. Conducted tests with faces from standard databases confirm that the proposed approaches for analysis of local image features provide robust, quite accurate, fast and low computational cost face sensing and facial expression interpretation.