Emotion recognition from scrambled facial images via many graph embedding

Abstract

Facial expression verification has been extensively exploited due to its wide application in affective computing, robotic vision, man-machine interaction and medical diagnosis. With the recent development of Internet-of-Things (IoT), there is a need of mobile-targeted facial expression verification, where face scrambling has been proposed for privacy protection during image/video distribution over public network. Consequently, facial expression verification needs to be carried out in a scrambled domain, bringing out new challenges in facial expression recognition. An immediate impact from face scrambling is that conventional semantic facial components become not identifiable, and 3D face models cannot be clearly fitted to a scrambled image. Hence, the classical facial action coding system cannot be applied to facial expression recognition in the scrambled domain. To cope with chaotic signals from face scrambling, this paper proposes an new approach - Many Graph Embedding (MGE) to discover discriminative patterns from the subspaces of chaotic patterns, where the facial expression recognition is carried out as a fuzzy combination from many graph embedding. In our experiments, the proposed MGE was evaluated on three scrambled facial expression datasets: JAFFE, MUG and CK++. The benchmark results demonstrated that the proposed method is able to improve the recognition accuracy, making our method a promising candidate for the scrambled facial expression recognition in the emerging privacy-protected IoT applications. Elsevier Ltd