Fully Automatic, Real-Time Detection of Facial Gestures from Generic Video

Abstract

A technique for detection of facial gestures from low resolution video sequences is presented. The technique builds upon the automatic 3D head tracker formulation of [11]. The tracker is based on registration of a texture -mapped cylindrical model. Facial gesture analysis is performed in the texture map by assuming that the residual registration error can be modeled as a linear combination of facial motion templates. Two formulations are proposed and tested. In one formulation head and facial motion are estimated in a single, combined linear system. In the other formulation, head motion and then facial motion are estimated in a two-step process. The two-step approach yields significantly better accuracy in facial gesture analysis. The system is demonstrated in detecting two types of facial gestures: “mouth opening” and “eyebrows raising.” On a dataset with lots of head motion the two-step algorithm achieved a recognition accuracy of 70% for the “mouth opening” and accuracy of 66% for “eyebrows raising” gestures. The algorithm can reliably track and classify facial gestures without any user intervention and runs in real -time.