In the area of automatic facial expression analysis, due to the inability to separate face pose from facial expression accurately, most of the facial expression recognition systems developed so far require the subject facing the camera directly without significant head movements. But once the face pose is precisely estimated, its effect can be eliminated successfully so that the estimated nonrigid motion of facial expression will be independent of the face pose. Subsequently, the user can move his head freely in front of the camera while the facial expression can still be recognized.
Via the proposed motion decomposition technique, the rigid facial motion related to the face pose and the non-rigid facial motion related to the facial expression can be separated successfully from a face image. Therefore, based on the recovered non-rigid facial motions from the face images, a computational model is constructed to model and understand the facial expressions with the use of Dynamic Bayesian Networks (DBN). Finally, a real-time facial expression recognition system is built so that the six basic facial expressions can be recognized successfully under natural head movements.