Research Goal | Eye Tracking | Gaze Tracking | Face Tracking |Facial Feature Tracking |Facial Motion Recovery | Facial Expression Recogntion

Real-Time Facial Feature Tracking Under Significant Facial Expressions and Various Face Orientations
Facial features, such as eyes, eyebrows, nose and mouth, and their spatial arrangement, are important for the facial interpretation tasks based on face images, such as face recognition, facial expression analysis and face animation. Therefore, locating these facial features in a face image accurately is a crucial step for these tasks to perform well. However, in reality, the appearance of the facial features in the images varies significantly among different individuals. Even, for a specific person, the appearance of the facial features is easily affected by the lighting conditions, face orientations and facial expressions, etc. Therefore, accurate facial feature detection and tracking still remains a very challenging task, especially under different illuminations, face orientations and facial expressions, etc. In our research, we proposed an effective approach to detect and track twenty-eight facial features from the face images with different facial expressions under various face orientations in real time. The improvements in facial feature detection and tracking accuracy are resulted from: (1) combination of the Kalman filtering with the eye positions to constrain the facial feature locations; (2) the use of pyramidal Gabor wavelets for efficient facial feature representation; (3) dynamic and accurate model updating for each facial feature to eliminate any error accumulation; (4)imposing the global geometry constraints to eliminate any geometrical violations. By these combinations, the accuracy of the facial feature tracking reaches a practical acceptable level. Subsequently, the extracted spatio-temporal relationships among the facial features can be used to conduct the facial expression classification successfully.
Publications:
(1) Yan Tong and Qiang Ji, “Automatic Eye Position Detection and Tracking under Natural Facial Movement”, Passive Eye Monitoring, Editor: Riad I. Hammoud, pp. 67-84, Springer 2007. (2) Yan Tong, Yang Wang, Zhiwei Zhu, and Qiang Ji, “Robust Facial Feature Tracking under Varying Face Pose and Facial Expression”, Pattern Recognition, Vol. 40, No. 11, pp. 3195-3208, November 2007. (3) Zhiwei Zhu and Qiang Ji, Robust Pose Invariant Facial Feature Detection and Tracking in Real-Time, the 18th International Conference on Pattern Recognition (ICPR), Hongkong, August, 2006. (4) Yan Tong, Yang Wang, Zhiwei Zhu, and Qiang Ji, “Facial Feature Tracking using a Multi-State Hierarchical Shape Model under Varying Face Pose and Facial Expression”, the 18th International Conference on Pattern Recognition (ICPR), Hongkong, August, 2006. (5) Yan Tong and Qiang Ji, “Multiview Facial Feature Tracking with a Multi-modal Probabilistic Model”, the 18th International Conference on Pattern Recognition (ICPR), Hongkong, August, 2006.
Demos:
Real-time facial feature tracking demo (short version)	Real-time facial Feature tracking demo (long version)	Real-time facial feature tracking demo with estimated 3D head pose
Latest Demo: Real time facial feature tracking under large pose and scale changes