The thermal imaging devices (Middle-wave and Longwave Infrared) record the thermal radiations from the objects. Unlike visible images, thermal images are less sensitive to illumination variations, provide additional information, and are more useful in some special environments such as at night. Recently, many works focus on utilizing the thermal images to solve computer vision problems. Like facial analysis in the visible domain, accurate facial landmark localization is important for thermal facial analysis. However, there are limited efforts to improve the facial landmark detection on thermal facial images. To solve this problem, we proposed several methods.
In the first work [1], we propose a constrained deep transfer feature learning method for eye detection on thermal facial images (Figure 1). The constrained deep transfer feature learning method learns the thermal features for eye detection by exploiting the knowledge in both visible and thermal domains (Figure 2). It iteratively performs transfer and feature learning in increasingly deeper feature spaces to generate intermediate thermal spaces to progressively approach the final thermal space. In addition, we propose to exploit the thermal domain knowledge and incorporate such prior knowledge as constraint during transfer learning to ensure that the transferred data satisfies certain properties of the thermal domain. Experimental results demonstrate the effectiveness of the proposed method.
Figure 1. Constrained deep transfer feature learning for thermal eye detection. (a)(b) Pairwise visible and thermal facial images (from the MAHNOB laughter database). (c) Thermal mean eye.
Figure 2. General framework of the constrained deep transfer feature learning method.
In the second work [2], we propose a method for tracking the 28 facial landmark points on the thermal facial images (Figure 3). The proposed method is based on the Constrained Local Model (CLM) method, which combines the local point detection and the face shape model. We also have a recent work that performs facial landmark tracking on thermal facial images based on the cascade regression methods.
Figure 3. Facial landmark tracking on thermal facial images.
[1] Yue Wu and Qiang Ji, "Constrained Deep Transfer Feature Learning and its Applications", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. PDF
[2] Zheng Zhang, Jeffrey Girard, Yue Wu, Xing Zhang, Peng Liu, Umur Ciftci, Shaun Canavan, Michael Reale, Andrew Horowitz, Huiyuan Yang, Jeffrey F. Cohn, Qiang Ji, and Lijun Yin, "Multimodal Spontaneous Emotion Corpus for Human Behavior Analysis", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. PDF