Human Detection, Tracking, and Activity Understanding

The Intelligent Systems Laboratory at RPI has long performed research related to human activity modeling and recognition. Specifically, our related efforts include two areas: computer vision and probabilistic graphical models. In computer vision, we have performed research in face detection and tracking, human detection and tracking, human activity modeling and recognition. These efforts have led to algorithms and software for real time multi-view face detection and tracking, multi-target human body detection and tracking, human action and facial activity recognition.

In probabilistic graphical models, we have significant experience in developing theories for graphical model learning and efficient inference for different graphical models including Hidden Markov Models, Conditional Random Fields, Dynamic Bayesian Networks, and the Influence Diagrams. In addition, we have applied graphical models to different applications including information fusion, human action modeling and recognition, facial activity understanding, image and video segmentation, and decision making. The latest research in human activity modeling and recognition has demonstrated the superior performance of the probabilistic graphical models over other competing methodologies. We are currently combining various graphical models to develop a unified hierarchical probabilistic model for activity modeling, representation, and inference.

Our current efforts include person-object activity/event recognition (DARPA VIRAT program), persistent detection, tracking and activity recognition of dismounts (sponsored by Army Research Office) and face recognition from distance for maritime applications (Sponsored by ONR). Our efforts in these areas have been supported by different governmental agencies including DARPA, ARO, ONR, AFOSR, and NSF.

Most Related Projects:

·  Human Tracking and Activity Recognition in Urban Environment (Sponsors:  DARPA VIRAT and Army)

·  Multi-people detection and tracking

·  Other Related Projects:

·        Human upper body tracking and hand gesture recognition (Sponsor: Mobinex)

·  Multi-view human face detection, tracking, and recognition (Sponsors: AFOSR, TSWG, and Darpa)

· A Robust Non-cooperative Face/Iris Recognition Scheme from distance in Maritime Domain (Sponsor: ONR via IAI)

·  Facial activity modeling and recognition (sponsor: Darpa)

·  Vision Based Human Computer Interaction:
Eye Detection, gaze tracking, facial feature tracking, 3D face pose tracking, facial expression analysis, and face tracking

·  Related Publications

·  Activity/action recognition datasets

For further information, please contact Prof.  Qiang Ji at