|
Lei Zhang |
|||||||
|
[Home] |
[Research] |
|
|||||
|
|
|||||||
|
An Extended Chain Graph and its Applications in Computer Vision Motivation: Both undirected graphical models (e.g., MRFs, CRFs) and directed acyclic graphical models (e.g., BNs, DBNs, HMMs) have limited modeling power and expressiveness. They cannot model heterogeneous relationships among random variables. For example, undirected graphical models cannot model "explaining-away" type of relationships, while directed graphical models cannot effectively model mutual dependence with a loop. In contrast, Chain Graph subsumes both undirected graphical models and directed graphical models and is therefore expressive enough for modeling extremely complex and heterogeneous relationships in probabilistic modeling. Unfortunately, the utility of Chain Graph in real application is still very limited due to some reasons including the difficulty of learning a Chain Graph with very general topology and lacking an efficient way to perform probabilistic inference in Chain Graph to solve complex problems. Existing Chain Graphs ever used in real applications typically have simplified structures, therefore could not fully leverage the beauty and power of Chain Graph. An Extended Chain Graph Model: To push the utility of Chain Graph in real world applications, we developed an extended Chain Graph modeling theory and successfully demonstrated its usefulness in solving two challenging computer vision and image analysis problems: human activity recognition and image segmentation. Specifically, we developed a suite of methods for constructing, learning, and inference in the extended Chain Graph model: § We construct the Chain Graph structure according to the nature and semantic meaning of the relationships between different variables (i.e., entities). We parameterize different types of links in the graph by taking into account the nature (e.g., causality or correlation) of these links and the difficulty of learning and inference. § Once the Chain Graph is constructed and the links are parameterized with appropriate potential functions or conditional probability distributions (CPDs), we derive the joint probability distribution (JPD) of all random variables based on the Global Markov Property in Chain Graph and using the idea of Master Graph and Component Subgraphs. § To learn the model, we developed a joint parameter learning method that combines an analytical learning and a contrastive divergence (CD) based numerical learning to learn different modeling parts of the model. This method maximally leverages the factorization of the JPD into different local functions (i.e., factors). § We finally convert the extended Chain Graph into a Factor Graph representation based on the factored JPD. We leverage principled inference methods developed for Factor Graphs, such as Sum-Product algorithm or Max-Product algorithm, to perform probabilistic inference and solve the problem. Below is a (relatively simple) example to illustrate the extended Chain Graph model and its factored joint probability distribution.
Figure 1. An extended Chain Graph model (left) for illustration of the concept and its represented joint probability distribution (right)
Applications: We have successfully applied the extended Chain Graph model to solve two challenging computer vision problems: multi-subject human activity recognition and 2D image segmentation. We also believe this model should be applicable to many computer vision and image analysis problems that require modeling of the heterogeneous relationships among different entities.
Example results: I) Here are the extended Chain Graph model for image segmentation and some image segmentation results on different datasets.
Figure 2. An oversegmentation of the input image (a), a part of the edge map for illustration (b) and an extended Chain Graph segmentation model (c) that corresponds to the part (b)
Figure 3. Example image segmentation results. In each group the first row is the raw images; the second row the edge map of oversegmentation; and the third row the segmentation masks. The left side shows examples for the Weizmann horse dataset and the right side shows examples for the VOC2006 cow images.
Figure 4. Examples of image segmentation results for Caltech Cars (1999) dataset
Figure 5. Examples of image segmentation results for MSRC2 dataset. The third row corresponds to the groundtruth labeling, where the dark color corresponds to areas without groundtruth labeling and therefore not considered for evaluation.
II) Here are the extended Chain Graph (both the static model and the dynamic model) for human activity recognition and some quantitative comparison with two Dynamic Bayesian Network (DBN) models and the static model. Please see our recent publication "Probabilistic Image Modeling with an Extended Chain Graph for Human Activity Recognition and Image Segmentation" on IEEE Transactions on Image Processing for more details.
Figure 6. The static Chain Graph model (top) and its dynamic extension, i.e. the dynamic Chain Graph model (bottom), for multi-subject human activity recognition
Table 1. Quantitative comparison between the dynamic Chain Graph model, two DBN models, and the static Chain Graph model for human activity recognition
Related publications:
|
|||||||
|
|
|||||||
