Bayesian Network Parameter and Structure Learning Under Constraints

Based on our current work in Bayesian Network parameter learning using qualitative constraints, we have developed software that performs parameter estimation in Bayesian networks of known structure under various qualitative constraints. The methods and software are very general in the sense that any constraint on parameters is allowed, which includes a wide range of ideas discussed in the literature, such as qualitative influences, synergies, range and ratio constraints as well as the. Besides including constraints on the estimated model parameters, the software also allows incorporating specific soft constraints on the prior model parameters such as constraints on the Dirichlet prior. When all constraints are convex (which is the case for almost all important constraints defined in the literature), parameter learning is performed using convex optimization, which ensures to find a global optimum in polynomial time.

The parameter learning software can also handle the situation when data is incomplete. For incomplete data, we extend the expectation-maximization idea, using our methods developed for complete data The expectation step is conducted as usual, but the maximization is treated as a constrained optimization problem.

Experiments with both synthetic and real data show that the BN parameter learning accuracy can be improved significantly with some simple qualitative constraints, and the improvements are especially significant when training data is either insufficient, incomplete or unrepresentative.

Besides the parameter learning software, we also developed software for Bayesian network structure learning based on our latest ICML paper from data and expert's knowledge. The software can efficiently learn the optimal BN structure for up to 100 nodes, much higher than the 30 nodes achieved by the state of the art BN structure learning software. A long version of the ICML paper has been accepted by the Journal of Machine Learning Research.

Both softwares are implemented in Matlab and they are seamlessly integrated with Kevin Murphy’s BNT library so that we can call the functions in the BNT library. The sturcture learning software can be download from the following link . We will soon releasee the constained parametrer learning software. Feel free to contact us if you want to obtain the software immediately.