Active User Modeling and Assistance in Human Computer Interaction

1. Active driver assistance with dynamic Bayesian networks

In this research, we focus on modeling users with multiple modalities in sensory data using Bayesian networks, and then providing various assistances to meet the need of users in a timely and accurate manner. A generic framework for user modeling is given, which takes in consideration the context, state, and profile of users from multiple modality data including non-intrusive and verbal information. Dynamic Bayesian network is applied to capture the constantly changing nature of such tasks, and to infer about user’s need. The value of information and utility for such a system are appropriately defined for various assistances, tests, and questions. To reduce uncertainty in providing assistance, asking questions allows the direct access to the hypothesis of user’s need. A reliability model is proposed for the related problem with information sources especially for the above questioning. The system will work in a closed loop to fine-tune the provided assistance. All these methods help the system to automatically and actively engage appropriate assistance and information retrieval selection through a dynamic and active information fusion strategy. We use the framework in a driver assistance system integrating the multiple modalities in the sensory data about user, which could automatically detect driver’s urgent need and provide timely assistance. More …

2. Affective or mental state assessment

There are several new research areas in computer technology that could contribute to active user modeling and assistance. The important one among them is affective computing. Affective computing focuses on the means to recognize emotional intelligence. It studies the computer that would have the ability to recognize, express, and “have” emotions. The affective states are important indication of the user’s internal state, and intension and need. We propose our generic framework to apply Bayesian networks in user state modeling. We call our model “Context-Profile-State-Observation” model. This model uses observable evidence to dynamically infer unobservable information about user’s long and short-term states. The “Context-Profile-State-Observation” model captures the contextual information influencing or indicating the user state, the various observations resulted from a particular user state, and the specific information related to user’s profile. More …

3. Integration of cognitive model and DBN user model in user assistance systems

Cognitive models provide a means for applying the knowledge from psychology to user modeling in the design, improvement and implementation of interfaces. An early example is the descriptive KLM/GOMS models to predict the time to finish tasks. Cognitive models are computational process models that simulate how users perform a task by their ability to do the task in the same way. The functions of cognitive models could range from predicting time, choice and errors, to assisting users, and even to acting as surrogates. Some applications of cognitive user models are the “cognitive tutors” in school education, and the model to study driving behaviors. Our research integrates DBN and cognitive models in an intelligent user assistance system to provide accurate assistance in complex operational environments. More…