An Information-Based Learning Approach to Dual Control

Dual control aims to concurrently learn and control an unknown system. However, actively learning the system conflicts directly with any given control objective for it will disturb the system during exploration. This paper presents a receding horizon approach to dual control, where a multiobjective optimization problem is solved repeatedly and subject to constraints representing system dynamics. Balancing a standard finite-horizon control objective, a knowledge gain objective is defined to explicitly quantify the information acquired when learning the system dynamics. Measures from information theory, such as entropy-based uncertainty, Fisher information, and relative entropy, are studied and used to quantify the knowledge gained as a result of the control actions. The resulting iterative framework is applied to Markov decision processes and discrete-time nonlinear systems. Thus, the broad applicability and usefulness of the presented approach is demonstrated in diverse problem settings. The framework is illustrated with multiple numerical examples.