Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPs
Publication Date
2010
Journal or Book Title
AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS
Abstract
POMDPs and their decentralized multiagent counterparts, DEC-POMDPs, offer a rich framework for sequential decision making under uncertainty. Their high computational complexity, however, presents an important research challenge. One way to address the intractable memory requirements of current algorithms is based on representing agent policies as finite-state controllers. Using this representation, we propose a new approach that formulates the problem as a nonlinear program, which defines an optimal policy of a desired size for each agent. This new formulation allows a wide range of powerful nonlinear programming algorithms to be used to solve POMDPs and DEC-POMDPs. Although solving the NLP optimally is often intractable, the results we obtain using an off-the-shelf optimization method are competitive with state-of-the-art POMDP algorithms and outperform state-of-the-art DECPOMDP algorithms. Our approach is easy to implement and it opens up promising research directions for solving POMDPs and DEC-POMDPs using nonlinear programming methods.
DOI
https://doi.org/10.1007/s10458-009-9103-z
Pages
293-320
Volume
21
Issue
3
Recommended Citation
Amato, C; Bernstein, DS; and Zilberstein, S, "Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPs" (2010). AUTONOMOUS AGENTS AND MULTI-AGENT SYSTEMS. 1292.
https://doi.org/10.1007/s10458-009-9103-z