Publication - Integrating Organizational Control into Multi-Agent Learning

Authors: Zhang, Chongjie; Abdallah, Sherief; and Lesser, Victor
Title: Integrating Organizational Control into Multi-Agent Learning
Abstract: Multi-Agent Reinforcement Learning (MARL) algorithms suffer from slow convergence and even divergence, especially in large-scale systems. In this work, we develop an organization-based control framework to speed up the convergence of MARL algorithms in a network of agents. Our framework defines a multi-level organizational structure for automated supervision and a communication protocol for exchanging information between lower-level agents and higher-level supervising agents. The abstracted states of lower-level agents travel upwards so that higher-level supervising agents generate a broader view of the state of the network. This broader view is used in creating supervisory information which is passed down the hierarchy. The supervisory policy adaptation then integrates supervisory information into existing MARL algorithms, guiding agents’ exploration of their state-action space. The generality of our framework is verified by its applications on different domains (distributed task allocation and network routing) with different MARL algorithms. Experimental results show that our framework improves both the speed and likelihood of MARL convergence.
Keywords: Communication Protocol, Coordination, Distributed AI, Learning, Multi-Agent Systems, Task Distribution
Publication: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, Vol: 2, pp. 757 - 764
Editor: Decker, Sichman, Sierra, and Castelfranchi
Location: Budapest, Hungary
Date: 2009
Sources: PDF: /Documents/cjzhang/czhang_aamas09.pdf
Reference: Zhang, Chongjie; Abdallah, Sherief; and Lesser, Victor. Integrating Organizational Control into Multi-Agent Learning. Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, Volume 2, Decker, Sichman, Sierra, and Castelfranchi, ed., pp. 757-764. 2009.
bibtex: 
@inproceedings{Zhang-465,
  author    = "Chongjie Zhang and Sherief Abdallah and Victor
               Lesser",
  title     = "{Integrating Organizational Control into
               Multi-Agent Learning}",
  booktitle = "Proceedings of the 8th International Conference on
               Autonomous Agents and Multiagent Systems",
  volume    = "2",
  editor    = "Sichman Decker and Castelfranchi Sierra",
  pages     = "757-764",
  year      = "2009",
  address   = "Budapest, Hungary",
  url       = "http://mas.cs.umass.edu/paper/465",
}