Publication - MASPA: Multi-Agent Automated Supervisory Policy Adaptation

Authors: Zhang, Chongjie; Abdallah, Sherief; Lesser, Victor
Title: MASPA: Multi-Agent Automated Supervisory Policy Adaptation
Abstract: Multi-Agent Reinforcement Learning (MARL) algorithms suffer from slow convergence and even divergence, especially in large-scale systems. In this work, we develop a supervision framework to speed up the convergence of MARL algorithms in a network of agents. Our framework defines a multi-level organizational structure for automated supervision and a communication protocol for exchanging information between lower-level agents and higher-level supervising agents. The abstracted states of lower-level agents travel upwards so that higher-level supervising agents generate a broader view of the state of the network. This broader view is used in creating supervisory information which is passed down the hierarchy. The supervisory policy adaptation then integrates supervisory information into existing MARL algorithms, guiding agents` exploration of their state-action space. The generality of our framework is verified by its applications on different domains (i.e., distributed task allocation and network routing) with different MARL algorithms. Experimental results show that our framework improves both the speed and likelihood of MARL convergence.
Publication: University of Massachusetts Amherst Computer Science Department Technical Report, Vol: 08, Num: 03
Date: 2008
Sources: PDF: /Documents/cjzhang/umass-cs-08-03.pdf
Reference: Zhang, Chongjie; Abdallah, Sherief; Lesser, Victor. MASPA: Multi-Agent Automated Supervisory Policy Adaptation. University of Massachusetts Amherst Computer Science Department Technical Report, Volume 08, Number 03. 2008.
bibtex: 
@techreport{Zhang-457,
  author    = "Chongjie Zhang and Sherief Abdallah and Victor
               Lesser",
  title     = "{MASPA: Multi-Agent Automated Supervisory Policy
               Adaptation}",
  volume    = "08",
  number    = "03",
  year      = "2008",
  url       = "http://mas.cs.umass.edu/paper/457",
}