Publication - Self-Organization for Coordinating Decentralized Reinforcement Learning

Authors: Zhang, Chongjie; Lesser, Victor; Abdallah, Sherief
Title: Self-Organization for Coordinating Decentralized Reinforcement Learning
Abstract: Decentralized reinforcement learning (DRL) has been applied to a number of distributed applications. However, one of the main challenges faced by DRL is its convergence. Previous work has shown that hierarchically organizational control is an effective way of coordinating DRL to improve its speed, quality, and likelihood of convergence. In this paper, we develop a distributed, negotiation-based approach to dynamically forming such hierarchical organizations. To reduce the complexity of coordinating DRL, our self-organization approach groups strongly-interacting learning agents together, whose exploration strategies are coordinated by one supervisor. We formalize this idea by characterizing interactions among agents in a decentralized Markov Decision Process model and defining and analyzing a measure that explicitly captures the strength of such interactions. Experimental results show that our dynamically evolving organizations outperform predefined organizations for coordinating DRL.
Keywords: Control, Cooperative Negotiation, Coordination, Distributed AI, Distributed MDP, Framework, Learning, Multi-Agent Systems
Publication: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, pp. 739 - 746
Editor: van der Hoek, Kaminka, Lespérance, Luck and Sen
Location: Toronto
Date: 2010
Sources: PDF: http://www.cs.umass.edu/~chongjie/pub/aamas2010_zhang.pdf
PDF: /Documents/cjzhang/aamas2010_zhang.pdf
Reference: Zhang, Chongjie; Lesser, Victor; Abdallah, Sherief. Self-Organization for Coordinating Decentralized Reinforcement Learning. Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems, van der Hoek, Kaminka, Lespérance, Luck and Sen, ed., pp. 739-746. 2010.
bibtex: 
@inproceedings{Zhang-482,
  author    = "Chongjie Zhang and Victor Lesser and Sherief
               Abdallah",
  title     = "{Self-Organization for Coordinating Decentralized
               Reinforcement Learning}",
  booktitle = "Proceedings of the 9th International Conference on
               Autonomous Agents and Multiagent Systems",
  editor    = "Kaminka van der Hoek and Luck Lespérance and Sen",
  pages     = "739-746",
  year      = "2010",
  address   = "Toronto",
  url       = "http://mas.cs.umass.edu/paper/482",
}