Publication - Coordinating Multi-Agent Reinforcement Learning with Limited Communication
| Authors: | Zhang, Chongjie; Lesser, Victor | ||||
| Title: | Coordinating Multi-Agent Reinforcement Learning with Limited Communication | ||||
| Abstract: | Coordinated multi-agent reinforcement learning (MARL) provides a promising approach to scaling learning in large cooperative multi-agent systems. Distributed constraint optimization (DCOP) techniques have been used to coordinate action selection among agents during both the learning phase and the policy execution phase (if learning is off-line) to ensure good overall system performance. However, running DCOP algorithms for each action selection through the whole system results in significant communication among agents, which is not practical for most applications with limited communication bandwidth. In this paper, we develop a learning approach that generalizes previous coordinated MARL approaches that use DCOP algorithms and enables MARL to be conducted over a spectrum from independent learning (without communication) to fully coordinated learning depending on agents´ communication bandwidth. Our approach defines an interaction measure that allows agents to dynamically identify their beneficial coordination set (i.e., whom to coordinate with) in different situations and to trade off its performance and communication cost. By limiting their coordination set, agents dynamically decompose the coordination network in a distributed way, resulting in dramatically reduced communication for DCOP algorithms without significantly affecting overall learning performance. Essentially, our learning approach conducts co-adaptation of agents´ policy learning and coordination set identification, which outperforms approaches that sequence them. | ||||
| Keywords: | Coordination, Distributed AI, Learning, Multi-Agent Systems | ||||
| Publication: | Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, pp. 1101 - 1108 | ||||
| Editor: | Ito, Jonker, Gini, and Shehory | ||||
| Location: | St. Paul, MN | ||||
| Publisher: | IFAAMAS | ||||
| Date: | 2013 | ||||
| Sources: |
PDF: /Documents/czhang_aamas2013.pdf |
||||
| Reference: | Zhang, Chongjie; Lesser, Victor. Coordinating Multi-Agent Reinforcement Learning with Limited Communication. Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, Ito, Jonker, Gini, and Shehory, ed., IFAAMAS, pp. 1101-1108. 2013. | ||||
| bibtex: | @inproceedings{Zhang-520,
author = "Chongjie Zhang and Victor Lesser",
title = "{Coordinating Multi-Agent Reinforcement Learning
with Limited Communication}",
booktitle = "Proceedings of the 12th International Conference
on Autonomous Agents and Multiagent Systems",
editor = "Jonker Ito and Shehory Gini",
publisher = "IFAAMAS",
pages = "1101-1108",
year = "2013",
address = "St. Paul, MN",
url = "http://mas.cs.umass.edu/paper/520",
}
|
||||
