Publication - Multiagent Reinforcement Learning and Self-Organization in a Network of Agents

Authors: Abdallah, Sherief; Lesser, Victor
Title: Multiagent Reinforcement Learning and Self-Organization in a Network of Agents
Abstract: To cope with large scale, agents are usually organized in a network such that an agent interacts only with its immediate neighbors in the network. Reinforcement learning techniques have been commonly used to optimize agents local policies in such a network because they require little domain knowledge and can be fully distributed. However, all of the previous work assumed the underlying network was fixed throughout the learning process. This assumption was important because the underlying network defines the learning context of each agent. In particular, the set of actions and the state space for each agent is defined in terms of the agent`s neighbors. If agents dynamically change the underlying network structure (also called self-organize) during learning, then one needs a mechanism for transferring what agents have learned so far before (in the old network structure) to their new learning context (in the new network structure). In this work we develop a novel self-organization mechanism that not only allows agents to self-organize the underlying network during the learning process, but also uses information from learning to guide the self-organization process. Consequently, our work is the first to study this interaction between learning and self-organization. Our self-organization mechanism uses heuristics to transfer the learned knowledge across the different steps of self-organization. We also present a more restricted version of our mechanism that is computationally less expensive and still achieve good performance. We use a simplified version of the distributed task allocation domain as our case study. Experimental results verify the stability of our approach and show a monotonic improvement in the performance of the learning process due to self-organization.
Keywords: Distributed AI, Learning, Multi-Agent Systems
Publication: Proceedings of the Sixth International Joint Conference on Autonomous Agents and Multi-Agent Systems, pp. 172 - 179
Location: Honolulu
Publisher: IFAAMAS
Date: May 2007
Sources: PDF: /Documents/abdallah_aamas07.pdf
Notes: "Best Paper Award" Nominee.
Reference: Abdallah, Sherief; Lesser, Victor. Multiagent Reinforcement Learning and Self-Organization in a Network of Agents. Proceedings of the Sixth International Joint Conference on Autonomous Agents and Multi-Agent Systems, IFAAMAS, pp. 172-179. May 2007. "Best Paper Award" Nominee.
bibtex: 
@inproceedings{Abdallah-432,
  author    = "Sherief Abdallah and Victor Lesser",
  title     = "{Multiagent Reinforcement Learning and
               Self-Organization in a Network of Agents}",
  booktitle = "Proceedings of the Sixth International Joint
               Conference on Autonomous Agents and Multi-Agent
               Systems",
  publisher = "IFAAMAS",
  pages     = "172-179",
  month     = "May",
  year      = "2007",
  address   = "Honolulu",
  url       = "http://mas.cs.umass.edu/paper/432",
}