Publication - Using Conflict Resolution to Inform Decentralized Learning

Authors: Cheng, Shanjun; Raja, Anita; and Lesser Victor
Title: Using Conflict Resolution to Inform Decentralized Learning
Abstract: Learning consistent policies in decentralized settings is often problematic. The agents have a myopic view of their neighboring states that could lead to inconsistent action choices. The fundamental question addressed in this work is how to determine and obtain the minimal overlapping context among decentralized decision makers required to make their decisions more consistent. Our approach is a two-phased learning process where agents first learn their policies offline within the context of a simplified environment where it is not necessary to know detailed context information about neighbors. These local policies are then applied in more complex “real” environments where it is expected that agents will encounter a much higher rate of inconsistencies (conflicts) with neighborhood actions. When conflicts are observed, agents switch to "special" states that augment local policy states with additional non-local state information and learn other actions to take in this specific situation. This results in action choices that are less likely to lead to conflicts. We evaluate our approach by addressing meta-level decisions in a complex multiagent weather-tracking domain. Experimental results show that our approach achieves good performance on utility and conflict resolution by exploring only a small fraction of the whole search space.
Keywords: Distributed AI, Learning, Multi-Agent Systems
Publication: Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, pp. 893 - 900
Editor: Ito; Jonker; Gini; Shehory
Location: St. Paul, MN
Publisher: IFAAMAS
Date: 2013
Sources: PDF: /Documents/cheng_aamas13.pdf
Reference: Cheng, Shanjun; Raja, Anita; and Lesser Victor. Using Conflict Resolution to Inform Decentralized Learning. Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems, Ito; Jonker; Gini; Shehory, ed., IFAAMAS, pp. 893-900. 2013.
bibtex: 
@inproceedings{Cheng-518,
  author    = "Shanjun Cheng and Anita Raja and Lesser Victor",
  title     = "{Using Conflict Resolution to Inform Decentralized
               Learning}",
  booktitle = "Proceedings of the 12th International Conference
               on Autonomous Agents and Multiagent Systems",
  editor    = "Jonker Ito and Shehory Gini",
  publisher = "IFAAMAS",
  pages     = "893-900",
  year      = "2013",
  address   = "St. Paul, MN",
  url       = "http://mas.cs.umass.edu/paper/518",
}