Awesome Safe MARL

A (very subjective) curated list of Safe Multi-agent Reinforcement Learning resources.

Preliminaries

Constrained Optimization

TODO

Control Theory

TODO

MBRL

TODO

Safe RL

• Constrained Policy Optimization, Achiam et al, 2017. Algorithm: CPO.

• Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation, Sootla et al, 2022. Algorithm: Saute MDP augmentation.

Graph Theory

TODO

Federated Learning

TODO

MARL

Books

• Part V of Algorithms for Decision Making by Mykel J. Kochenderfer, Tim A. Wheeler and Kyle H. Wray
• Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations by Yoav Shoham and Kevin Leyton-Brown
• Multi-Agent Coordination: A Reinforcement Learning Approach by Arup Kumar Sadhu and Amit Konar 💵
• Distributed Optimization-Based Control of Multi-Agent Networks in Complex Environments by Minghui Zhu and Sonia Martinez 💵

Papers (survey)

• Multi-agent Reinforcement Learning: An Overview, Busoniu et al, 2010.
• Multi-agent deep reinforcement learning: a survey, Sven Gronauer and Klaus Diepold, 2021.
• A Survey of Multi-Agent Reinforcement Learning with Communication, Zhu et al, 2022.

Papers (research)

POMDP for Multiple Agents

TODO

Value Factorization

• Multiagent Cooperation and Competition with Deep Reinforcement Learning, Tampuu et al, 2015.
• Value-Decomposition Networks For Cooperative Multi-Agent Learning, Sunehag et al, 2017.
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning, Rashid et al, 2018. Algorithm: QMIX.
• QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning, Son et al, 2019.
• QPLEX: Duplex Dueling Multi-Agent Q-Learning, Wang et al, 2020.

Policy Gradient

• Counterfactual Multi-Agent Policy Gradients, Foerster et al, 2018.
• Distributed Multi-Agent Reinforcement Learning by Actor-Critic Method, P. Heredia, S. Mou, 2019.

Communication

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning, Foerster et al, 2016.
• Learning Multiagent Communication with Backpropagation, Sukhbaatar et al, 2016.
• TarMAC: Targeted Multi-Agent Communication, Das et al, 2019.
• Learning to Ground Multi-Agent Communication with Autoencoders, Lin et al, 2021.
• Differentially Private and Communication Efficient Collaborative Learning, Ding et al, 2021.

DLQR

• Distributed Q-Learning for Dynamically Decoupled Systems, Siavash Alemzadeh, Mehran Mesbahi, 2018.
• D3PI: Data-Driven Distributed Policy Iteration for Homogeneous Interconnected Systems, Alemzadeh et al, 2021.
• Data-Driven Distributed Optimal Consensus Control for Unknown Multiagent Systems With Input-Delay, Zhang et al, 2018.
• Distributed Q-Learning with State Tracking for Multi-agent Networked Control, Wang et al, 2020.
• Distributed Adaptive Linear Quadratic Control using Distributed Reinforcement Learning, Daniel Goerges, 2019.
• Distributed Linear-Quadratic Control with Graph Neural Networks, Fernando Gama, Somayeh Sojoudi, 2021.
• Distributed Online Linear Quadratic Control for Linear Time-invariant Systems, Ting-Jui Chang, Shahin Shahrampour, 2020.
• Learning Distributed Stabilizing Controllers for Multi-Agent Systems, Jing et al, 2021.

Vehicle Platoon

• Distributed Nonlinear Model Predictive Control for Connected Autonomous Electric Vehicles Platoon with Distance-Dependent Air Drag Formulation, Caiazzo et al, 2021.

Engineering

Building Multi-agent Environments

• Multi-Agent Learning Environments, blog post by Lukas Schäfer
• PettingZoo Neurips'21 paper
• PettingZoo Documentation
• ma-gym with minimal-marl, set of baselines for vanilla MARL problems by Anurag Koul
• Ray Multi-Agent and Hierarchical Environments

Related awesome lists

• Awesome MARL