Regular Papers

International Journal of Control, Automation, and Systems 2023; 21(9): 3057-3067

https://doi.org/10.1007/s12555-022-0555-0

© The International Journal of Control, Automation, and Systems

Distributed Multi-agent Target Search and Tracking With Gaussian Process and Reinforcement Learning

Jigang Kim, Dohyun Jang, and H. Jin Kim*

Seoul National University

Abstract

Deploying multiple robots for target search and tracking has many practical applications, yet the challenge of planning over unknown or partially known targets remains difficult to address. With recent advances in deep learning, intelligent control techniques such as reinforcement learning have enabled agents to learn autonomously from environment interactions with little to no prior knowledge. Such methods can address the explorationexploitation tradeoff of planning over unknown targets in a data-driven manner, streamlining the decision-making pipeline with end-to-end training. In this paper, we propose a multi-agent reinforcement learning technique (MARL) with target map building based on distributed Gaussian process (GP). We leverage the distributed GP to encode belief over the target locations in a scalable manner and incorporate it into centralized training with decentralized execution MARL framework to efficiently plan over unknown targets. We evaluate the performance and transferability of the trained policy in simulation and demonstrate the method on a swarm of micro unmanned aerial vehicles with hardware experiments.

Keywords Distributed system, Gaussian process, multi-agent reinforcement learning, target search and tracking.

Article

Regular Papers

International Journal of Control, Automation, and Systems 2023; 21(9): 3057-3067

Published online September 1, 2023 https://doi.org/10.1007/s12555-022-0555-0

Copyright © The International Journal of Control, Automation, and Systems.

Distributed Multi-agent Target Search and Tracking With Gaussian Process and Reinforcement Learning

Jigang Kim, Dohyun Jang, and H. Jin Kim*

Seoul National University

Abstract

Deploying multiple robots for target search and tracking has many practical applications, yet the challenge of planning over unknown or partially known targets remains difficult to address. With recent advances in deep learning, intelligent control techniques such as reinforcement learning have enabled agents to learn autonomously from environment interactions with little to no prior knowledge. Such methods can address the explorationexploitation tradeoff of planning over unknown targets in a data-driven manner, streamlining the decision-making pipeline with end-to-end training. In this paper, we propose a multi-agent reinforcement learning technique (MARL) with target map building based on distributed Gaussian process (GP). We leverage the distributed GP to encode belief over the target locations in a scalable manner and incorporate it into centralized training with decentralized execution MARL framework to efficiently plan over unknown targets. We evaluate the performance and transferability of the trained policy in simulation and demonstrate the method on a swarm of micro unmanned aerial vehicles with hardware experiments.

Keywords: Distributed system, Gaussian process, multi-agent reinforcement learning, target search and tracking.

IJCAS
July 2024

Vol. 22, No. 7, pp. 2055~2340

Stats or Metrics

Share this article on

  • line

Related articles in IJCAS

IJCAS

eISSN 2005-4092
pISSN 1598-6446