Context-Aware Relational Learning for Cooperative UAV Formation

Robust cooperative unmanned aerial vehicle (UAV) formation in complex 3D environments is hampered by reward sparsity and inefficient collaboration. To address this, we propose context-aware relational agent learning (CORAL), a novel multi-agent deep reinforcement learning framework. CORAL synergistically integrates two modules: (1) a novelty-based intrinsic reward module to drive efficient exploration and (2) an explicit relational learning module that allows agents to predict peer intentions and enhance coordination. Built on a multi-agent Actor-Critic architecture, CORAL enables agents to balance self-interest with group objectives. Comprehensive evaluations in a high-fidelity simulation show that our method significantly outperforms state-of-the-art baselines like multi-agent deep deterministic policy gradient (MADDPG) and monotonic value function factorisation for deep multi-agent reinforcement learning (QMIX) in path planning efficiency, collision avoidance, and scalability.