The US military has been researching AI pilots for several years. Now, participants in a competition have reached a milestone on the road to automated aerial combat.
The AI aces competed in the Alpha Dogfight Trials. The competition is part of DARPA's Air Combat Evolution program, which aims to automate air combat and increase human pilots' confidence in their future AI counterparts.
The Alpha Dogfight Trials initially pitted AI pilots from different companies against each other in a simulation, including an AI from defense giant Lockheed Martin and Heron Systems, a company specializing in automation.
The simulation is based on the open-source software JSBSim, in which the AI pilots control virtual F-16 fighter jets. The jets are armed with only one forward-facing machine gun, forcing them into direct dogfights that require a high level of flying and aiming skills.
AI vs. Pilot: five battles, five victories
There were a total of three competitions, the first in November 2019, when prototypes of the systems were presented. The first aerial battles took place in January 2020, when the AI pilots competed against programmed opponents from Johns Hopkins University.
In the third competition, which has now been completed, the AI pilots first competed against each other in a tournament. The Heron Systems flight AI was able to eliminate the competing Lockheed Martin AI.
It then won five out of five rounds against an Air Force pilot sitting in a virtual cockpit wearing VR headsets.
Deep reinforcement learning as the foundation for success
Heron Systems' AI pilot was trained exclusively with reinforcement learning. According to the developers of the winning system, competitors had tried to combine this learning method with manually programmed expert systems.
Inspired by the success of AI systems such as Deepmind's gaming AI AlphaStar, Heron Systems used a variant of the successful self-play method, in which the AI competes against itself countless times: In total, the developers trained over 100 AI pilots with their own goals, neural network architectures, and control options over the aircraft.
A different AI pilot then had to repeatedly compete against each of the 100 other pilots. According to the developers, this approach meant that their AI pilot did not specialize in one type of opponent and was therefore less prone to error.
Will human fighter pilots soon be a distant memory?
The Alpha Dogfight Trials were a complete success from an AI perspective. But whether an AI pilot can beat a human in real-world conditions remains to be seen, say Heron Systems developers.
The human pilot was at a distinct disadvantage in the simulation because the AI pilots had direct access to the coordinates, speed, and other information of the opposing aircraft. The human pilot, on the other hand, had to rely on his eyes in a low-resolution VR headset.
However, experiments such as Deepmind's AlphaStar have shown that AI systems can beat humans even without direct access to simulation data. The Starcraft AI uses only visual cues and can still deal with phenomena like fog of war on the map.
In real-world aerial combat, however, Heron Systems wants to do away with visual AI analysis of the environment. Instead, the company wants to work with radar and other sensor data that can be tapped directly from the aircraft's systems. This would save computing power and make better use of existing systems.
The next step is the use of an AI pilot in drones and unmanned aircraft. This presents many new challenges, such as the influence of wind or complex control systems. However, more computing power and specialized AI training could solve these problems.
Heron Systems believes that the biggest hurdle is building trust among the pilots who will have to fight side by side with these AI systems in the future. The nearly 85 percent win rate in simulated dogfights is a good start, says one developer.
