DeepMind developed an AI capable of self learning, defeating StarCraft II players

The development of artificial intelligence (AI) is being so meteoric that you can't even imagine how far it will go. 

AI will change video games
AI will change video games

Only two years ago it was news that AlphaGo, an AI developed by DeepMind, a subsidiary of Google, had defeated the best Go player in the world. A few months later, DeepMind demonstrated that AlphaZero AI is capable of attaining a level of superhuman mastery in chess, go and shogi, and that it also does so by learning to play against itself. 

In rapid succession, AI has been overcoming increasingly difficult challenges. Subsequent advances have been related to games where there are many players and where there is incomplete information, such as poker or Shooting video games, achieving sounded successes. 

Thanks to all this, it is increasingly clear that artificial intelligence will be able to solve real problems in changing and complex environments, which could be useful in fields such as medicine, biochemistry and who knows what else. One wonders if the AIs will write books, direct financial markets or military campaigns.

Earlier this year, AI AlphaStar managed to defeat professional StarCraft II players, a real-time strategy video game characterized by its dynamism, its complexity and the enormous diversity of options available to players. However, this intelligence achieved its success based on the advantage of knowing the position of the enemies. This week, artificial intelligence has just climbed a new step in its evolution. 

DeepMind researchers have just published a study in the journal Nature In which they demonstrate that their AI is better than 99.8% of StarCraft II players. Unlike other attempts, this is the first artificial intelligence capable of reaching a supreme level in a multiplayer game, in which professionals compete, governed by the same conditions as human players.

The researcher Oriol Vinyals - DeepMind

The researcher Oriol Vinyals - DeepMind
This is a dream come true . I was a pretty serious StarCraft II player 20 years ago and I have been fascinated by the complexity of this game for a long time, said Oriol Vinyals, director of research. AlphaStar has achieved the level of great teacher - the maximum in the ranking of this video game - only with a neural network and general purpose learning algorithms that were unimaginable just ten years ago, when I was researching AIs for StarCraft based on rule systems.

This is the key point. It is not that AlphaStar is born knowing how to play perfectly, thanks to the dictates of the programmers, or taking advantage of the speed of reaction of a machine. 

The bottom line is that it is capable of learning on its own and achieving extraordinary performance even in an extremely complex and changing environment. Thanks to that, you can defeat professional players who have a high level of training and do so on equal terms.

StarCraft II: a thrilling war

To understand the importance of this milestone, it is necessary to understand what the video game is like. StarCraft II is a real-time strategy game released in 2010, set in a futuristic world. 

It is a simulation in which several players build a base, improve technology and produce units with which to fight and extract resources, in games ranging from 15 or 20 minutes to hours. It has millions of players worldwide, professional teams and an important tradition in universities and research institutions that work in the field of artificial intelligence. The reason is that it constitutes a highly challenging environment for these systems.

One of the battles fought by AlphaStar - DeepMind
One of the battles fought by AlphaStar - DeepMind

To start, each player can choose three different races, "Zerg", "Protoss" and "Terran", each of which has its strengths and weaknesses. The game begins with units of workers to collect resources, distributed by the map, and build buildings. Throughout the game, each player has to manage several small tasks of each unit, plan long-term movements and strategies and respond to those of the enemy. 

In the fighting, there are units specialized in defeating certain enemies and tactics to counter certain attacks. All this happens in real time, sometimes in thrilling battles, and the movements of the enemies are hidden by a fog of war unless scouts are sent.

In addition, this approach involves additional challenges. At every moment, there are more than billions of options for movements and actions, well above the 1026 options for each go movement. 

In addition, since the games last for tens of minutes, artificial intelligence does not immediately learn what the result of each of its actions is.

How to make a machine that learns by it self?

All this is what AlphaStar has had to face. Thanks to the use of a series of techniques, detailed in the "Nature" article, the machine has obtained a better score than 99.8% of StarCraft II players registered on the game server. 

In addition, AlphaStar has reached the level of grandmaster, the maximum possible, for the three available races. "We hope that these methods can be applied to many other domains," the authors explained.

How have they achieved all this? AlphaStar training began with direct learning from the game data, from which it managed to be better than 84% of active players and tested a series of options for action. 

Then, broad-spectrum machine learning techniques were applied, such as neural networks. The machine learned by playing against itself, receiving rewards, imitating and solving a kind of amnesia that made her return to lower skill levels. Finally, on this occasion, multiple agents were also used, autonomous artificial intelligence entities that have their own objectives, algorithms and methods.

This last point is one of the most important: the scientists created "La Liga", a group of agents destined to find errors in the strategies of other principal agents.

"The key idea of ​​the League is that playing to win is not enough," the study authors explained. "Instead, we need leading agents, whose goal is to win against everyone, and also the " exploitative " agents, who sacrifice themselves for the team and focus on helping the main agent get stronger."

Thanks to all this, AlphaStar was able to play online on the server, with the same conditions as any human player, and anonymously. The machine used an interface with information comparable to what a human would use and restrictions on the speed and number of simultaneous operations were added to make this intelligence as capable as a professional player.

"These results provide strong evidence that general learning techniques can prepare artificial intelligence systems to work in complex and dynamic conditions, " the authors said in the study. 

"The techniques we have used to develop AlphaStar will help improve the security and robustness of artificial intelligence systems and, we hope, can serve to move towards real world applications."

How does AlphaStar play StarCraft II?

The artificial intelligence (AI) developed by DeepMind has had the help of several professional StarCraft II players, who wanted to tell what the "personality" of this machine is.

According to Dario TLO W√ľnsch, player of the Team Liquid team, the style of this AI is impressive but not superhuman: The system has a lot of ability to assess its strategic position, and knows exactly when to face an enemy or not to do it, ”he said in a statement. However, the result is not from another planet: "It does not seem superhuman, or located beyond the level that a human could theoretically reach." 

In this sense, Grzegorz «MaNa» Komincz, another player of the Team Liquid has agreed: It is still possible to find some of the weaknesses of the system.

How does AlphaStar play StarCraft II?
How does AlphaStar play StarCraft II?

According to Diego Kelazhur Schwimer, player of the Panda Global team, AlphaStar is a' fascinating and heterodox player, with the reflexes and the speed of the "pros" but with totally own strategies and style. 

In his opinion, although some of his tactics are strange, it is possible that "they are actually the best way to play."