What’s Behind the OpenAI bot that Beat the World’s Best Dota Player

Last week, history was created in the realms of Dota 2 and AI. At The International, Dendi, one of the best players of the game, was beaten mercilessly in a 1v1, by a bot, two times in a row. The bot in question was made by OpenAI, a startup of Elon Musk.

OpenAIBot vs Dendi Dota2_small

Now you may have some questions. What is Dota? What is OpenAI? And why is this even important? To start with, Dota is a free-to-play MOBA game in which you control Heroes and your aim is to destroy the opponents Ancient. Many Dota fans may take offence to the last sentence as there so much to Dota then I have summed up.

The game is very complex with a very steep learning curve to it. I had embarked on a mission to learn Dota some time ago and it took me a week just to go through a basic How To video, where they explain the strategy of the game, what weapons, spells, heroes do and how to mine creeps. And this makes the whole AI vs Dendi battle important because it illustrates just how advanced AI and ML has become. So let’s see what’s powering that bot and do we need get scared about AI uprising.

Rise Of The Planet Of The AI

AI becoming the Skynet is a debatable problem. Elon Musk believes that if not kept in check, a Skynet like scenario is likely to happen. On the other hand, Mark Zuckerbergsays that AI is harmless and will not cause Doomsday Scenario. Elon Musk started OpenAI with this goal.

To prevent misuse of AI and research into AI which will not turn against us. And I am on his side.

But that’s not what we are here to discuss. The main topic is to know what’s powering the bot made by OpenAI. To get an understanding of it, I read some research papers and blogs on the subject and also contacted my friendAnmol Sharma who is an AI & ML researcher.

Reinforcement Learning

This is the name of the tech behind the OpenAI’s Dota bot. But before we delve into it, we need to understand how general AI, or specifically AI in Video Games work. AI is a very board term and contains many topics such as Machine Learning, Deep Learning and Neural Networks to name a few. To crudely define AI, we can say it’s the ability of a computer to learn from information and make decisions based on it’s learning. The more it learns, the more it’s correctness of decision increases.

How Is It Different From Normal Bot’s?

Bots in general video games may look like they are intelligent but it’s not a true form of AI. Almost all of these bots in games are programmed based on set If..This..Then..That statements. They just follow the statements and take actions based on the conditions written in the statement.

AI Bot Programming Example
An example of an If..Else.. statement for an AI bot

Above I have showed a very simple example. Statements in real games are much more complex with conditions within conditions. The more well defined and detailed these statements are, the more the bot will seem to be intelligent.

Rewards and Punishments

In AI lingo, what we call as bot is known as an Agent. The goal is to make the agent learn whatever we want it to learn. The agent learns through the same method as your dog learns, through rewards and punishments. Whenever the agent makes a mistake it is penalized and whenever it makes the correct guess or gives correct answer, it is rewarded. Through this system, the agent learns to associate the actions that it shouldn’t do to avoid penalty and perform the action(or similar actions) which give rewards.

AI_basic_Algorithm
An Algorithm is backbone of AI

If you closely think of it, it’s just like making a decisions based on certain conditions, much like algorithms do it. As I have shown above, a game bot, coming upon an explosive barrel, will take actions as per the algorithm. Algorithms in AI, obviously are vastly complex than this, which makes the whole system intelligent.

Coming to our OpenAI bot, according to the developers, it learnt the whole game in time of 2 weeks. During the whole time of 2 weeks, it was online 24×7 and playing against a copy of itself. That’s a remarkable milestone considering the complexity of the game and the fact it could beat Dendi.

Why Are Driver-less Cars Still Not Here?

This was one of the first question that popped in my mind when I heard about this whole event. If a bot can learn such a game and train itself to a point of beating top players, why are we still manning the wheel. And this was the same question I shot at my friend, Mr. Anmol Sharma, who has also researched on driveless cars. This is what he had to say.

In a game, that’s developed by a human, the number of variables are less. Plus, if the agent(bot) makes a mistake, the worst penalty is a round loss. In real world, one mistake is death.

Even though Dota is a complex game, the possible combinations of moves, spells and tactics, though still absurdly large, are still limited. And mistakes are allowed without any real penalty. But in case of a driver-less car Agent, mistakes can lead to dangerous accidents and even death.

waymo_car_2017_small
Waymo’s new Autonomous Car for 2017

In real world, the number of variables which the agent has to monitor are endless and situations arsing due to these variables, like another car suddenly changing lane, pedestrian jumping in front of car, are also endless. Plus, as Jeremy Clarkson put it, they will have to make moral and ethical decisions when on the verge of an immediate accident(though his theory is not valid).

The Future

We already know of Human controlled robot performing intrinsic surgeries. The next leap is AI-controlled robots performing it. And this is just in medical field. AI will sure be the dominant new technology in the next decade in many fields. But, AI can also be misused for nefarious purposes such as DDoS attacks, Ransomware Bots and hacking other PCs or systems of corporations.

TAY MS
The last public interaction of a bot was not so smooth

In regards to Dota and other games, we can expect to see such intelligent bots being made for other games also, against whom players can train to perform even better. And did I mention OpenAI will be doing a 5v5 at next year’s International? So I am off to brushing up my Dota skills to at-least defeat the in-built dumber bots. Divine Rapier here I come!

Ajinkya Bhamburkar

By

See more posts by this author.

Show CommentsHide Comments