What can we learn from OpenAI?
There is a misconception in our community about the purpose of OpenAI—many wrongly assume that it was created to play Dota. That isn’t the case. Dota is a testing ground for a team of talented developers, who want to gauge how well their algorithms can learn to solve problems. Turns out, they do it pretty damn well.
The funny thing about the whole experiment, however restricted it was in terms of game rules, is that for the most part the AI converged on the same strategies and tactics as human players did the last 15 years. One hero in mid, double or triple sidelanes, rune control etc.—all that is pretty in line with what we as humans do when we play Dota. It looks and feels like a regular match.
Of course OpenAI turns things like coordination, focus fire, and macro decision-making up to eleven and it looks glorious, but this isn’t something humanly possible, and it's not going to be a focal point of our blog. There are other things the AI does differently. Things we, as players, might want to consider and possibly incorporate in our process of learning Dota.
Early laning
Human players really prefer their static laning in most games. They get there, looking or hoping for a favorable matchup, maybe switch once and then mostly stand there farming, trading blows and doing regular Dota things. OpenAI disagrees with this approach.
Rewatching the replays of both games, it is easy to see how frequently the AI switches lanes, rotating in or out to look for an edge. As soon as some hero gets to level 3, it starts looking for an opportunity to pressure and punish, and for AI it doesn’t matter, whether it is a core or a support who rotates—the only thing that matters is the success of the rotation.
From a strictly theoretical perspective it makes a lot of sense: if you rotate a hero out of a dual lane, you still have a hero in lane to soak up XP, so you are not losing economically on the map. Killing an enemy hero is almost definitely going to yield higher returns than the cost of the TP scroll and then, after the gank, you can rotate one or two heroes out, potentially starting another gank on the other side of the map.
It is not an XP-starved trilane that stayed in lane from the start, sharing XP, but rather fully capable three heroes, who converge on two enemy heroes with equal farm and level. And then, almost immediately, depending on the usage of resources, the same can be done on the second lane.
Naturally, humans are limited in this regard, as they can’t necessarily calculate precisely the chance of success and compare the expected values of rotation vs. non-rotation, but we’ve seen a similar approach back in 2015 from a team called CDEC and their star player, Agressif. Incidentally, his signature hero for the tournament was Gyrocopter as well, and he would also constantly rotate and maximize his hero’s efficiency. Up to a point it certainly worked and no one gets to the second place at TI by chance.
Farm Distribution
The outcome of this chaotic laning stage is that the spread of levels and gold on the OpenAI team is a lot more even, than what we are accustomed to, but the important part is that the sum of it is generally higher than that of the human team.
Unfortunately, the observers didn’t show the experience advantage of teams in the early game, but simply looking at levels alone, OpenAI always seemed to have a lead. They didn’t have “sacrificed” supports, running around level three 10 minutes into the game without boots. They preferred to have five capable and battle-ready heroes.
There is a concept of “playing around a hero” in Dota: professional teams in the current meta generally prefer to have two tempo-heroes who come online decently early and can make plays with the help of their supports around the map. OpenAI simply turns every hero they have into a hero that can be played around, at least until 15-20 minutes into the game, drastically increasing the amount of opportunities they have.
Naturally, over time, the inequality starts to spread and better farming heroes start getting more resources, but the inequality is never as drastic as with human teams. The AI doesn’t want to have a “top net worth” hero that will eventually win them the game. They simply want to maximize their chances of winning and decided that spreading resources is the best way to do it.
XP priority and buybacks
Another interesting aspect we’ve touched upon slightly is that OpenAI really likes its XP. Because of it, starting at a certain point, they start buying back frequently and seemingly unnecessarily. Buyback, TP to shrine and start farming jungle is usually a sign of tilt for a human player, but AI doesn’t get toxic or salty. It just wants to win.
Starting at around level nine, the AI would instantly buyback on most of its heroes to start getting things done on the map. The equal net worth spread ensured that even the “core” buybacks weren’t too costly, while most of their lineup had ways of pushing waves or flash-farming.
This ensured that the AI retained its XP lead, almost always had its lanes outpushed and was capable of fighting as a team at all times. Ironic, how OG was the team to experience FTW-buybacks from the opponent.
Closing thoughts
There is no denying that OpenAI won, for the most part, through its ability to make correct split-second decisions. Looking at how they focus fired targets, used their spells and calculated how long they could survive for in teamfights, it gets complicated to associate ourselves with the bots and believe that there is anything we can learn from them. After all, at this point in time, in a restricted game mode, OpenAI bots are a lot closer to “perfect Dota players” than any human players are.
Human players are not AI: they can’t know how much HP exactly they will have after an enemy spell usage, how much damage they will take over the course of a disable when being attacked by a certain hero or how long exactly it is going to take for the enemy hero to cover a certain distance.
The question is, given our knowledge of our own imperfection, is there a reason to try and replicate strategies and tactics OpenAI applies in Dota? AI that is fully aware of its own high capabilities through millions of trials and errors.
We don’t know if there is a reason and whether there will be a reward for it in the end, but we believe it is absolutely worth trying. Because unlike AI, that gets smarter and progresses from one generation to the next through victories, humans are more than capable of learning from mistakes and losses.
ayy hooman too nooob
what we should learn from the AI is that we should let our supports farm as well as our carries and that nobody should afk jungle
in other words... everyone should farm, but everyone should also prioritize taking objects or getting kills over farming in all stages of the game
so, stop tilting like a bitch if your support took your farm because thats exactly what he should be doing
maybe we should not divide players into supports and cores after all
Pure speculation here, but it seems like maintaining a consistent net worth and xp value across all heroes mitigates the potential to die, meaning you can commit a little more. However, it means you have to rely more heavily on everyone focusing the correct targets, chain your stuns better, etc. Similar to the six million dollar slam, if you know you can pull it off, you have little to lose. The AI can rely more on coordination than we can, so what it perceives as weakness is a little opposite. It kind of feels like a left field greed strat.
No any tier 1 teams playing against OpenAI only tier 2 as Og and Alliance, against them should play some VP, VG, Liquid, Secret, EG, maybe they can win.
For me the biggest takeaway from watching the two games is that the AI isn't selfish. If a TP rotation is needed they do it immediately - no hesitation. They don't take a millisecond to kill that last creep. If they need a ward or dust, they'll buy it - they had Gyro and Sven in the same lane (the offlane, no less). Neither functioned as a traditional support, and OG had a Riki on their team: the number one rule when playing against a Riki is to have wards and dust. Riki caused some headaches for OpenAI early on, but the AI played so hyper-aggressively right from the start that Riki never really got anywhere. He struggled to get pick-offs because the AI was never alone for more than a few seconds.
@guuz speaks the truth
@forgant hahahahaha, everyone knows OG is better than EG of those teams at least
@guuz Then why do all pro teams favor having a pos 6 support in the current meta with the exception of NP pos 5? There simply is not enough farm on the map for everyone to take it. Of course a core player should allow his support to farm the lane if there is an upcoming important item that is 200 gold away but in general the reason why one hero is a support and another a core is its ability to scale with items. For example: an Oracle, Warlock, Silencer, Disruptor (with the exception of AGHS, which is a game changer) with or without items are basically the same. A TB, PL or even something with earlier timings like Lycan/Razor needs items to be effective and get exponentially stronger with them.
I think these OpenAI do not give us too much of information due to limited hero pool and the lack of human factor on one side of the team (no hype, no tilt, no despair), so they should not be given as an example of how the game should be approached, especially in a PUB environment.
MENTALLY EXHAUSTED is exactlly right, tiny hero pool is like playing a different game.
It would be interesting to note about last hits in the laning stage: are they focused on cores or supports?
im not good at this game but one thing for sure, AI supports do farm!
This unfolding of future now sends me shivers. I'm scared of humans using AI to kill, opress, spy, segregate and more and more. With each milestone passed.
@mentally exhausted Thats why I said everyone should farm but also prioritize taking objects and killing over farming.
You farm on the way to your object/kill, you farm until your teammates arrive, you farm while you push. Basically you just farm on your way to your next goal.
So it doesnt matter if you didnt farm a lot because thats not the reason why you were there to begin with.
But indeed even pro players dont play making moves all the time, its much easier to simply have part of your team making moves while your carry farms, because the enemy team wont make enough moves to punish your strategy... Until a well coordinated team like the AI comes in and screws you over for having afk jungles.
Viper was the best example of this. In every game humans will choose getting Nethertoxin first for better farming, but the AI prioritize killing over farming so he gets Poison Attack first for the best killing potential. Viper still farmed, but he was focused on making rotations and only farmed in between rotations and thats exactly what we should do.
Time to add all the heroes to the Open AI bot pool. If you think about the game of Dota in relation to chess in terms of the amount of possible moves for any given game state, simply adding a dozen heroes to the pool adds millions of variables to any potential game when it comes to AI processing specific scenarios.
If the AI can destroy pro teams even with the full hero pool I will be thoroughly impressed. Right now tbh having a 17 hero pool just isn't even close to Dota. The bots aren't considering billions of variables that could occur with the full hero pool available.
I don't necessarily doubt they can do it since they are AI after all, but what they are doing right now is nothing compared to actually mastering Dota with all the heroes.
I'd be curious to know if the bots have a breaking point where they actually start getting less intelligent (or more noob) per average game because too many variables are present with the full hero pool and the data they use in their algorithm starts to become much less decisive and accurate in terms of their overall game play.
Just a theory that could occur... not sure if the current processing power and memory access could limit them in such a way.
From what I take from this, is the fact that the laning stage of game should be looked into... Even with 2-1-2 lanes, you could send 1 guy from a lane make it 3-1-1 for a gank and then send the lanes 2 guys over to the other lane to make it 1-1-3 and get 2 ganks off in quite a rapid succession... Within a minute or even faster if spell cooldowns permit... And you could then send the original last laner to top to make it 2-1-2 again, essentially swapping the lanes.
When I started Dota, I wondered why one lane is "better" than the other and the AI seems to agree with my initial assumption... The same amount of gold and xp walk down each of the lanes...
Then again, I am a bottom dweller here... Not much is going to change there... But you higher ups might see more of these changes implemented. I am still waiting on the shit tier item builds reach 2017...
@quuz I have to disagree, OpenAI plays the deathball strat only - and yes, in that case a more balanced xp distribution is common, but in normal Dota (full roster available) that just won't work consistently as the only strat (too many heroes that can easily clear waves and hold towers were unavailable to capitalize on it). A single KOTL could quite likely hold the game for long enough that the xp (and gold) difference between the "even farm team" and "classic position 1&2" would become too big to fight against. OpenAI plays on an insane timer that doesn't allow stepping off the gas, which works, as long as there is nothing to stop you - which there is, in normal Dota.
I think alesseo is right. If all heroes were available I don’t think the AI strat works. The limited heroes favor death ball... that’s why it values WD and other heroes that are either early game or provide sustain. I think AI is cool but It still feels kinda MEH because of the restrictions
I would be more surprise if human win against bot
Like AlphaGo
well said
چیکار کنم رنکم از اول دیواین باز بشه
I think the opposite. Imagine OpenAI mastering Invoker or Meepo ? With perfect timings and execution, humans have no chance against it.