Home About EIS →

StarCraft AI Competition Results

The AIIDE 2010 StarCraft AI Competition has come to a close. The challenge given to competitors was to build the best performing bot for an immensely popular, commercial game. The competition consisted of four tournaments of varying complexity. This was the first year the competition was held and it turned out to be a success. Even though no prizes were offered, twenty-eight teams participated in the competition. My presentation on the competition provides an overview of the  participants and results.

The showcase game of the competition was a bot versus human match. In the exhibition match, =DoGo=, a World Cyber Games 2001 competitor played against the top ranking bot of the competition.  The result was an exciting man versus machine match highlighting the state of the art in real-time strategy game AI.

While the expert player was capable of defeating the top performing bots in the competition, the results are quite encouraging. Read on for complete results.

Tournament 1: Micromanagement
The first tournament evaluated bots in unit micromanagement scenarios.The winner of this tournament was FreSCBot, which uses multi-agent finite-state machines. The runner-up was Sherbrooke, which also uses state machines. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (red) and Sherbrooke (blue).

Tournament 2: Small-Scale Combat
The second tournament evaluated bots in small-scale combat scenarios. The results of this competition mirrored the results of the first tournament: FreSCBot won and Sherbrooke was runner-up. Complete tournament details are available here and the full results are posted here. The video below shows the final match between FreSCBot (purple) and Sherbrooke (orange).

Tournament 3: Tech-Limited Game
The third tournament tested bots in a tech-limited StarCraft environment which requires reasoning at strategic and tactical levels, but omits much of the complexity of the full version of StarCraft. The winner of this tournament was Mimic Bot, which attempted to mirror the opponent’s strategy while also performing a gas steal and applying fallback strategies. The runner up was Botnik, which executes a Zealot rush strategy. Complete tournament details are available here and the full results are posted here. The video below shows the final match between MimicBot (red) and Botnik (teal).

Tournament 4: Full Gameplay
The final tournament was a best-of-five, double-elimination tournament simulating a professional gaming competition. The winner of this tournament was Overmind, a Zerg bot that effectively scouted its opponents, interrupted their economy, and performed Mutalisk harassment until victory was ensured. The runner-up was Krasi0, a Terran bot with excellent defense and pushing capabilities. Complete tournament details are available here and the full results are posted here. The video below shows an exert from one of the games between Overmind (yellow) and Krasi0 (brown).

Highlight #1:  Krasi0 versus Skynet
During this minute of action between bots Krasi0 and Skynet, it was difficult to distinguish whether the game was being played by skilled human players or the StarCraft equivalent of Deep Blue.

Highlight #2: Berkeley’s Mutalisk Evolution
Contemporary StarCraft wisdom tells us that the best way to use mutalisks is to clump them. In human versus human battles, this makes it difficult to single out the weaker mutalisks, because the units are stacked on top of each other. However, UC Berkeley’s team identified a flaw in this tactic; it reduces the damage output of each individual mutalisk, because not all mutalisks will fire when using this tactic. Instead, they employed a model in which mutalisk are always moving, maximizing damage output while simultaneously maximizing movement.

Conclusion
The inaugural StarCraft AI Competition was a huge  success! I would like to thank Blizzard, BWAPI, AAAI, and all of the participants in making this event a reality. The next competition will be hosted by the University of Alberta at AIIDE 2011.

Update: Competition Videos
I’ve recorded a few of the more interesting games from the StarCraft AI Competition. All replays from the competition are available on the competition website and I encourage people to post additional videos and commentaries.

Here’s one of the matches from the finals between Krasi0 and Overmind (Replay):

This video shows the initial match between =DoGo= and Krasi0, with DoGo playing as Protoss (Replay):

One of the highlights of the competition, Skynet versus Krasi0. This video shows the match-up on Heartbreak Ridge (Replay):

The semi-finals between Chronos and Overmind (Replay):


About the author:  Ben Weber is a PhD student at UC Santa Cruz. Read more from this author


This entry was posted in Academics and tagged , , , , , , . Bookmark the permalink. Both comments and trackbacks are currently closed.

14 Comments

  1. Foaad Khosmood
    Posted October 15, 2010 at 1:19 AM | Permalink

    That’s really great Ben. Inspiring! Have you thought about a round-robin prelim round before an elimination round?

    Good job organizing all this!

  2. FTWinston
    Posted October 15, 2010 at 8:29 AM | Permalink

    Pretty impressive across the board. Not sure what the gas steal in MimicBot vs Botnik achieved though, except a tiny delay in the Botnik’s upgrade time, at the expense of having 1 more zealot early on, which looked like it would have been handy.

  3. bosco
    Posted October 15, 2010 at 9:06 AM | Permalink

    That was great,i think all the teams should also have provided their code to the public.And is there a possibility of a similar API being released for Wings Of Liberty?

  4. Posted October 15, 2010 at 10:46 AM | Permalink

    The gas steal was most useful in earlier stages of the competition. During the final match the steal was not useful, but it did delay the opponent from attacking.

  5. Jim Raynor
    Posted October 15, 2010 at 10:47 AM | Permalink

    Really cool competition. Just wondering … can the bots in the competition beat the built-in Starcraft AI? Maybe consider bot vs. standard StarCraft AI as a tournament type next time? Bots that win against the AI created by professional game programmers would certainly create even further interest from the game industry.

  6. Posted October 15, 2010 at 10:47 AM | Permalink

    Releasing source was not a requirement for the competition. Some participants have posted their source: http://eis.ucsc.edu/StarCraftParticipants

  7. Posted October 15, 2010 at 6:12 PM | Permalink

    I don’t have actual results myself, but I strongly suspect that most of the entrants could beat the built-in Broodwar AI, which is heavily script-based and not terribly efficient in terms of resource usage, especially after the initial rush. That was actually a major motivation for our StarCraft related work here at EIS: RTS gameplay is such a hard problem that even industry AI teams are often unable to come up with something that’s challenging for human players without cheating.

  8. Jacob
    Posted October 19, 2010 at 9:32 AM | Permalink

    I noticed the next one is at the U of A in Edmonton. Any chance this will be open to the public, in terms of viewing? I live in Edmonton, and would definitely like to check this out.

  9. Belle Daffy
    Posted November 29, 2010 at 12:22 PM | Permalink

    I am always surprised to hear people speaking about machines vs. men (or women) in any capacity. The fact is that behind every machine, regardless of what it does, stand men (or women).

    So, to speak of machines vs. men makes no sense. Speaking of “Men behind machine vs. men without machine or men in front of machine” would make much more sense to me.

  10. Paul
    Posted January 19, 2011 at 5:52 AM | Permalink

    The page says that all replays are posted, but links to a page which only has some human Vs. AI games, not replays of the AI games.

  11. Posted January 19, 2011 at 12:47 PM | Permalink
  12. Joe
    Posted January 19, 2011 at 8:54 PM | Permalink

    Will we see this someday soon for starcraft 2? Also one big question is when can we finally play against it? I want some practice in brood war with no lag and these AIs would be perfect practice and interesting to see in person in an actual game.

  13. Posted January 19, 2011 at 11:11 PM | Permalink

    The bots are currently available for download, but running against the bots is not trivial. Hopefully, there will soon be a more streamlined approach in place for playing against the bots.

    As far as StarCraft 2, I do not expect any tools such as BWAPI to be available for several years, if ever.

  14. Joe
    Posted January 20, 2011 at 3:54 PM | Permalink

    Alright. Hopefully in the future someone can make it easier to actually play against these bots without needing multiple machines and all of those programs. I also think later on it would be a good idea and much more impressive if there is an apm limit put on the computer or slightly above the 400 or so average of top tier players and maybe multitask like a person also. Although I realize at the moment its probably not possible to do and give the bot a realistic chance at winning. Its still impressive for a game as complex as BW.

8 Trackbacks