1 of 19

AI Competition Design

For Multi-Agent Environments / Games

Stone Tao

Lux AI Challenge and UC San Diego

1

AI Competition Design

2 of 19

Why talk about AI Competition Design?

2

AI Competition Design

3 of 19

AI Competitions

Research

Produce research
Potential to transform fields
Compare apples to apples

Programming Fun

Learning by doing
Creativity

3

AI Competition Design

Moreover, competitions are important as they serve the basis for a lot of foundational AI research and programming fun. From the ImageNet competition to the more recent MineRL competitions, these all spawn off a ton of research and transforms entire fields of study. Importantly, they also let us compare apples to apples, and notify the broader research community method X is the best, let’s study it more, or method Y is interesting, let’s branch off of it.

On the fun side of things, they can serve as a fun way to play games in a way, but with programming. Moreover, for competitions like battlecode which started in 2003, they’ve generated a legacy of battlecoders who have not only learnt a lot, but also been battlehardened by extremely fun but difficult AI programming competition.

Now I primarily want to discuss what makes a good AI competition today as there often are great competition ideas that seldomly get noticed or the attention they should receive because of design choices. Today I’ll be discussing them based on my insights from the Lux AI challenge season 1’s competition

4 of 19

4

880+ stars

1100+ Teams

20,000+ Submissions

8,000,000 Million matches run

Loads of fun!

Lux AI Season 1 by the numbers

AI Competition Design

5 of 19

5

Research

Competition

Quality

Rock Paper Scissors

Neuro MMO

Halite

Battlecode

Lux AI

Procgen

Battle Snake

Terminal

Deep Racer

AI Habitat

Screeps

Mine RL

Ideal

AI Competition Design

Simple 2D diagram

In general we can place the current multi-agent / game competitions along two axes, although there are a number of other dimensions I’ve tried to sort of PCA down to 2 dimensions and make things interpretable this way.

The research dimension covers a number of aspects, is this competition encouraging research, does it facilitate good research, and ultimately has it resulted in good research. Now research here doesn’t necessarily have to be ground breaking new results, I would consider someone’s master thesis on a competition also research. Many of these competitions often have students submitting their sort of senior projects on them and (I know since a few people actually sent me them for Lux AI).

The competition quality is a bit more vague, but it covers almost everything else necessary for a “good” competition. It answers the question of, to what extent would I enjoy competing in this competition even if i didn’t get rewards like money, publications, etc. This covers for example, the novelty of the competition, is it still awesome and interesting each time, does it bring interesting game concepts that would be interesting to tackle and solve? It also covers more user-centered design components such as is this competition user-friendly, does it have good feedback loops for development.

Some examples shown in this diagram are impacted by many many factors. For example, battle snake is super user-friendly and great simple visualizer, but it’s such a simple environment with basically no novelty that I cannot place it any higher to the right. Procgen is quite impactful in terms of research but competing in it is not super easy, potentially long wait times, among other problems.

Halite almost has everything you want in a good competition and rightly so it had at its peak 4000+ competitors in one season. Lux, while not matching in competitor count, somehow is the most popular on github, perhaps evidence that lots of people thought it looked cool but a higher percentage were disappointed compared to halite.

Now today I’ll focus mostly on…

—---------—---------—---------—---------—---------—---------

Brief analysis of each competition

RPS: doesn’t really need to be explained. The popular one that ran for a while ended up being incredibly inaccurate in ranking and really the best bots were those that took advantage (by chance matchups) of badly programmed bots.

Battle Snake: Effectively same game each year, and not super easy to start off with since it requires you to set up and maintain your own server. No research, its just the snake game (too easy)

Terminal: Same game every year with some configurations. Gated competition. Slow servers and low feedback. Not very researchable (bad engine implementation / not gym style). Very nice visualizer and game design, but gets boring eventually.

Battlecode: Super super super fun (got me into AI, not biased at all), but lacks a lot of user-friendly components. Slow matches (due to the game design) and poor visualizer. Impossible to really do too much extensive or useful research on due to the slowness of the games unless you have massive compute and also all in java.

Halite: One of the best competitions ever made. Sleek visualizer, simple easy to understand rules but tons of intricate methods and tactics to try and win. Not much “typicall” research that i’m aware of but many people posted in depth posts about their methods, from heuristics, to more advanced heuristics / image filtering based techniques to pixel-wise control policies with CNN based models in halite 3.

Lux AI: Good visualizer, engine is slightly slow but generally fast enough for RL people to still use and win the competition in season 1. Facilitated all kinds of approaches and many posts, various project papers/thesis (and one paper in a conference submission that i’m aware of :)). Generally simple rules and lots of room for different strategies.

AI Habitat: Need a GPU for anything; low popularity sort of for a competition backed by a whole company? Difficult to understand docs and tooling for those not in robotics

6 of 19

Competition Quality

6

AI Competition Design

7 of 19

Great Competition Environments

Human Centered Design

7

AI Competition Design

8 of 19

Great Competition Environment

8

I’ll be considering “preference-invariant” features

AI Competition Design

9 of 19

Great Competition Environments: Speed

Faster development time
Faster research

9

AI Competition Design

Environment speed is a no-brainer. If it’s faster, you develop faster, you get results faster, and you can do research faster and get more “return” per unit time waiting during training/experimentation.

And if there ever was the case that you wanted a slower environment speed, it’s always easier to go from fast to slow.

(In fact, I’ve actually wanted to make an RL environment that requires balancing execution time and planning+solving. In particular, it’s asynchronous mostly. Two or more teams submit actions whenever they like to. If one team submits earlier than the other, then perhaps they have a slight advantage in e.g. achieving something first like collecting a resource. But an earlier action submission means less time spent evaluating the action, so it might be an interesting balance. How long is an agent willing to wait before taking an action?)

10 of 19

Great Competition Environments:

Low Skill Floor, High Skill Ceiling

Encourage less experienced competitors to join

Increased Activeness
More visibility
More data, more diversity

Exciting problems to solve

This feature usually stems from simple rules that interact in interesting, emergent ways

Achieving low skill floors through

Simple rules
Intuitive rules

10

AI Competition Design

Low skill floor and high skill ceiling is a common phrase we use in the lux ai team. It generally refers to environments where creating a solution that gets some success is easy, but mastering it is difficult with lots of room to grow.

The key point is that you want less experienced competitors to feel like they can still compete and learn something, even if they get poor results, they at least get something is nonzero. Depending on circumstance your competition may or may not want more competitors regardless of experience. In the case of Lux AI, one of our key goals is accessibility so we try to build competitions that don’t restrict on programming and or research experience much. Another interesting point to observe is that when we gather more competitors, we generally produce more diverse approaches as well as the resulting programmed agent behaviors, resulting in highly diverse data in gameplay.

Moreover, if there is a low skill floor and high skill ceiling, it’s like your environment has simple understandable rules that interact in interesting, emergent ways. Some common game-design examples often used include grid-based movement. With or without collisions, just 4 different actions of simply moving up down right left can create a ton of combinatorially complicated patterns and as the number of agents scale, the more can happen.

Usually high skill ceilings are easy to create. Simply just add more, non-conflicting rules and features to the design pot and usually it’ll just be harder. Low skill floors however, will easily rise up and become potentially too difficult for most people to understand and compete. So a solution here to help widen the ceiling and floor gap is to introduce interactive and highly dynamic simple rules, in addition to also grounding your rules with intuitive concepts and themes. For example, in lux ai season 1 we introduced tree regrowth as a feature. Trees can be cut down for wood resources, but will grow back slower and slower the more it is cut down, a very simple and intuitive concept, yet somewhat complicated agents must learn to precisely control the tree growth and farm it when it is optimal, as well as protect their trees from enemy units. If it wasn’t a tree that had regrowth, this concept would be a little bit more difficult since there are two other resources in the game that can be collected, but cannot regrow. So instead of competitors memorizing which resource has which property, they can rely on prior knowledge to easily recall that ok “trees regrow, coal and uranium doesn’t magically regrow”.

11 of 19

Great Competition Environment: Progression

Environment difficulty ramp

Natural progression of strategy
Configurable environments (tutorial setting vs hard mode)
Different stages of competitions

Ideal “performance vs time spent” curve

Next slide

11

AI Competition Design

Progression is a bit different. It’s something that’s often baked in to every great video game. You want to ease a player into the game by slowly introducing more and more difficult mechanics until the player is aware of all the mechanics and must leverage them to play the game successfully. But does this apply to say a robotics manipulation and navigation environment?

For less-research focused competitions like Lux, halite, and battlecode, there are generally “obvious” levels of difficulty that makes the lives of competitors who write heuristic or hybrid agents much easier as well. If you knew that in general for Lux, you needed to control units to collect wood, build citytiles, fuel citytiles with more resources, you can generally construct some simple high-level strategies and implement them at a lower-level with ML or heuristics. As you progress through more and more difficult high-level strategies that are layered on top of each other, you generally notice your agent improve.

In research focused competitions, a strong example of this is Neuro MMO’s design of competition environments where the first stage uses a Player versus environment style environment, which is much easier to tackle since you don’t need to worry about an unknown opposing agent, but moves up the difficult with the addition of a Player versus player environment. Moreover, their environments support multiple map dimensions, allowing competitors to choose which ones to work on, with smaller being easier to test and learn on usually.

Now for competitions where the environment cannot have a progression such as say a robotics environment where it’s not obvious how you can improve on a solution since that’s actually just hard research, progression can come in the form of “simpler environments”. A lot of great research does this often, but it can also be done in competitions too where researchers benchmark their approach on simple to understand environments that perhaps most prior and naive methods can do ok on too, until they start tackling progressively harder environments that require more tricks and techniques.

12 of 19

12

Average

Performance

Over All

Participants

Time Spent

Ideal

Too Hard

Too Easy

Unbalanced

AI Competition Design

Average participant performance x time spent curve

Another form of progression comes in terms of time spent.

I often like to illustrate the ideal competition environment design with this curve. In general, the design and rules should enable on average, a curve, (labeled in gold), where the performance of your agent in terms of perhaps best average episode return or ELO score on a ranking leaderboard is logarithmic with respect to time spent.

For research competitions this may matter less, but for competitions that are seeking for the general public to compete, people who aren’t expecting to conjure amazing research and are just there to try and win prizes and/or have fun, having such a behavior in your environment is quite important in order to keep the competition accessible as well as interesting.

The big complaint people often have is they might spend a month on their approach, only to find out someone who spent a single day out performs them greatly. Discrepancies like this can happen, but should be kept at a minimum and hopefully kept low in magnitude. This kind of situation is evidence that a game might not be well balanced and I mark it in the purple line on the graph. Effectively at some point in the competition, participants figure out a super quick and easy solution to get a decent ranking, and suddenly all the new competitors follow suit, making longer term competitors feel they have wasted their time potentially. Another issue is if the competition is simply too hard where the best solution does about as good as the baseline, then obviously there is little research value coming out of the competition and a ton of frustration as well. Or in the reverse side, it’s too easy and there’s also just no research value either, in addition to potentially wasting more time as people who might think there’s room for improvement, find out there was none. These are marked here by the red lines.

Another point I want to emphasize that may be obvious but is sometimes overlooked by some competitions is the open-sourcing/provision of baselines, which is why I put the curve intercepting the performance axis above 0. If someone spent no time, they should at least be able to submit something that works and runs.

13 of 19

Great Competition Environment: Determinism and Fairness

Deterministic

Reproducibility (great for research!)

Fair (no bias to either team due to map, game dynamics etc.)

Every match is more valuable to watch, losses/wins are not because of bad luck
Less matches needed to determine rankings accurately

13

AI Competition Design

A ton of fun games are often have two or more teams and each team has different properties, think about the Civilization game series, FPS games like Counter strike, League of legends. So how come at Lux we never make AI environments that allow teams to have different compositions and properties? Well it comes down to contributing to the goals of accessibility and better feedback.

Importantly, if a game is always deterministic, then games you win and lose are more difficult to attribute to chance and randomness. Similarly, if a game is fair where each team starts on not just equal ground, but the same ground, then every match result is attributed to mostly your agent’s performance, not randomness

This results in two important things. One, there is better feedback for competitor per match played. Every time you watch a replay to analyze your agents performance you know that your wins and losses are because of your agent’s behavior, and you don’t need to throw out matches that have little value because the outcome was because of luck. Another point is that with more fair matches, there is less of a need to run extra matches to refine ranking calculations. Typically many matches are run because there is some amount of randomness. In the case of Lux the randomness typically only comes from map generation.

14 of 19

Great Competition Environments

Human Centered Design

14

AI Competition Design

15 of 19

Human Centered Design: Fast Feedback

Higher retention of competitors
More engagement
Measurements

Time until feedback (evaluation results)
Frequency of feedback (if applicable)
“Clicks” until submission

15

AI Competition Design

Fast feedback is not easy to achieve, but is nonetheless a very very important thing to strive for. Throughout Lux AI season 1, we were constantly bombarded with questions about how often submissions would be matched up against another submission, as well as tons of concerns over the accuracy of the final rankings.

Feedback here refers to fast evaluation feedback. Basically users, especially in competitions where they can’t realistically determine their ranking until they submit, want some fast feedback on their submission. E.g. did it work, is it doing well, are there any failures and more.

With fast feedback we often see a much higher retention of competitors compared to other competitions. Battlecode for example is notorious for having slow feedback, if not, no feedback at all, especially for a AI programming competition. There was actually one season where the ranked online leaderboard didn’t run until 2/3s of the competition had finished already. I think every single time, the number of competitors is incredibly high initially, but ultimately the number of submitted bots in the end is much less.

Some simple ways we gauge how fast our competition’s feedback is includes sort of like a time to first byte metric which is time until evaluation results begin to appear, frequency of evaluation results if that applies, and also clicks until submission.

16 of 19

Human Centered Design: Accessibility

Design Accessibility

Colorblind Friendly Design
Intuitive Interfaces (Visualizer and tooling)

Debugging Accessibility

Text and visual-based

Developer Accessibility

Easy to install and run

16

Concept art of the Lux AI visualizer

AI Competition Design

Accessibility is absolutely huge. An inaccessible competition can instantly cut off a section of your audience and render your competition unusable even.

In Lux, we tackled a few kinds of accessibility.

The commonly thought of one is designing for visibility and readability. In fact, Kaggle had people email them about a color-blindness problem before where the colors were indiscernible. At a certain scale you are more likely to have color-blind people competing. Another typical one is for people with glasses or bad eyesight, thanks to our team’s designer, we got some well designed higher contrast interfaces with good font choices.

Another one is debugging accessibility. From our feedback at the end of the season, competitors frequently used visual debugging tools in addition to text based ones by having agents also submit debug actions which are logged and saved in replay files. These included just basic ones like drawing lines, shapes, and text, which is actually inspired by the debugging tools battlecode 2020 provided. While people could build these tools themselves, it’s obviously time-consuming and providing these has resulted in people even using them to help explain their strategies in their forum posts on kaggle

Finally, I want to point out developer accessibility. Making a competition accessible requires making the programming part feasible, which means easy installation and running. Without these, competitors can easily feel unsupported and leave a competition without submitting anything. For Lux, we actually didn’t do a great job of this and a ton of competitors complained about cross-platform issues where our environment had some issues running on windows, or some tooling having bugs. While we did our best to ensure multi-language support, there were also still more languages to support like Go that an insignificant number of people wanted support for.

Now typically with competitions requiring a lot of compute, it’s difficult to avoid needing a GPU, especially in robotics competitions where visual RL pipelines are simply too slow without a GPU. But the main thing is it’s necessary to make an effort to reduce compute required, in addition to ensuring things like cross-platform support if possible. A really poor example of this is I think one of the Allen AI robotics competitions that required 4 GPUS to even run efficiently, whereas other competitions doing similar problems only need one.

17 of 19

Human Centered Design: Open Source

Let the community help you
Moving towards what is demanded by competitors
Encourage research that branches off of your work

17

AI Competition Design

18 of 19

Human Centered Design: Aesthetics

80% of respondents said good art/visuals increased the likelihood of competing
Stand out amongst the other competitions
Worth hiring a graphics artist if you can

18

AI Competition Design

80% of respondents in our survey from lux ai season 1 said good visuals actually increases their likelihood of competing in the competition. Our competition was actually one of the more viewed competitions on Kaggle, although I don’t have exact metrics, but the number of page views is beyond millions. I suspect that we could have had a ton more competitors had we fixed some of our accessibility issues earlier on and not changed some of the game parameters after the first month.

But it goes without saying that a good picture can hook someone into clicking your competition and reading more, which often times I think is the bottleneck for research heavy competitions that focus too much on competition research value, but too little on making the competition well designed and also remotely interesting to consider looking at. If your competition has loads of prize money, I honestly recommend finding a way to spend some of that money on improving the competition’s visibility via good art and marketing.

19 of 19

Thanks for listening

If you want to chat more, let’s chat!

Discord: StoneT2000#2103

Twitter: @Stone_Tao

Email: stonezt2019@gmail.com

19

AI Competition Design