Proposal: AI Tournaments

[agentx]

I made contact with 0AD two updates before and was thrilled. Without any human enemy I learnt fighting the AI. Now it is no longer a matter of survive but whether I win within 40 minutes or so a random game. This surprises me, shouldn't an AI know everything about internal game mechanics and remain in principle unbeatable?

I don't expect the AI conquering the seas or building sophisticated walls around its cities but managing the simpler tasks first. So I thought, what could be removed from the game, so the AI could win? Like in playing chess with a beginner without towers and/or knights on my side. I played without champions, heroes, walls, fences, tower guarded resource traps, siege weapons or didn't advance to city phase all to find out what the AI is capable of.

I was not impressed, the AI doesn't repair its last civic center at 1% health while most units stand around freezed. It kept sending female units gathering resources guarded by 25 archers only to let them die one by one. It build civil centres randomly on the map without resources in sight and/or without any defense. Also I have the impression it tends to acquire resources just because it could and doesn't mind the cost.

What works good is the startup phase, without the need to use a mouse it builds up an army very fast. But if one survives the first 3 or 4 attacks often only a depopulated city is left, easy to take down. But I don't want to troll here, actually the AI perfectly demonstrates 0AD is an exciting game challenging the player to develop new strategies in every round and spends hours of entertaining game play.

May be it is worth to start over from the beginning. The AI should be capable to deal with minimal resources at any game phase. There is no reason for units standing around freezed. If a particular resource costs too much blood, it should be abandoned. Most of these defects are listed in this thread. What doesn't work is obvious, the challenge is to make it work.

Some AI techniques are mentioned here, judging from what I internally use I think a good AI needs all of them. Whether and how a given resource can be exploited needs planning and monitoring. If a large scale attack happens all tasks should be dropped and the behavior falls back to defense only and depending on the outcome another fallback to building up. Ultimately a hardwired AI can be probably beaten any time by a human player on the long run, so some kind of learning is required to keep the player rethinking his approach.

I tried to understand the AI sources and it seems they demonstrate very good the use of the API and was excited to see it's scripted in JavaScript. Isn't that the language most spoken among developers? Aren't there already libraries dealing with machine learning, planning, neural networks, agents behaviour? Isn't SpiderMonkey getting near native speed soon? How can all these existing resources be effectively exploited?

So this is my proposal: Set up an AI tournament for 0AD. Needed is a standard map and the game stripped down to minimal features. Only civic centers, resources and barracks. No healer, champions etc. Let developers select two AIs from the command line to watch them fighting. Include the winner into next alpha.

What do you think?

[niektb]

The idea sounds good.

I wonder though, if there are enough people willing to create one.

Still, indeed, there are a lot of possibilities for creating self-learning (and other) AI's.

Unfortunately I don't think the game is capable of running such advanced AI's. (It's already suffering from performance problems)

[RoekeloosNL]

Maybe its time too rewright the AI and get better AI and finally better preformance.

[implodedok]

Maybe its time too rewright the AI and get better AI and finally better preformance.

Great idea! So what are you waiting for?

Roll up your sleeves and get coding buddy! :-)

[dvangennip]

I guess a tournament as a way of attracting bright ideas for solving AI issues works when the problem to be solved is relatively simple. This means a reasonable amount of time invested can give some good results. In the case of 0 A.D. I don't think that putting in (let's say) 100 hours is going to give an AI capable of beating Aegis, because it's such a complex task. This is most likely the reason very few people actually work on the AI code, it just takes a lot of time to understand what's going on.

That said, simpler AI's might focus on doing one thing very well. Maybe such an AI can't beat Aegis, but against similar bots it can give a nice tournament setting. If you would do something like setting a 1 month challenge for people to hand in their bot code, it might yield some interesting results. Of course, intermediate leaderboards could be nice; pitting the various bots against each other to get the scores. It would require a relatively complete basic API for the AI's to interface with the system for getting data and pathfinding and such things (I'm not sure how this is currently split between basic AI and Aegis AI).

For example, one bot could aim to win by simply rushing with starter resources. It might give a win every 4/10 times, which perhaps is as good a result as another more complex bot. That would be interesting to see. Not because of Aegis crushing performance (I'd be surprised), but it might yield some insight in the various ways to create a bot for 0 A.D.

[sanderd17]

A tournament would require we have two different AIs

Currently we only have one hard-coded AI, so no way to create a tournament. But indeed, if someone ever makes a genetic AI, then tournaments are the only way to improve it (though tournaments against humans, and with all features enabled would result in the best final AI).

[Romulus]

This is a good idea.

But I think what he actually means is, that many AI versions get coded, and then setup a tournament to constantly prove on the blunders.

I mean in reality this is exactly why civilizations have wars. Its the constant "tweaking" the bettering of technology and problems and obstacles the enemy throws at you, necessity sets in, and necessity is mother of invention, innovation and progression.

[Echelon9]

Take a look at the Berkeley Overmind AI that took out the Starcraft AI tournament a few years back.

http://arstechnica.com/gaming/2011/01/skynet-meets-the-swarm-how-the-berkeley-overmind-won-the-2010-starcraft-ai-competition/

What about a competition for new AI entrants to compete against current Aegis, perhaps on a limited set of features?

[wraitii]

Limited set of feature isn't good enough. All features of an AI are closely linked. You need to be able to beat it on all features.

Also I'd like to say that since I arrived here, I have seen no-one start serious work on AIs.

0 A.D. is also limiting because it requires extreme flexibility.

[Romulus]

Also I'd like to say that since I arrived here, I have seen no-one start serious work on AIs.

0 A.D. is also limiting because it requires extreme flexibility.

Funny you should say that. Because looking at your bot just seeing what's there, I'm contemplating on making my own.

I think as this is supposed to be for my mod I am trying to get going (one man so-far "me") I will make a Roman one, giving it strategies mostly adopted to Rome.

But all in all, I think the concept of a competition will only heighten emphasis on the flexibility, and sort of "forces" the sophistication factor.

[Radagast.]

Hey, a genetic algorithm is a goal, but even without it was not easy to win against Aegis on certain maps especially if you don't save the simulation (this takes the AIs out I guess). Now wraitii has written the new version and it's looking very promising. I encourage all of us to track the commits more closely by looking at all changesets (diff, the differential/gear symbol). This way, one learns about how 0A.D. works and I'm convinced this is a time saver. First, because from examples one sees typical errors that can happen (once they are corrected and you examine the changeset). Second because there is no need for posting development reviews/updates because it's obvious from the commits.

There were really plenty of other people who committed really useful features. In the long we need to add genetic evolutionary algorithms and walkable meshes, once I get around it, I will give the capturing of units (and conversion of loyality to own tribe only after some years have passed - so that caputered units still can be freed by an army). I also wish to give herding and coralling a chance because I'm into open source and an ancient historian hobbyist (while aerospace student in reality). Not sure how much time and how quickly I can dig into these ventures, because of other currently much more important (because real world) open source projects (physical machines that shall save our communities of reprap, blender, opensourceecology, .. some time and also solve some of the energy transmission problems we have in the world by allowing for Do-It-Yourself small energy plants for exchanging central by distributed power).

AIs fighting each other in alliance (2 teams, each 3 civilizations) was what I tested with the old qBot and old Aegis (from a month ago probably). The result was that one AI was completely destroyed. One other tried to help out (I think by accident) and two others were involved in heavy fighting on their frontiers. Strangely a large set of mountains seems to have saved one of the tribes. Then a dispute for the last free piece of land next to ourselves ensued. While a small army tried of mine tried to keep the fields clear because it was important land for an ally. There was so much interest in that area that several factions tried to build a civic center and one even started sending an army. Almost brought to the brink of doom, the last men were gathered to start a counterattack on the monarchy of Corinth by land and sea at the same time as a last ressort. Both armies were teared down but the Corinthians gave up in building the civic center from that point on (perhaps this was random). The interests of the other city-states still was high, then I saved the history simulation and once restarted some days later all was quiet and no more interesting actions happened (either because both AIs had issues to continue their efforts because they lost track or because one tribe (the ally) succeed in building a civic center on the last land available.

Good work nevertheless. Not only the devs, also the artists! (the last days also many new goodies and stables and new textures for corn were added!).

Edited February 11, 2014 by Hephaestion

[agentx]

To me it look like the hurdles to start with an AI are currently quite high. It seems everybody expects kind of miracles not seeing it would take full-time months to beat Aegis while starting from scratch. The idea with the tournament was to lower these hurdles as far as possible. I think one AI is enough to start, seriously what is needed to make 0AD run Aegis vs. Aegis?

From that point everybody with minimal knowledge can contribute at least by tweaking parameters. Debugging would be easier too since manual action is not required. If a game could continue from a saved mile stone automatic testing is very close. There are many other AI tournaments, every single one appears on the Hacker News front page. 0AD has on the plus side a pleasing visual environment. I can't think of a better client to demonstrate a capable JavaScript AI.

What are you afraid of? That military steps in and offers funding?

[wraitii]

The thing is that running Aegis vs Aegis isn't that interesting. I mean I do it when I develop, my Aegis vs SVN aegis, but it's mostly to check for regressions: it's pretty easy to know how Aegis will react to any particular situation since it's all hardcoded.

I personally don't have the knowledge to make an efficient enough genetic AI, and I assume it's incredibly complicated. I won't try that, but I believe in the hardcoded approach enough if we make it clever.

I agree that the AI behaves a lot better in the early-game than in the later stages, this is something I intend to work on, but it'll be a slow progress, because I can't give the AI 100% of my time (both in practical terms and "thinking" terms, I just get bored sometimes).

I agree that trying the AI with several different settings could work, but ultimately right now it's all subject to changing because of actual gameplay changes, so that's why I don't really think it's a good idea right now.

[sanderd17]

I believe in a more genetic approach. But for that, you first need a hard-coded example that works good. Then you can turn more and more parameters (attack times, male/female ratios, building orders ...) into genetic ones.

Until the point that the AI doesn't know about everything in game (it doesn't know about ships, healing, garrisoning to fire arrows, trading, ...), there's no point to make it genetic, as it's all connected (if you use one resource to create a unit, you won't use it to create a healer or a trader).

[agentx]

> seriously what is needed to make 0AD run Aegis vs. Aegis?

Solved that with:

"D:\Games\0 A.D. alpha\binaries\system\pyrogenesis.exe" -autostart=Aitest01 -autostart-ai=1:aegis -autostart-ai=2:aegis

and a simple map:

The confrontation was short, four units died, two females got stuck after building a storehouse. I think I invest some weekends to see what is needed to watch an epic battle on a map like this (without stone and metal).

A genetic AI would require a complete abstraction. Every available action needs to be referable as a string, so new generations could be mixed and mutations have a data structure to work on. That's probably the far end of the road. Also two good gen sets do not necessarily have clever offspring. Somehow a fit function has to decide off game whether there is progress at all. On the other hand eventually very optimized AIs would be generated - kinda AI mining, though.

[wraitii]

you can run that on any map, including random maps. Here's the script I'm currently using:

./pyrogenesis -quickstart -autostart="unknown_land" -autostart-ai=1:aegis -autostart-ai=2:aegis-autostart-random=25255 -autostart-civ=1:gaul -autostart-civ=2:rome

You can change the civs, the random seed, the map, and even the Ai difficulty (using -autostart-aidiff=$PlayerID:$DifficultyLevel)

[Teiresias]

@agentx:
AIs succeeding in an AI tournament are not necessarily the ones most funny to play (against as a human). IIRC, on the heavengames AI tournaments of AoK, the finalist "Missionary monkey" could win by monk spamming. (The monk is a unit which could convert an enemy entity to your side.) Very effective but 'lame' from a human point of view. Others used a lawnmower tactique, with the winner being the one with better startup/resource supply.

Second, during my AI experiments (AoK, AoM, 0aD) i got the impression many problems of a good AI are computationally hard: The planning part for example, should consider whether some mine site is "hardly accessible" due to natural obstructions and should care to not place a house right in front of it - i had this happen to my test AI on the Acropolis_05 map. Similar problems arise e.g. for towers - in the end, you'd need a kind of "automatic building department" which pre-calculates and updates a city layout plan as the map changes, taking into account future entity walking routes.

The ideas about machine learning seem helpful to me in another way: Afaik, for a lot of game-AI problems no 'intelligent' algorithms are known at all, e.g. to decide where to attack enemy territory, when to retreat etc. Additionally, such algorithms had to be realtime-compatible (big problem!). Laconia comes to mind.

Currently, i see four practical problems with agentx proposal:

1. According to trac #2322, all AIs now live in a global compartment. So i presume an AI could cheat by patching another AI script away?
2. For machine learning, i presume AIs should have a kind of persistant storage space e.g. to save 'experience' about entities matching up each other in battles, or weights of neural networks. Such 'cookies' might lead to security problems (privacy attack)
3. As far as i got from IRC logs, the AI interface is considered non-stable and might be redesigned completely should performance problems demand so. Some people even suggested rewriting everything in C++, or to a production-system as in AoK. If this is done, it might render all efforts put into Js AIs void.
4. Consider wraitii in an elder discussion : "1 working bot and one sub-par experimental bot are not interesting compared to having one really good bot, I believe.". (Having problems adjusting the link properly, it is in one of the last posts of the thread).

@wraitii:
Also I'd like to say that since I arrived here, I have seen no-one start serious work on AIs.

I'm crafting an AI privately, but at the moment it's more of a collection of tools than a fit, so i didn't want to bother anyone. In case of interest, drop me a PM (i am offline most of the time).

@all:

I had similar thoughts when asking about the purpose of the AIs. Any comments still welcome.

Offtopic:

Personally, i like the idea of AI tournaments. Maybe it could be driven even further by providing a 'viral function' where an AI in good condition could split into two independant players - don't know whether this is possible in the pyrogenesis engine. Something similar to bacteria splitting.

[wraitii]

@agentx:

AIs succeeding in an AI tournament are not necessarily the ones most funny to play (against as a human). IIRC, on the heavengames AI tournaments of AoK, the finalist "Missionary monkey" could win by monk spamming. (The monk is a unit which could convert an enemy entity to your side.) Very effective but 'lame' from a human point of view. Others used a lawnmower tactique, with the winner being the one with better startup/resource supply.

Second, during my AI experiments (AoK, AoM, 0aD) i got the impression many problems of a good AI are computationally hard: The planning part for example, should consider whether some mine site is "hardly accessible" due to natural obstructions and should care to not place a house right in front of it - i had this happen to my test AI on the Acropolis_05 map. Similar problems arise e.g. for towers - in the end, you'd need a kind of "automatic building department" which pre-calculates and updates a city layout plan as the map changes, taking into account future entity walking routes.

The ideas about machine learning seem helpful to me in another way: Afaik, for a lot of game-AI problems no 'intelligent' algorithms are known at all, e.g. to decide where to attack enemy territory, when to retreat etc. Additionally, such algorithms had to be realtime-compatible (big problem!). Laconia comes to mind.

All very good points. I want to stress the idea that it's both computationally hard and needs to be real-time: it's not really that hard to make a really good city planner. Making it work real-time is a lot harder. And this goes for most AI issues.

I vaguely remember about your AI efforts, I'm not really interested unless you feel like there's something interesting, in which case you'd probably show it anyhow

[Radagast.]

Always interested in your efforts ... I believe if you build a complete universe but noone is there to discuss, show and improve it .. then it's all senseless!!

I'm quite happy with your efforts and I think to keep the parameters low is one way to get towards a dynamic, and later on more easily to a more genetic algorithm.

[FeXoR]

wraitii:

Maybe, just maybe, the reason for no other AI development takes place is because the "API" does not support AIs in general but more Aegis or at least an AI designed quite like Aegis. About 100 lines of code are needed to use the API for a bot that does not throw errors and otherwise does... nothing.

This hardly seams like an API to me at all...

I didn't playtest the actual Aegis (since I had some time again there where the serialization issues) but I believe it's quite good so that's not my point. It's only about the API that is hard to understand and inflexible AFAIK.

I'm still interested in designing a different AI and really think more then one AI is a good idea.

AI tournaments (even with only Aegis around) could be used for balance testing (if Aegis can be easily configured so different types of units are used).

Edited February 12, 2014 by FeXoR

[niektb]

We should try and recreate the functionality from ORTS. This game engine is known for its flexibility and is used for AI research and is hack-free.

https://skatgame.net/mburo/orts/

Edited February 12, 2014 by niektb

[agentx]

> According to trac #2322, all AIs now live in a global compartment. So i presume an AI could cheat by patching another AI script away?

That's a show stopper. Is it true AIs can change each other code? Or even worse, can manipulate/destroy arbitrary game objects (store-builders, etc)?

[Radagast.]

I think we can easily solve be by adjusting the write access to certain folders and then wrap a rewrite of AI algorithm genetic files into try {} and catch nothing so to not make the failed write attempt to a game building or such visible.

This way the AIs can only change its own files. But there will be another easy solution sure that I overlooked.

For the genetic algorithm I think we can have a hybrid Aegis. Attacking and Defending will be easily more generic by storing the outcome of attacks in form of:

• How many units lost?
• How much ground won/lost? If lost, then next time send more troops for defence.
• How long did it take?

This way it is automatically ensured that the AI will not try very often to attack e.g. uphill because of high losses (once we have the slow-down of uphill attacking units and stamina) ..

Looks like then the AI automatically learns military tactic by trial and error. The same can be applied for gathering from resources: Resources where losses are high (on its way to or while working there ...) will be first tried to be freed by military action - if the losses are high there too and not much ground was won, then the losses of the dropsite will still be high and finally be abandoned (and marked as unaccessible/dangerous in a list of dropsites attached to the civic center or faction.

Thanks for the links. Didn't know there exist many different strategies for skat ...

[Teiresias]

wraitii:
No offense taken, in fact i believe it is better to work independantly (and i do not look at the aegis code - on purpose), as this way creativity is not hurt by an attempt to go the other guy's route.
However, i'm not sure it is simple to build a "really good city planner". When i started my economy project, i found it particular hard to define what "really good" is, in particular on edge cases. The same goes for military defense: When i look at a map, i can tell immediately how i'll set up my defense, but i cannot tell how i made my mind.

Hephaestion:
Currently, i try to get a better economy handling on "nasty maps" in particular (e.g. the Acropolis style), but progress is very slow as i can commit about 1h/day max. to 0AD. A bit more on weekends, but still...

Regarding the "showstopper", what i mean is the AIs live in one common JavaScript object while 0AD is running and one could patch the code of another one in memory using regular JS techniques. I might try and write a proof-of-concept on such attack depending on the result of this thread and the actual SpiderMonkey layout. Write-protecting files won't help preventing that. Currently, AFAIK AI scripts cannot access the hard disk anyway.

Regarding genetic algorithms or neural nets, i'm afraid they'll take a non-acceptable training and computing time. At the current state of the AI API interface (which basically exposes just the units and the territory map), it's likely to die from combinatory explosion.

[sanderd17]

Regarding the "showstopper", what i mean is the AIs live in one common JavaScript object while 0AD is running and one could patch the code of another one in memory using regular JS techniques. I might try and write a proof-of-concept on such attack depending on the result of this thread and the actual SpiderMonkey layout. Write-protecting files won't help preventing that. Currently, AFAIK AI scripts cannot access the hard disk anyway.

And what's the problem with it? There are so many ways to cheat in a single-player game. Why would we have to limit cheating by the AI player?

As long as we don't include cheating AIs in the main game, there's no problem. A single player can't patch his AI to cheat, as the AI computations are double-checked with the other player.