Further AI development

[wraitii]

I'm opening a new topic since the API version 3 has now been committed. Kind of a conclusion on the past 2 weeks' work before I head back to school, and probably abandon developing 0 A.D. for a good 2/3 months. Warning: this is a huge wall of text, mainly intended for my own reading later, so that I may pick up where I left off without too much trouble .

So basically I'm fairly happy of how much progress has been made on Aegis in these 2 weeks of holidays I got (on top of the work I did back then). Defense has been fixed to a fairly great extent, the attack has improved, the economy has learned how to deal with technology and phases without really losing much in terms of efficiency (there's room for improvement, though), and generally everything has been made more robust than it was before.

There is however still a lot to do, as most players and myself can already tell. I would like it if we could release a "Hard" AI that did not cheat. Currently hard mode cheats by gathering 33% faster (and very hard 66%). In order to do this, many things to consider:

First, the AI build order needs to be perfected. I've watched replays of mainly Quantumstate and Alpha123, but there is still work to be done, everything is not perfect. Furthermore the economy manager still isn't completely rational in the way it assigns workers, which makes the AI lose a few precious seconds now and the, but this is particularly problematic in the early game. Also, it doesn't trade.

Secondly, the AI is still very hardcoded. I've switched most of the "time" checks to "number of workers" checks but this is far from enough. The AI is generally hardly aware of itself when it takes decision, thus leading to average decisions, and mostly to a lot of trouble with unexpected cases. In particular, raiding the AI early currently tends to cripple it, as it does not recover well enough, trying to start attacks instead of focusing on rebuilding an economy, things like that.

Thirdly, building placement in general needs to be improved. the code for dropsite placement is in itself fairly efficient, and returns generally pretty good results. However, the code that decides when to build a dropsite is extremely inefficient. Placement of most other buildings is basically random with a few tweaks and doesn't really follow any particular logic, when it probably should.

Third big issue: the defense system is still quite flawed, both from the "no sense of self" effect of the AI, which will make it defend useless things, defend in obviously losing battles, or not defend enough when it ought to, and the many flaws in the logic. It's basically very "instant" defense, it doesn't really consider what happened in the past or what might happen. Case in point: a tower on the border near an AI dropsite will basically make it get all its workers killed as it doesn't really notice.

Fourth big issue(s): attack and unpredictability. The first subpoint is that the AI sucks at attacking. Basically it makes relatively balanced armies, but it sends them to the enemy and doesn't really consider how to use these units the most effectively, making its attack mostly useless against a skilled (micromanaging) player.

Secondly, the AI is extremely predictable. It has only one real "strategy", if you can call it so, which is also extremely hardcoded and not at all enemy dependent. Work on that is less of a priority, of course. The AI should probably handle the above points first, and then we will be able to consider making it actually do interesting things.

Finally, I'll post again the "to-do" list I had posted on the AI API version 3 topic back in early november, as most of the points are still valid

-No real naval support. It can handle attack over water, in a basic way, but that's it (Update: that actually might have become slightly buggy)

-No handling of multiple-island realms. For Aegis, its starting island is all there is on the map.

-No handling of the cases where the AI runs out of resources to gather. It will start lagging and clogging the queue manager. Linked with the above points.

-The defense system is really basic. It can't handle continuous frontier skirmishes, placement of defense buildings is not logical, no wall building, it's not reacting perfectly to attacks either. No sense of "this is a dangerous area, I should avoid going/collecting/building there". No support for building units that counter the enemy favorite units. So many other things...

-No diplomacy support (see Fexor's post in the general development forum)

-Economy is still perfectible: drop sites placement and building could be optimized, as could be building construction, or even Build Orders in general. There are some slightly insane behaviours here and then. Generally this is probably the area where the AIs hold up the best, particularly Aegis. (Update: see above.)

-Very partial technology support. Aegis goes up in phases, but it's not flexible. The API support for tech may not be perfect either. (Update: actually, it now researches all techs, but chooses them randomly. So work needs to be done on that).

-Partial attack system. It's OK at strategy, but bad at tactics and will generally attack inefficiently (though Aegis is better than qBot at that).

-no LOS/scouting support. AIs see everything. (Update: this is actually linked to making the AI react naturally: even when cheating, it should be aware of its enemies and take decisions based on them)

-optimization is far from perfect, though it's slowly getting better. Some possible improvements: hierarchical entity collection filtering, getting the AIs not to iterate over all units in a single turn once in a while but balance the load... (Update: probably better now, but there is still ton of room for improvements).

A last word about something that is still bothering me: the queue manager. Currently it gives "accounts" to each queue which grow over time as the AI collects resources. It works fairly well, but it has a tendency to make the AI sit on resources, particularly when the gathering rates are low and the needs high. I think going for the most human approach here is what would work best, but I must say I actually have little idea of what the "human" approach is.

Basically we might need the rest of the AIs to register what they will probably want to train/build in the next minutes for the future needs to be fairly stable (which will also help econ…). Then we need the AI to actually pass stuff, making sure it doesn't sit on tons of food just because the next item needs a little wood, and to do it with the maximal efficiency. I am not sure what the best system for that is, but I think the current queue system isn't quite perfect.

Edit: moved to general forum.

[Pureon]

Thanks for the write-up wraitii

[wraitii]

I just moved the topic from the dev forums to the public forum (removed posts about that too for readability). Copied, in spoilers, are 3 of my posts originally in the dev forums about some more specific AI stuffs. Again, I stress that this is mostly informative.

Defense:

Ideally, AI defensive capabilities should be able to meet these 3 big criteria:

-Handling and awareness of territories and their borders

-Understanding of the global dynamics of the game: who's strong, who's where, their strengths and weaknesses, dynamically.

-Understanding of what the AI territory is made of: important zones, important buildings, what to defend and why.

Now to handle this, there are some specific points that should be done. First, the AI must make proper use of its manpower. It has citizen soldiers, units being created for attacks, female villagers. The third category is fodder for the enemy and should be kept away from fighting. The second could be used for fighting, but might also be kept for starting a counter-attack. The first are then the obvious defenders.

How to pick those defenders? The AI must have proper threat recognition. I can see a few different types of threats:

-land threats. Armies. There are some subcategories: the lonely scouting unit, the small army, the raiding party, the big army.

-naval threats. Ships, of course. Those should probably be left to a "naval manager" that would commission the ships and use them against the offender(s)

-building threats. Those are specific because they are static. They're things like a tower or a fortress attacking passing units or endangering an area. They also can't be dealt with with 5 unprepared soldiers.

Understanding what a threat is is important to deal against it. However, it's not all there is to it. I'm going to introduce here something that most players do without even thinking of it: a "Danger map".

The map, particularly with the territories, possible narrow corridors and chokepoints, sea areas and so on is very heterogeneous in term of security: some areas will be controlled by enemies, some areas will often be travelled by enemy troops, some areas are in range of towers or fortresses. The human player avoids these areas when sending its troops to collect resources, when pathing, or generally when doing anything but attacking.

To simulate this, I believe AIs should have a "Danger map". For every attack or enemy unit detected, it would raise the "danger level" at this location. Any tower, and the danger level would raise too. Naval danger would do the same. A big attack would make the area notably more dangerous.

This would have a simple effect: the AI could know both where to avoid gathering, and where to defend more efficiently. It would perhaps build towers in notoriously dangerous areas, to reduce the danger level, or put some patrols. Perhaps build walls if the situation calls for it.

With this system, the AI could both respond to threats, and/or register them for later knowledge. An isolated tower that watches a forest would be properly handled (perhaps, if the forest is vital, by sending a catapult to take it down, or, if not, simply by not gathering there).

This could be combined with a "base" system, where the AI keeps tracks of its many bases. This would be handled generally in the economic and military managers, and would also serve other purposes such as multi-island support. It would help the AI react sanely, by sending to defend only units form the proper base, or by shifting defense means from bases to bases.

Moving on, the AI right now has a major flaw: it can't handle skirmish. Units will chase enemies across the map, and the AI will continuously send troops to deal with enemies crossing the border in a zone where the borders are next to each other. It wastes resources, units, and isn't efficient. AIs need to handle territories differently, with towers, standing units, and perhaps walls. It also needs to stop sending units there.

Thirdly, and more generally, the AI needs to answer properly against different land threats. Buildings like towers should either be left alone or be destroyed by means of big rocks, that is catapults. Lonely units should be tackled by fast-moving cavalry units or by nearby land units in general. Raiding parties must be dealt with accordingly: foot soldiers won't be able to catch cavalry units. The AI should probably garrison females in the path of the army (and for advanced defense, might try to sacrifice ½ units to lure the enemy raid into the defending army). In case of big armies, there is a general need to answer with both appropriate units and the appropriate units. There are basically 2 possible answers: either the enemies are too numerous and the AI must focus all its units to tackle the attack (and if there are multiple simultaneous attacks, might want to shift the balance of defenders to guarantee a minimal loss), or the army is big but manageable, in which case the AI might want to keep its attacking units aside and launch a counter-attack right after, or even start a diversion that would make the enemy attack less efficient (assuming human). I'd add that the AI might want to keep statistics about the enemy: preferred units, or paths of attacks, or something are worth considering.

With such improvements, I believe the AI would already have a pretty comprehensive and adaptable defense system, ready to defend against most threats. However, there are unexpected cases that might happen and that should probably be taken care of.

First, the nightmarish "one unit raid". Basically, a small group of unit that regularly attacks a given area, or harasses the AI's position in a given area. A right combination of danger map + setting towers might solve the issue mostly, but the AI may want to recognize this area as a particular area where garrisoning is probably not efficient. The loss in resource gathering is not worth the possible loss of a female or a soldier (though if there are soldiers locally, they might suffice to defend said position).

Secondly, there is the "odd unit" case. A lonely catapult, a simple monk, whatever. The AI should deal appropriately and deal with the unit, but it may want to adopt a different attitude and try to convert it or something.

Finally, I think the defense manager should be adapted to deal with some of the possible attempt at gaming it by the human. Such things as sending a group of female gatherers in the enemy territory: they would currently not be detected as they are not dangerous. However, this is obviously not acceptable. The human might also try gaming the scouting by sendings merchants, sheep, etc.

I'm saying this as the defense manager could be extended to extend some "strategic" variables, such as "what of me is known?". In case the AI discovers a scout, it might set a "the position of my CC is known" flag or something. Such information could probably be helpful in a variety of cases.

Given this, the AI would probably have an acceptable defense system. The danger map is the core idea. The rest are improvements which haven't been added yet.

Multi-base support:

I'll begin by clarifying what I mean by that, as I believe it's not obvious. The AI currently has a very approximative understanding of the map: I've given it an "accessibility" map which separates each individual blocks of land (two points are in the same area if they can be reached by foot). The AI is assigned an "index", corresponding to the area in which its initial CC is. Anything in the same area is fair game, anything outside is inaccessible.

This is flawed in more than one way. First, the AI may not have a CC at the beginning. The AI might have two. Worse, the AI might have a CC and its units might be on another island entirely. Or the initial area is not sufficient (see the Migration map). There are far too many odd cases for this to be sufficient.

Simply improving the logic for assigning the "index" is not enough. The AI must be able to deal with having different economic or military bases over multiple islands.

The logic extends to big land-only maps, such as Oasis. Arguably, some resources are far enough from the CC that you might want to see them as belonging to another "island".

This should be handled, and is what I define as multiple base support.

The idea is simple enough, and once again is an imitation of basic player behavior (though players tend not to take it to eleven, which the AI could.) On an island, the player will build dropsites or military buildings, or even CCs, that will be a new base. It will avoid transferring units from a base to another unless it's necessary, as it's not practical. It won't defend with units from both islands. Though they are not unconnected, these two islands/bases are seen as widely independent. In a big land map, there might be a few zones where the player started building farms or military buildings. Though the logic is less important here, the human player will avoid switching units between bases, as this is expensive time-wise, unless it's absolutely necessary.

The AI currently doesn't do it. It should. I think to be done properly, the AI needs to see bases as having one or two functions:

-economic activity (extendable to training if there is a CC)

-military outpost/base: training of units.

(One could imagine more or subdivide in "cavalry base/infantry base/siege base" or anything, I guess).

Obviously, the initial base/island would have these two functions. Then later on, the AI might build military outposts fitting the second function, or economic areas fitting the first, or even both. This is obviously easy for islands: the AI starts a new base on a different island, and might transfer or give it some functions.

A base should be the "primary" one, and would be the most strongly defended. This could shift, on maps such as migrations. Bases could lose some roles.

On island maps, the split between bases is obvious: a different island. Units would avoid being traded and would think first in their own base, and if the task can't be achievd would try switching to another base. The transport would be dealt with by a "naval manager", which I'll deal with anytime.

On land maps, the split is harder to define, but a sufficient distance would probably do. Each CC might also be a base for a simple interpretation.

All buildings should belong to base, even dropsites, so units will know how to deal in their own base.

Given the current architecture of the military manager and the economic manager, it might be worth it to have a manager per base instead, and units would switch between bases and be dealt with by different managers.

Attack plan and defense should remain global. The second because it is more logical, though there is a need for the defense manager to be aware of bases. The first because it simply makes more sense: you might start an attack plan from a military outpost, but try sending siege units recruited in another base. Furthermore, given that they are planning for attack, they are not really AI-centric but rather enemy-centric.

Basically, this is very simple logic that would efficiently allow the AI to handle water maps. On migration, the AI would detect that the starting area is insufficient, and would focus on looking for another, better, area, where it would shift all functions. The AI might build outposts. Things like that. Simple enough, but likely necessary.

Naval Manager

I've tackled how to deal with multiple islands before, so this will focus on the actual management of the navy. A complicated point indeed. I'm going to make a case for a separate naval manager, separate from the economic or military manager. The reason for this being that stuffs over water are largely independent from stuffs not over water, and that in case of convergence, calling the naval manager to do some specific stuffs seems to me like the most simple way to go. Furthermore, naval logic might be pretty different from land logic in the future (with ramming, realistic ship movements and so on), so it seems like keeping it separate is good form.

Now, let's first start by describing what the naval manager would have to achieve:

-trade over water. This is a pretty specific function, that should probably be handled by the economic manager. However, the naval manager would train and maintain the fleets, with the economic manager telling the naval manager how many to keep.

-defense over water. I believe in this case the defense manager would send a list of target or an area to protect/patrol/whatever, and the naval manager would commission enough ships and handle the task.

-Attack over water. This overlaps the former in term of "naval supremacy", but also means (safe) transport or units over water.

Now, the later is problematic for one reason: attack plans are currently dealing with transport over sea, and I doubt letting the naval manager have the control is a good solution. However, the attack plan could commission transport ships and could ask the naval manager to defend them.

Thus the naval manager would hardly do anything on its own. It would protect trade lanes on its own, assure naval supremacy in some cases, but mostly the other manager would ask for:

-patrol, defense, killing some specific units.

-defending existing ships

-attacking or "securing" a given area.

It would also handle maintaining the fleets.

[scroogie]

Very interesting read. Its great to gain some insight like this!

Its already enjoyable to play against Aegis, although I noticed some glitches when I tried it yesterday. I played on Sahel Watering Holes and at some time the AI, apparently undaunted by death, intended to build a CC in one tight corner that was left neutral territory. Of course I had some defense at the entrance to my island, but it kept trying to squeeze through with a horde of workers and civil soldiers. So I guess the point "No sense of "this is a dangerous area, I should avoid going/collecting/building there"." is indeed quite important. Looking forward to even more great improvements!

[Lion.Kanzen]

Great Work now, its more a challenge play against AI. and more variety. you make base of Developing of a great AI.

[sanderd17]

the 'dangerous building site' problem is indded a very obvious one, but also the 'dangerous path to a safe location' is unknown. Sometimes, a location can be reached via 2 paths, and the AI makes the wrong choice.

Maybe some other thing, do you plan to make the easy modes more fun? Easy modes should also raid a bit more at the start of the game.

[niektb]

Hi everyone,

My name is Niek, I'm 17 years old and I'm planning to join the programmers here.

I decided to start with the AI, because there aren't many of them and because I already know JS, but I can't find any documentation about the development of AI's with 0 A.D.

Can anyone provide some help?

P.S.:

Is this the right place to ask this sort of questions?

Edited July 22, 2013 by niektb

[wraitii]

Hi Niek,

It's good to see new developers interested in the project. AI development is actually very undocumented, as nobody took the time to document it properly (in part because it changes a lot, and fairly often). There are a few things on the Wiki though here.

As lead AI developer (de facto), I'm working on Aegis (in the AI file, it's currently qbot-wc). I'm currently rewriting a lot of code on that too, so it's probably best for you not to start with that bot. If you want to get used to the code, check out how qBot works (in the AI files, it's the "qbot" folder, which includes the "common-api-v2" folder).

If you have any questions, PM me or try to catch me on IRC (#0ad-dev)

(it's not out of place, and there's no real place to ask this unless you want to post a proper application, in the dedicated subforum).

[Josh]

Only one thing I wanted to add. I've recently watched some 0AD gameplay videos, and on easy the AI still can be way too hard for beginning players. It might be nice to add an even easier "sandbox" level for those beginning players and maybe even replace some of the extra "easy" bots with the new "sandbox" difficulty on Aegis.

[Nolanjoker]

for a sandbox gameplay, the Ai can attack with small group of soldiers. and they never do military upgrades. and hsve s small economy.

[Pureon]

Only one thing I wanted to add. I've recently watched some 0AD gameplay videos, and on easy the AI still can be way too hard for beginning players. It might be nice to add an even easier "sandbox" level for those beginning players and maybe even replace some of the extra "easy" bots with the new "sandbox" difficulty on Aegis.

This has been discussed within the team and is something that we (thanks wraitii ) will work on. We want to ensure everyone gets to enjoy the game no matter the experience they have with RTS games.

[infyquest]

We need to work on a sea faring AI, before we can embark on this new quest.

[Nolanjoker]

We need to work on a sea faring AI, before we can embark on this new quest.

good point, put we must wait the pathfinder first.

[niektb]

If you have any questions, PM me or try to catch me on IRC (#0ad-dev)

How do I post a PM here at this forum?

Edit: I'm going to start a thread.

Edited July 23, 2013 by niektb

[greenknight32]

PM is very simple, click on the avatar of the member you want to PM to open their profile, click the "Send me a message" button there.

[Pureon]

How do I post a PM here at this forum?

Edit: I'm going to start a thread.

You could have just asked questions here to keep current AI discussion in one place...

[wraitii]

Okay, I'm posting the files to the upcoming version of Aegis for debugging. Please note that this will probably not work fully like expected, and please report any weird behavior and any errors.

I have not yet put back some optimizations I did for the earlier Aegis, so it'll probably be (slightly) slower too.

You will find it under Aegis (wc) in the game menu.

It's a bit of a hassle to install, sorry about that.

Unzip "simulation/components" in the "binaries/data/mods/public/simulation/components" folder.

Unzip "simulation/components/interface" in the "binaries/data/mods/public/simulation/components/interface" folder.

Unzip aegis.zip and common-api-v3.zip in "binaries/data/mods/public/simulation/ai"

Unzip ccmpAIManager.cpp in "source/simulation2/components". It's needed if you want to activate debug mode (which is currently on by default). You will need to recompile.

aegis.zip

common-api-v3.zip

simulation:components.zip

Simulation:components:interface.zip

CCmpAIManager.cpp.zip

[sanderd17]

It's a bit of a hassle to install, sorry about that.

You have something against diffs?

[wraitii]

Yeah, for some reason I decided against that because there is a whole folder that's new.

I'll post a diff later today though, because it's not logical thinking.

edit: done.

edit2: actually I'll crash if you finish any foundation, I'll post a fixed diff later.

edit3: should work now. At least that is fixed.

AI.patch

[infyquest]

any changelog for thispatch, so people can test for specific changes

[wraitii]

No real changelog, I've been rewriting a lot of stuffs so I just need to check if anything breaks. The AI is likely less capable than current Aegis at the moment, but that's to be fixed soon.

If you need to focus, I'd focus on attacking the AI and see what it does.

Edit: okay new version, debug is now off by default. AI learned how to hunt again, fixed some bugs, added some optimizations, and changed a few other stuffs.

I'm not sure if the AI is currently strongest on medium, hard or very hard difficulty.

Please update to latest SVN, I already committed some non-strictly-AI changes, the patch will only affect the AI folder.

edit2: of course the patch didn't apply… Fixed.

edit3: Posted the two folders as an archive for the patch-deficients like me. Put the two folders in public/simulation/AI (erasing the current ones)

AI.patch

files.zip

[gameboy]

Error report:

<p class="warning">WARNING: JavaScript warning: simulation/ai/common-api-v3/terrain-analysis-pathfinder.js line 18

reference to undefined property gameState.ai.terrainAnalyzer.map</p>

<p class="error">ERROR: JavaScript error: simulation/ai/common-api-v3/terrain-analysis.js line 120

TypeError: this.map is undefined

([object Array],true,500,false)@simulation/ai/common-api-v3/terrain-analysis.js:120

([object Array],[object Array],2,2)@simulation/ai/common-api-v3/terrain-analysis-pathfinder.js:90

([object Object],[object Object])@simulation/ai/qbot-wc/aegis.js:257</p>

<p class="error">ERROR: AI script Deserialize call failed</p>

<p class="warning">WARNING: JavaScript warning: simulation/ai/common-api-v3/terrain-analysis-pathfinder.js line 18

reference to undefined property gameState.ai.terrainAnalyzer.map</p>

<p class="error">ERROR: JavaScript error: simulation/ai/common-api-v3/terrain-analysis.js line 120

TypeError: this.map is undefined

([object Array],true,500,false)@simulation/ai/common-api-v3/terrain-analysis.js:120

([object Array],[object Array],2,2)@simulation/ai/common-api-v3/terrain-analysis-pathfinder.js:90

([object Object],[object Object])@simulation/ai/qbot-wc/aegis.js:257</p>

<p class="error">ERROR: AI script Deserialize call failed</p>

[niektb]

Hi guys,

I've just returned from my holiday and I've got a bunch of idea's for my upcoming AI.

I decided to name it Brennus, derivated from the Galic leader who attacked Rome and almost conquered it.

It relies heavily on scouting, so scouting is the key. Futhermore I'm going to make a dynamic strategic-tree.

Using the information from the scouts, it detects key information about strength, troop types and so on, and then the AI determines (using the tree) the best way to behave.

I decided to use APIv2 at startup, and later on I'll upgrade. (as soon as v3 is really, really stable)

Roadmap:

- Startup:

First I want an AI that works in every aspect of the game. This one won't be released.

- Scouting programming

First the basic scouting routine.

Second the ability to detect information about the enemy.

- Tree design startup

I start with some simple decisions and then work to advanced strategic battle-level.

- Micromanagment

After I've set up a simple AI, I'm going to implement all buildable / trainable stuff, getting a defence set up and some extra tree-decisions concerning balance of units.

- Advanced functionality

When all this works, I'm going to implement advanced functionality like all civs, more than 2 players, naval support and (vs human) psychological warfare based on Sun Tzu's

Art of War

.

- Bugfixing.

Note: This list is of course subject to changes.

I would love to get some tips / suggestions / something else. If someone wants to coöperate just PM me. (I could use some help to finish it before alpha ∞. )

Greets,

Niek

Edited August 12, 2013 by niektb

[gameboy]

@wraitii:On this issue, do you have any way to solve it anyway?

[niektb]

@gameboy: Where did you get the errorlog?