Public Domain Dataset for Machine Learning

[irishninja]

I think it would be really cool if we could start creating a public dataset of replays for a standard scenario (especially in light of recent things like AlphaStar). Although I am most interested in creating the dataset for training ML agents (such as imitation learning - maybe to initialize a reinforcement learned agent), it could also be cool for comparing player strategies using a common baseline, or for potentially find shortcomings with the current AI. 

I think the dataset should be public domain to keep it open for anyone to use for fun, publication, etc. One natural scenario to start with would be the Acropolis Bay 2 map where both the human player and the AI are Spartans. The AI difficulty can be set to whatever people want but stronger AI is more impressive, of course 

If you have a replay you would like to donate to the dataset (using the specific scenario mentioned above), feel free to post it here! By donating the replay, you are giving consent for the data to be made public domain and, therefore, able to be freely used for research, fun, and the like!

Update:

Rather than Acropolis Bay, the Mainland map is preferred (using the small map size) using the same civilization and players listed above. However, any and all replays are welcome and appreciated!

Edited April 5, 2019 by irishninja

[sarcoma]

Not many play that map, better use best 0ad map, mainland.

[irishninja]

Yeah,  we could make it a more popular map since we haven't collected any replays yet - we certainly won't want to change the map after replays have been posted. However, I am concerned about mainland since it appears to include some procedural generation (making the learning problem harder as it would need to generalize to the different variations produced by the procedural generation). It would be nice to start with a fixed (skirmish) map to make the learning problem easier.

Is there a more popular (2 player) skirmish map that might be a better fit? 

Edited April 4, 2019 by irishninja

[nani]

@irishninja I would still recommend using mainland as sarcoma said for multiple reasons:

• Mainland is, by far, the most played map (more replays)
• Mainland is, as you say, a random generated map so it would help to not overfit the ML solutions
• Mainland is usually played by very competitive players, so the data would be more meaningful (no offense nub players)

Edited April 4, 2019 by nani

[irishninja]

That makes sense. What size map would be best for this 1v1 scenario? Medium or small?

How do people feel about controlling the seed to the PRNG so we get the same map? My concern is with the number of replays required before being able to learn something meaningful. Although it would be great to generalize, this would likely increase the data requirements a bit whereas it might be an easier starting point with a fixed map. However, the downside is that this might be annoying and could limit the number of replays...

Thoughts?

Edited April 4, 2019 by irishninja

[Nescio]

If you increase the map size for each additional player (i.e. tiny for two, medium for four, giant for eight), then the average number of tiles per player will be constant (64).

What exactly are you trying to find out? Did you consider running AI vs AI games?

[Feldfeld]

I think the problem with AI vs AI is that AI always do about the same thing, so the algorithm wouldn't learn strategies to win, it would just learn to imitate the AI i think.
If that's of interest I have 1.7 GB of replays including 600MB on a23, most of them being played on mainland, human vs human

[Stan`]

Would be interesting to have them yes. Dunno where you could upload them though.

[Feldfeld]

After doing a parametized hard compression on 7zip it amounts to 200 MB

[Feldfeld]

http://www.mediafire.com/file/b9650400tcso7h4/replays.7z/file

It will be available for 14 days.
It could be important to write a script to remove fgod from mod list in commands.txt if all those files are used

Edited April 4, 2019 by Feldfeld

[elexis]

https://wildfiregames.com/forum/index.php?/forum/468-alpha-23/

 

[irishninja]

@elexis - Thanks for the link! I was checking those out earlier and they seem like a useful resource (I counted 46 posted there + 1 with no metadata.json file). I thought I would go ahead and make this post anyway so we could:

• make sure we had a dataset where we knew people were fine with it being used for research, publication, general merriment, etc
• try to collect a dataset with a standard configuration. It becomes a bit harder problem if we want to generalize across a bunch of different scenarios (particularly the team based scenarios - multi-agent reinforcement learning gets a bit tricky and would require a lot of examples). If we have a dataset with a large amount of data using the same configuration, then hopefully people could tackle a simplified version of the problem before tackling generalization to more complicated scenarios.

Another cool thing about facing the built in AI is that it appears that there could be some cool capabilities that could be leveraged while training. After digging into the replay files a bit, I realized that only human players' actions are recorded whereas the AI is just initialized with the same random seed then behaves deterministically. This is pretty cool since it should be possible to do things like run the replay to a specific point and then replace the human player with an experimental AI and see how it would perform (since the opposing AI could still react to it as it would if it was playing the experimental AI the whole time).

That being said, any and all data people would like to donate would be great and I think could be really useful for exploring AI/ML agents in 0 AD!

[irishninja]

How does everyone feel about changing the scenario to a 1v1 (human vs petra) on a small mainland map? Any objections?

Edited April 5, 2019 by irishninja

[jonbaer]

At one point I gather a lot of what was going on w/ Halite and Starcraft to do something RL-based in 0AD @ https://github.com/0ad4ai but the way I had it setup was to just rip apart the /public default mod down to barebones like you describe 1vs1 on a continental map and mini-game through resources (first to 100, etc.).  When I was researching Starcraft the obvious part that stood out was the strategies around dynamic build orders to match what the layout of the map was in terms of resources.  I enjoyed exploring this but did not get too far, I really like what was done w/ Hannibal but the main problem ended up being the new builds would be out of sync so a fork of the public mod was needed outside of the engine, I don't know if that is still anything remotely possible because there are way too many civs to deal with and some folks I even passed the idea along too didn't want to download the entire game.

[Feldfeld]

5 hours ago, irishninja said:

How does everyone feel about changing the scenario to a 1v1 (human vs petra) on a small mainland map? Any objections?

The advantage is that it would be more standardized but the disadvantage is that you'll find less replays than PvP. I have a decent number of those human vs petra in the folder i dumped but most of them are in previous alphas i think, and also many of those were tests and not actual games so be careful.

[thankforpie]

19 hours ago, nani said:

• Mainland is usually played by very competitive players, so the data would be more meaningful (no offense nub players)

imagine roasting yourself in your own posts

Edited April 5, 2019 by thankforpie

[irishninja]

@jonbaer Nice collection of RL work related to RTS games! Did you get a chance to start applying any of these approaches to 0 AD? I have seen that there have been a few posts about using python to implement an AI as well as discussion about OpenAI Gym integration. I think integration with gym would be pretty cool.

@Feldfeld Good point. I think I will add an edit to the main post to ask for any replays people would like to donate and, if they happen to have any replays on small mainland (human vs petra), it's an added bonus but not by any means required. Hopefully this way we can encourage a specific standard but not by any means be restricted to replays of that format!

[jonbaer]

A Gym/Universe setup would be great but 0ad isn't a simple Atari 2600 environment ;-) I haven't looked much @ it since Hannibal dev kinda died down but in retrospect really what I think the launcher (https://github.com/agentx-cgn/Hannibal/blob/master/launcher.py) was doing was to dump game state and let a sub python process (ML/RL) analyze and then pass it back up to one of the managers in Petra when decision making on the next action (gather/attack/defend/etc).  Really the first step is to try to strip down pyrogenesis to bare bones w/ 1 civ and 1 map somehow (and of course keep it inline w/ master branch), otherwise you are looking @ 2GB public.zip - in nutshell the eye candy graphics nothing pertaining to the bare bone of the AI planning.  At the moment it is easier to test ideas w/ something micro like, https://github.com/santiontanon/microrts and then bring it back into 0ad, probably as a mod itself.

Another interesting thread I came across: 

 

Edited April 6, 2019 by jonbaer

[jonbaer]

Just to keep this conversation going a bit, I came across this while watching the Tensorflow Dev Summit videos, https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/video/starcraft.py ... in light of what I think happens w/ SC2 AI build-ups it is more of passing minimap visual information into decision making, another more high level route.  BTW I noticed quite a few discussions in regards to forks of 0AD, something I would like to see of any forks is the possibility on making 2 different projects, one simply for the Pyrogenesis RTS engine and the other for Civs straight down the line to being loadables or mod-based.

[Stan`]

An Engine split is planned. As for splitting the civs not really possible because they are interdependent and I don't see how having 13 folders is gonna make things easier for us ^^

[fatherbushido]

2 hours ago, jonbaer said:

[...]

something I would like to see of any forks is the possibility on making 2 different projects, one simply for the Pyrogenesis RTS engine and the other for Civs straight down the line to being loadables or mod-based.

Hi!

What do you mean by that? (can you detail it a bit more?)

[elexis]

Sounds like splitting game content (tempates + map + art) from the pyrogenesis engine (code): #5366