Jump to content


Community Members
  • Content Count

  • Joined

  • Last visited

Community Reputation

7 Neutral

About irishninja

  • Rank

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Especially when using machine learning in 0 AD, it is really valuable to be able to record game states as this enables things like off-policy reinforcement learning, imitation learning, and facilitates quantitative analysis of gameplay. That said, it would be great if we could "expand" a 0 AD replay into a log of state, action pairs which could then be used for the beforementioned tasks. I am imagining a command line flag for logging state, action pairs when running an existing replay. Currently, I have implemented a primitive approach to this [here](https://github.com/brollb/0ad/pull/1) which simply prints states to stdout. Although this is not the ideal interface, it has enabled me to wrap the game with a script to reconstruct the states with the actions from the commands.txt file. If anyone is interested in trying it out, there is also a docker image for the branch on GitHub which I have used to "extract" the rich game states from a replay. I am interested in cleaning it up and adding this capability back to 0 AD but thought I would start here to get feedback and input from the community!
  2. In case anyone is following this thread, there is currently a ticket opened (and patch submitted with the update!): https://code.wildfiregames.com/D2197 and https://trac.wildfiregames.com/ticket/5565
  3. Thanks for the quick replies! @stanislas69 I have been digging into it a little bit more and it seems that the main call that fails is `getReplayMetadata` (which generates the metadata.json file saved for the replays). It seems that without the metadata.json, the replay doesn't show up in the replay list so I figured it was a file that I needed. Currently, that file will not be generated if it is running in nonvisual mode and overriding this manually results in an error since it seems the `getReplayMetadata` command can only be called from the script interface used by g_GUI (not available if running in nonvisual). Does that make sense? I actually am a fan of the way the AI is treated during replays :). Although it is different from the humans, expecting the AI to be deterministic (and controlling the random seed) is cool since we could hypothetically resume a replay at any point (if it is player vs bot) and the bot would adapt accordingly. If it was simply replaying fixed actions (and not recomputing the AI's behavior), this wouldn't be possible. This feature isn't available now (afaik) but would be cool when exploring things like imitation learning. Awesome to hear that you are open to contributions! I am interested in using 0 AD as an RL sandbox but would like to keep any fork as small as possible since maintaining forks is no fun. If possible, it would be really great if all RL experimentation related features could be added to 0 AD itself (and toggled with cli options to not change any of the current behavior, of course) . @bb_ and @stanislas69 I have actually been working on my own fork where I am exposing a GRPC interface to the game for controlling the player (actions are then logged to the replay - at least when there is a GUI). It is still a work in progress; I can currently control units and issue player commands but still need to add support for some other features like loading a scenario (maybe defined dynamically), saving replays when nonvisual, and smooth GUI movements when watching an RPC-controlled game.
  4. I am currently looking at experimenting with different AIs and would like to be able to run different experiments headlessly then be able to watch/demo selected replays. I have noticed that 0 AD currently does not support saving replays when nonvisual and it seems largely due to a reliance on GetSimulationState (from the GuiInterface). I haven't yet dug deeply enough to be able to see how difficult it would be to refactor the code so GetSimulationState does not relay on the GUI (I am a bit concerned about some of the calls to `QueryInterface` which I haven't yet dug into). Does this seem feasible? If so, I would appreciate any pointers or suggestions. Also, if I am able to make this change, would 0 AD be willing to uptake this or would it only live in my fork?
  5. @jonbaer Nice collection of RL work related to RTS games! Did you get a chance to start applying any of these approaches to 0 AD? I have seen that there have been a few posts about using python to implement an AI as well as discussion about OpenAI Gym integration. I think integration with gym would be pretty cool. @Feldfeld Good point. I think I will add an edit to the main post to ask for any replays people would like to donate and, if they happen to have any replays on small mainland (human vs petra), it's an added bonus but not by any means required. Hopefully this way we can encourage a specific standard but not by any means be restricted to replays of that format!
  6. How does everyone feel about changing the scenario to a 1v1 (human vs petra) on a small mainland map? Any objections?
  7. @elexis - Thanks for the link! I was checking those out earlier and they seem like a useful resource (I counted 46 posted there + 1 with no metadata.json file). I thought I would go ahead and make this post anyway so we could: make sure we had a dataset where we knew people were fine with it being used for research, publication, general merriment, etc try to collect a dataset with a standard configuration. It becomes a bit harder problem if we want to generalize across a bunch of different scenarios (particularly the team based scenarios - multi-agent reinforcement learning gets a bit tricky and would require a lot of examples). If we have a dataset with a large amount of data using the same configuration, then hopefully people could tackle a simplified version of the problem before tackling generalization to more complicated scenarios. Another cool thing about facing the built in AI is that it appears that there could be some cool capabilities that could be leveraged while training. After digging into the replay files a bit, I realized that only human players' actions are recorded whereas the AI is just initialized with the same random seed then behaves deterministically. This is pretty cool since it should be possible to do things like run the replay to a specific point and then replace the human player with an experimental AI and see how it would perform (since the opposing AI could still react to it as it would if it was playing the experimental AI the whole time). That being said, any and all data people would like to donate would be great and I think could be really useful for exploring AI/ML agents in 0 AD!
  8. That makes sense. What size map would be best for this 1v1 scenario? Medium or small? How do people feel about controlling the seed to the PRNG so we get the same map? My concern is with the number of replays required before being able to learn something meaningful. Although it would be great to generalize, this would likely increase the data requirements a bit whereas it might be an easier starting point with a fixed map. However, the downside is that this might be annoying and could limit the number of replays... Thoughts?
  9. Yeah, we could make it a more popular map since we haven't collected any replays yet - we certainly won't want to change the map after replays have been posted. However, I am concerned about mainland since it appears to include some procedural generation (making the learning problem harder as it would need to generalize to the different variations produced by the procedural generation). It would be nice to start with a fixed (skirmish) map to make the learning problem easier. Is there a more popular (2 player) skirmish map that might be a better fit?
  10. I think it would be really cool if we could start creating a public dataset of replays for a standard scenario (especially in light of recent things like AlphaStar). Although I am most interested in creating the dataset for training ML agents (such as imitation learning - maybe to initialize a reinforcement learned agent), it could also be cool for comparing player strategies using a common baseline, or for potentially find shortcomings with the current AI. I think the dataset should be public domain to keep it open for anyone to use for fun, publication, etc. One natural scenario to start with would be the Acropolis Bay 2 map where both the human player and the AI are Spartans. The AI difficulty can be set to whatever people want but stronger AI is more impressive, of course If you have a replay you would like to donate to the dataset (using the specific scenario mentioned above), feel free to post it here! By donating the replay, you are giving consent for the data to be made public domain and, therefore, able to be freely used for research, fun, and the like! Update: Rather than Acropolis Bay, the Mainland map is preferred (using the small map size) using the same civilization and players listed above. However, any and all replays are welcome and appreciated!
  • Create New...