Redesigning the simulation system

[Ykkrosh]

I've just merged a large change into SVN, which is a step towards rewriting the gameplay code (basically everything that is concerned with objects in the world). (I've been using Mercurial to develop this code before merging into SVN, since it's much nicer). (The new system is disabled by default - you have to use the "-sim2" command-line flag to enable it).

It's taken a lot of work so far, and will need a lot more work, but I believe it's worthwhile because of some fundamental difficulties with the old implementation, mainly:

• Scripting - writing non-performance-critical code in JS is usually much nicer than C++. The old system has a complex JS<->C++ interface (lots of classes and functions, lots of raw JS API used on the C++ side, etc), and a lot of gameplay that could have been written in JS was instead in C++.
  
• Serialization - saving games to disk, comparing multiplayer clients for sync errors, etc. The old system didn't handle that at all, and bolting it on would be very difficult and buggy, since the simulation state was spread over lots of objects with no easy way to keep track of everything.
  
• Synchronisation - our multiplayer system assumes every player performs precisely the same computations given the same inputs, but we also want to support multiplayer across different OSes and processor architectures. In particular, that makes floating point computations (as used in quite a few places in the old code) very dangerous, so the simulation code should use integer or fixed-point instead. (FP is fine for graphics code, since that doesn't have to be synchronised.)
  
• Testing - automated tests are very useful, but the old system made it hard to test parts of the code in isolation since there were lots of dependencies, and so it had no tests and quite a few bugs.
  

The new system attempts to solve these problems (and some others): gameplay is split into lots of fairly simple, mostly self-contained components, each of which is responsible for its own serialisation, and each can be tested in isolation, and each can be implemented in either C++ or JS without adding any complexity to the other code that has to interface with it. The new system probably introduces problems of its own, but hopefully they can be resolved, and continuing on by patching the old system (instead of redesigning and replacing it) does not seem feasible.

The current status is that the new system basically does what it was designed to do, and provides a stable infrastructure for implementing the proper gameplay code on top of, but the gameplay hasn't been implemented yet. (You can click on units and tell them to walk to a point, but that's about all). There is documentation on Trac briefly covering some of the high-level concepts and low-level design. That page also links to the list of Trac tickets, and there are lots more TODOs in the code itself - if anyone wants to help with these, please feel free . There isn't a concrete plan for the design of the gameplay code yet, but that's something I want to look at soon.

So, that's what I've been working on for a while, and I think it's going in the right direction so far - any feedback would be appreciated

[Ykkrosh]

A quick update on this after four weeks: I've mostly been working on adding an outline of the gameplay design - there's some Atlas editor integration, building placement, population counters, GUI interaction, pathfinding, combat, and minor other things. Currently planning to add some resource gathering and (extremely primitive) AI players too, and that should be alright for an initial outline. Then there's a load of details to fill in, and then it might be almost playable

[feneur]

You've been making great progress these last couple of weeks (and before as well, it's just more visible/easy to understand now ). Keep it up, and hopefully we'll get more people on board to help you (you're a great inspiration, so it would be surprising if other programmers wouldn't be motivated )

[Jeru]

[WhiteTreePaladin]

Indeed, you have made absolutely wonderful progress. I only wish I knew enough to be of help. Maybe when the focus shifts to JS I can contribute.

[Ykkrosh]

I've made this the default now, so you should get the new system when running the game or editor normally. (Use the "-sim1" command-line flag if you want to run the old one instead.)

This is still very incomplete - the gameplay feature status page gives a rough overview of what's needed, and many things that previously worked are either missing or poorly implemented. But I think the foundation is pretty solid now, so it's a good time to start implementing features properly without (hopefully) running into any roadblocks in the future.

The new code shouldn't crash, so please report any problems of that kind, though gameplay bugs (e.g. units walking on top of each other and on the bottom of the sea) probably aren't worth reporting now since they're just a consequence of the known incompleteness.

The in-game GUI is also quite ugly (I just hacked it together with whatever textures were already available) and needs some proper design attention, but it should fix problems like the update latency that the old design suffered from, as well as the scary complexity of the old GUI XML code and deeper problems like the interaction with the deterministic network-synchronised simulation state.

A lot more of the code is scripted now - if you look in binaries/data/mods/public/simulation/components/ and binaries/data/mods/public/gui/session_new/ then there's lots of files you can play around with (and the game should automatically reload and immediately start running any changes you make, without needing to restart the game) if you're interested in that kind of thing. (Need more documentation for all of this, of course - hopefully that's something we'll work on soon.)

There's a lot of rough edges to clean up over the next few months, but I think this is a good starting point that will allow some real progress