Advice on tracking AI bot API changes?

[agentx]

The AI stores events e.g. attacks as attack plans (ongoing + planned ones). So if this attack class is serialised completely, then the saved games should indeed work.

This is getting funny, how can an AI serialize events which are going to happen AFTER the game was loaded from file? Do you guys have a wormhole somewhere hidden?

Or is the purpose of saved games not to continue playing after loading?

[niektb]

No. Those plans are serialized in the save game, and deserialized on loading. (Am I missing something?)

[agentx]

No. Those plans are serialized in the save game, and deserialized on loading. (Am I missing something?)

There is no problem loading a game, there is a problem continuing a loaded game. Does that distinction makes any sense?

Anybody interested to repair the "Load Game" button?

[Radagast.]

Or is the purpose of saved games not to continue playing after loading?

you're the master.

Okay, so you think of Events that are just on their way to the engine or which events are these? Or do you mean dynamic data, which can't be stored if I remember Sander's words correctly?

After a game is loaded, the AI has to rebuild the Events. I'm not sure if it does, probably not - but how can we fix it. What do you think? I would genuinely be interested in proper saved & properly continued loaded games.

Edit: I just can't get why my Units/AI decision tree (and hence their thinking) should be lost if they are

• serialised,
• stored,
• reloaded.
• continued. <-- thx Agentx for pointing out

Why should the gamestate then not continue. All the objects can be deserialised, so finally ALL is included (at least all that we serialised before).

But I think, you know what you are speaking about. So could you clarify even more? Or even fix it at the end. I saw your Hannibal progress - so I think you're the man for it.

Edited April 26, 2014 by Hephaestion

[agentx]

After a game is loaded, the AI has to rebuild the Events. I'm not sure if it does, probably not - but how can we fix it.

OK, I got it, I entered uncharted land. There is a whole continent full of game amusement no one has ever enjoyed. But I'm alone and nobody speaks my language. Not a single soul ever continued playing a saved game.

Maybe an analogy helps: a movie and a projector. The movie is suspenseful, you are sitting on the edge of your chair, chewing your nails. Then nature makes its call: You have to go for pee, too much ale. Fortunately the projector has a pause button. You press the button, light dims down, the projector goes silent and you to the men's room.

Coming back, you think the movie continues where you paused. But this is what happens: Light comes back, the projector makes noise again, you see the last scene, and that's it. The projector does not continue playing. Whatever you do it still shows that last image. Then you hear a voice in your head telling you: Don't worry, it has found where you stopped. See that image, is that nothing? Don't be so greedy.

But I'm greedy, I want to continue playing saved games. And I'm tired hearing about serialization, I'm talking about what happens AFTER, that very the moment the API ceases to continue. So, you silly voice, please go out of the way and shut down.

> Or even fix it at the end.

You want me to touch the API? Who is going to decide that?

[Radagast.]

We're a community project. Contributions are always welcome. (change it, create a diff vs SVN and create a trac ticket)

And nice story indeed.

If the API doesn't continue this may be. We should fix it indeed. (this then really is unrelated to deserialisation if there is a bug in getting things rolling again)

Edit: I just thought it would continue its logic after reload, i.e. continue using the objects/memory if they were loaded correctly. But if there is a problem in this continuing .. then okay. And btw. the picture is not standing, it's currently starting over again and this is because the AI side IS NOT EVEN SERIALISED as wraitii stated. So if this step is missing, then even the step AFTER the loading has no effect - even if it would work (but apparently it doesn't as you have spotted a bug. I would love to see this going, and I think I'm not alone.).

Edit: I have added point 4 to my list: continuing.

Edited April 26, 2014 by Hephaestion

[niektb]

But to make sure the devs agree with and the changes will be committed you'd best contact Wriatii or mimo (the authors of Aegis and Petra respectively.)

Edited April 26, 2014 by niektb

[Radagast.]

For API changes they surely want to see it in a patch and then discuss it in the trac ticket and in IRC. (but it would be indeed helpful if wraitii(author API3) and mimo had a look at this topic.)

Edited April 26, 2014 by Hephaestion

[agentx]

For API changes they surely want to see it in a patch and then discuss it in the trac ticket and in IRC. (but it would be indeed helpful if wraitii(author API3) and mimo had a look at this topic.)

Well, forget about. I'm not going to listen to silly voices again. I don't see an even perfectly working patch taking this hurdle. If Wraitii would actively support loading games, why is this strange discussion happening? Don't misunderstand me, I don't want a blank check, I need reasonable criteria.

[Radagast.]

Saved games are a desirable feature. And if there is a limitation in the API then we have to change it. I don't think your patch will be rejected if it's working. In opposite, they will be happy when the bug is fixed.

It's not like KiCAD, where they once said "we don't use it because we don't want more features if they bring new code". No offense, KiCAD is a strong project and I'm happy it exists. Without it I'd be pretty doomed.

Edit: If you can solve the problem with extending common API functions, then you could even offer it as a mod.

So not even a review.

Edited April 26, 2014 by Hephaestion

[agentx]

If you can solve the problem with extending common API functions, then you could even offer it as a mod.

So not even a review.

Interesting. The bug starts in /simulation/ai/common-api/baseAI.js. But currently it looks more like the C++ side has problems to feed HandleMessage with proper params. Can you see any difference between loaded and fresh games here: http://trac.wildfiregames.com/browser/ps/trunk/source/simulation2/components/CCmpAIManager.cpp

[sanderd17]

Well, forget about. I'm not going to listen to silly voices again. I don't see an even perfectly working patch taking this hurdle. If Wraitii would actively support loading games, why is this strange discussion happening? Don't misunderstand me, I don't want a blank check, I need reasonable criteria.

You don't need a blank check. If you fix a bug, without introducing a new one, and the code is clear enough, then it will be added to the main game. That counts for anyone.

I didn't know the game was messed up after deserialization. I never noticed it, as the ai doesn't even deserialize correctly.

[agentx]

Here is the ticket: http://trac.wildfiregames.com/ticket/2495 Let me know when the API delivers proper objects, I'll do what I can to remove this roadblock. It can't be that difficult, because the C++/CCmpAIManager side knows how to initialize the game objects.

[Radagast.]

That's good news, agent.

I've not yet reached that part of the engine. Wraitii - which also tweaked our awesome water - could perhaps know what to do best to fix it.

If Wraitii has no time either, then we have to fix it ourself somehow. But this then not makes it into Alpha 16 for sure.

[sanderd17]

It certainly won't be fixed for A16. We should be going into feature freeze in 3 days. I doubt anyone will solve it against that time (the team is quite busy these days).

[Radagast.]

yeah, it's buried deep.

Still have to find the issue and need you help Agentx. I recall the stacktrace:

• main.cpp#Line207 static int ProgressiveLoad()

  Status ret = LDR_ProgressiveLoad(10e-3, description, ARRAY_SIZE(description), &progress_percent);

• ...
• ps/Game.cpp#L186 CGame::LoadInitialState()

  bool ok = m_Simulation2->DeserializeState(stream);

• simulation2/Simulation2.cpp#Line788 CSimulation2::DeserializeState(std::istream& stream)

  bool CSimulation2::DeserializeState(std::istream& stream){       // TODO: need to make sure the required SYSTEM_ENTITY components get constructed    return m->m_ComponentManager.DeserializeState(stream);}  

• simulation2/system/ComponentManagerSerialization.cpp#Line309

  // Deserialize, with the appropriate template for this componentif (entTemplate)    component->Deserialize(entTemplate->GetChild(ctname.c_str()), deserializer);else    component->Deserialize(noParam, deserializer);

• simulation2/components/CCmpAIManager.cpp#Line825

  virtual void Deserialize(const CParamNode& paramNode, IDeserializer& deserialize){    Init(paramNode);                 ForceLoadEntityTemplates();                 m_Worker.Deserialize(deserialize.GetStream());    m_JustDeserialized = true;}

• simulation2/components/CCmpAIManager.cpp#Line604void Deserialize(std::istream& stream)

  begin reading from here ^

  Line 645 is where all AI commands are reloaded:

  uint32_t numCommands;deserializer.NumberU32_Unbounded("num commands", numCommands);m_Players.back()->m_Commands.reserve(numCommands);for (size_t j = 0; j < numCommands; ++j){    CScriptVal val;    deserializer.ScriptVal("command", val);    m_Players.back()->m_Commands.push_back(m_ScriptInterface->WriteStructuredClone(val.get()));}

Thus each component is deserialised, every tech reloaded, ... now we have to found why it doesn't continue.

Could you read until Line 681. That's what happens. There is nothing more that happens. But I think it's already quite a lot .. and cool.

[agentx]

Looking on how the API structured, I would expect this top level process:

New Game :

[b]ENGINE               BOT[/b]read map  prep entities  prep maps  call        >      CustomInit(gameState, sharedScript)                       prep selfstart game  step +1  update events  update entities  update maps  call        >      OnUpdate(sharedScript)[/font]                       analyse maps, entities, events                        fire commands // train, move, build, etc process commands goto step

Load game:

ENGINE               BOTread saved game  prep entities  prep maps  call        >      Deserialize(sharedScript)                       prep selfstart game  step +1  update events  update entities  update maps  call        >      OnUpdate(sharedScript)                       analyse maps, entities, events                       fire commands // train, move, build, etcprocess commandsgoto step

One issue is, Deserialize got called without gameState, and the sharedScript lacks important information. This is API-wise managed by BaseAI.prototype.HandleMessage / Init.

Probably long time ago there was misunderstanding what the engine and what the bot has to serilaize/deserialize, leading to assumption the bot has to serialize everything (maps, entities, events, +), however this can't work. The moment Bot.OnUpdate or API.HandleMessage is called, the engine has to deliver all/same information whether load or new game.

If could read C a little better, I would try to find out if the engine is capable to provide the API a gamestate with same layout + information from a saved game and a loaded map. If the answer is yes, I would start dancing around my chair, because then a solution is very close. But I fear the engine doesn't run update maps, events, entities in case of load game.

[Chakakhan]

It seems to me that game save/load needs to be abstracted from the AI code. It should happen automagically.

[Stan`]

You can't since you need to save the ai state ^^

[Chakakhan]

Ah, so the state is not in the C++ code it is in JS. I need to look at the code to see how the API is implemented. We were talking about putting a lot of the objects back in the C++ side and providing interfaces to JS several years back because of the overhead caused by the AI. I will review the code and read forums to see what came of all of that.

Cheers!