Continuous checking if a file is exists

[ramtzok1]

Hello again!

As part of my ML project, I'm trying to read a  txt file which contains a chance of PetraML to win the fight, now on the machine learning which is written in Python it calculates the possibility and returns it as a txt file as I mentioned and I'm trying the following code:

m.DefenseManager.prototype.getMLData = function()
{ 
	warn("asdasdasdasdasdasd");
	while (true) {
		if(Engine.FileExists("simulation/ai/petraML/mlData/answer.txt"))
			break;
	}
	warn("asdasdasdasdasdasd546545645665");
	let chance = parseFloat(Engine.ReadFile("simulation/ai/petraML/mlData/answer.txt"));
	Engine.AppendToBuffer("a");
	Engine.WriteToFile("simulation/si/petraML/mlData/gameState.txt");
	return chance; 

};

The problem is the function is getting stuck inside the while true look, I looked inside Engine.FileExists function and found out it returns true the whole time but the game is still calling it for some reason.

Why do you need that?:

The ML receives a file loaded with the units and calculates the possibility to win and return it as an answer.txt file and in the meantime, this function is waiting for the file to arrive but as I said it's getting stuck there.

Is there a way to make this kind of loop? I need a way to transfer the data between the module and the game while the AI is waiting for an answer to come.

Thanks in advance!

[Guest]

I really don’t think you want blocking behavior there while the file is non-existent. Being in two different environments, that might lead to issues.

You might want to remove the loop and just call getMLData periodically. Every 5 turns or so. Which would be updating the chance property of def manager. So, rather  than getMLData returning the chance, it updates the value and maybe you can add another method to retrieve the latest chance value in def manager.

(I lack some context so maybe I understood the problem wrong)

[ramtzok1]

9 minutes ago, (-_-) said:

I really don’t think you want blocking behavior there while the file is non-existent. Being in two different environments, that might lead to issues.

You might want to remove the loop and just call getMLData periodically. Every 5 turns or so. Which would be updating the chance property of def manager. So, rather  than getMLData returning the chance, it updates the value and maybe you can add another method to retrieve the latest chance value in def manager.

(I lack some context so maybe I understood the problem wrong)

For your request here's some more information:

We are overriding the functions AssignDefenders and NeedsDefenders so if you compare between our PetraML and the regular Petra

m.DefenseManager.prototype.assignDefenders = function(gameState)
{
	if (!this.armies.length)
		return;
	let armiesNeeding = [];
	// let's add defenders
	for (let army of this.armies)
	{
		this.sendDataToML(gameState, army);
		let needsDef = this.getMLData();
		 if (needsDef > 0.8)
			continue;

		let armyAccess;
		for (let entId of army.foeEntities)
		{
			let ent = gameState.getEntityById(entId);
			if (!ent || !ent.position())
				continue;
			armyAccess = m.getLandAccess(gameState, ent);
			break;
		}
		if (!armyAccess)
			API3.warn(" PETRAML error: attacking army " + army.ID + " without access");
		army.recalculatePosition(gameState);
		armiesNeeding.push({ "army": army, "access": armyAccess, "need": needsDef });
	}

	if (!armiesNeeding.length)
		return;
	// Ram: DON'T DELETE THAT.
	// It helps us to gather our fighting units.
	
	// let's get our potential units
	let potentialDefenders = [];
	gameState.getOwnUnits().forEach(function(ent) {
		if (!ent.position())
			return;
		if (ent.getMetadata(PlayerID, "plan") == -2 || ent.getMetadata(PlayerID, "plan") == -3)
			return;
		if (ent.hasClass("Support") || ent.attackTypes() === undefined)
			return;
		if (ent.hasClass("Catapult"))
			return;
		if (ent.hasClass("FishingBoat") || ent.hasClass("Trader"))
			return;
		if (ent.getMetadata(PlayerID, "transport") !== undefined ||
		    ent.getMetadata(PlayerID, "transporter") !== undefined)
			return;
		if (gameState.ai.HQ.victoryManager.criticalEnts.has(ent.id()))
			return;
		if (ent.getMetadata(PlayerID, "plan") !== undefined && ent.getMetadata(PlayerID, "plan") != -1)
		{
			let subrole = ent.getMetadata(PlayerID, "subrole");
			if (subrole && (subrole == "completing" || subrole == "walking" || subrole == "attacking"))
				return;
		}
		potentialDefenders.push(ent.id());
	});
	let i = 0;
	for (let aMin = 0; aMin < armiesNeeding.length; aMin++) 
	{
		let potentialEntity;
		i = 0;
		this.sendDataToML(gameState, armiesNeeding[aMin]);
		let winningPetential = this.getMLData();
		if(winningPetential >= 0.8)
		{
			armiesNeeding.splice(aMin, 1);
			continue;
		}
		let potentialID = 0;
		while(winningPetential < 0.8 && potentialDefenders.length >= potentialID)
			{
				if(potentialDefenders[potentialID] === undefined)
					continue;
				potentialEntity = gameState.getEntityById(potentialDefenders[i]);
				let currDist = API3.SquareVectorDistance(potentialEntity.position(), armiesNeeding[a].army.foePosition); // Gets the current distance between the potential dif and army
				if(currDist > 40000)
					continue;
				armiesNeeding[aMin].addOwn(potentialDefenders[potentialID]);
				armiesNeeding[aMin].army.assignUnit(gameState, potentialDefenders[potentialID]);
				this.sendDataToML(gameState, armiesNeeding[aMin].army);
				potentialDefenders[potentialID] = undefined;
				potentialID++;
				winningPetential = this.getMLData();
			}
		if(winningPetential >= 0.8)
			armiesNeeding.splice(aMin, 1);
		if(!armiesNeeding.length)
		{
			API3.warn("Everything is assigned!");
			return;
		}
		if(potentialID >= potentialDefenders.length)
		{
			API3.warn("Out of defenders!");
			break;
		}
	}


	// If shortage of defenders, produce infantry garrisoned in nearest civil centre
	let armiesPos = [];
	for (let a = 0; a < armiesNeeding.length; ++a)
		armiesPos.push(armiesNeeding[a].army.foePosition);
	gameState.ai.HQ.trainEmergencyUnits(gameState, armiesPos);
};

now the variable needsDef has to be assigned to the probability because then we can't decide what to do with enemy's attack.

Do you still believe checking every 5 turns is still a good idea (I can't remember how much is a turn maybe 20ms?)?

I'm not sure if making property is the best because we are talking about the chance of each army.

[Stan`]

2 minutes ago, ramtzok1 said:

20ms

200ms SinglePlayer 

500ms Multiplayer 

Those might increase depending on the game load

[Guest]

1 hour ago, stanislas69 said:

200ms SinglePlayer 

500ms Multiplayer 

Those might increase depending on the game load

To expand on that, those are simulation turns. Which is not the same as an AI turn.

An AI turn is every 8 simulation turns which translates to 4 seconds on multiplayer.

Regarding the actual code posted. I can only post something about it later today.

Edited April 19, 2019 by Guest

[ramtzok1]

5 minutes ago, (-_-) said:

To clarify, those are simulation turns. Which is not the same as an AI turn.

An AI turn is every 8 simulation turns which translates to 4 seconds on multiplayer.

Regarding the actual code posted. I can only post something about it later today.

No problem.

[Guest]

So, the MLdata is used for assigning defenders based on the chance of success an army has right?

I suppose you are passing the army to the ML to do its magic.

I still think a better choice would be to have chance as a property. But not in DefManger but in DefArmy object. And have the chance updated for each army.

m.DefenseArmy.prototype.update would retrieve the chance value from the ML and update it.

And then, you can get winning potential for each army using a m.DefenseArmy.prototype.GetSuccessProbability; function.

This most likely needs changes to the whole stack though.

Edited April 20, 2019 by Guest

[ramtzok1]

20 hours ago, (-_-) said:

So, the MLdata is used for assigning defenders based on the chance of success an army has right?

That's right.

20 hours ago, (-_-) said:

I still think a better choice would be to have chance as a property. But not in DefManger but in DefArmy object. And have the chance updated for each army.

m.DefenseArmy.prototype.update would retrieve the chance value from the ML and update it. 

And then, you can get winning potential for each army using a m.DefenseArmy.prototype.GetSuccessProbability; function.

So if I understand correctly, I need to move my sendDataToML and getMLData functions to defenseArmy, there, inside m.DefenseArmy.prototype.update after the for-loop I guess I send the army and getting the probability from the ML?

If so, I still run into the problem, the game stuck waiting for an answer.

20 hours ago, (-_-) said:

This most likely needs changes to the whole stack though.

I don't fully understand what do you mean.

[Guest]

Something like this in the update function. It doesn’t wait for the file to proceed to the next update. It just checks each update. Non-blocking behavior. I don’t think the last function name is correct, but such a function does exist.

if (Engine.FileExist(..) && Engine.LastMtime(..) != g_lastReadTime)

     this.chance = foo;

     g_LastReadTime = Date.now(); // this wont be equal to last modified time, but you get the idea.

And yes, it really is an ugly hack probably.

Edited April 21, 2019 by Guest

[Stan`]

Too bad you can't call CmpTimer... It executes stuff on each frame.

[ramtzok1]

15 hours ago, stanislas69 said:

Too bad you can't call CmpTimer... It executes stuff on each frame.

You got my attention,   I want to know more about this function.

Can you please give me more details on where it is located? I couldn't find it in the code.

16 hours ago, (-_-) said:

Something like this in the update function. It doesn’t wait for the file to proceed to the next update. It just checks each update. Non-blocking behavior. I don’t think the last function name is correct, but such a function does exist.

if (Engine.FileExist(..) && Engine.LastMtime(..) != g_lastReadTime)

     this.chance = foo;

     g_LastReadTime = Date.now(); // this wont be equal to last modified time, but you get the idea.

And yes, it really is an ugly hack probably.

I will try what you said and come back if I still get stuck, I also looked for the second function and couldn't find something with that name or references like: looking for "Last" or "Time" I also tried to look into any "scripting" folder, perhaps I missed it?

[Stan`]

2 minutes ago, ramtzok1 said:

You got my attention,   I want to know more about this function.

Can you please give me more details on where it is located? I couldn't find it in the code.

It's a system component in the Js code but it's not meant for AI it's only used for entities.

[Guest]

Engine.GetFileMTime(..) is the correct function name.

[ramtzok1]

On 4/21/2019 at 7:27 PM, (-_-) said:

Something like this in the update function. It doesn’t wait for the file to proceed to the next update. It just checks each update. Non-blocking behavior. I don’t think the last function name is correct, but such a function does exist.

if (Engine.FileExist(..) && Engine.LastMtime(..) != g_lastReadTime)

     this.chance = foo;

     g_LastReadTime = Date.now(); // this wont be equal to last modified time, but you get the idea.

And yes, it really is an ugly hack probably.

Hi again!, I'm ram's partner and I was  sitting on this now a bit more now.  I had 2 questions in mind: 
1.Wouldn't the second condition be always true? what is the first inital value of lastReadTime anyway?
2.When does the army.update func runs? How can we be sure it coincides with our editings to the Defense Manager itself?

Thanks a lot!

[Guest]

1. I assume you mean the comment. I was referring to the timestamp difference. The files wont be read in real time and there would be a delay too. It was more of pseudo-code.

2. When the Defense Manger's update function runs.

[elexis]

45 minutes ago, ramtzok1 said:

1.Wouldn't the second condition be always true? what is the first inital value of lastReadTime anyway?

Quote

var g_LastMTime = 0

....
if (Engine.FileExist(..)))

{

  let mTime = Engine.GetFileMTime(..);

  if (mTime != g_LastMTime)

  {

     ...

    // File marked as processed:

     g_LastMTime = m_Time;

     }

}

Using the same reference in both cases should be a bit safer than Date.now().

The file modification time of the file changes everytime the file is modified. Not sure if the AI VFS transports that properly. If it does, why would the condition always be true?

The filemodificationtime will be unix timestamp I guess, so I suppose everytimestamp before right now would work, most simply 0. (One could also start with undefined and extend the condition to test for undefined if one wants to treat the first read specially)

(File I/O as a dynamic interface to the AI is a bit weird, an HTTP server would be an alternative that is probably not less weird. Creating an ML algorithm inside JS would also be a possibility, but would prevent reuse of existing ML tools.)

(I guess it's not crucial to process every file written, otherwise there might be hypothetical concurrency problems.)