BUG - Game incredibly slow/freeze at start because because of 1 player.

[Atrik]

A27 has a new bug that occurs, reasonably often where a player will make the game freeze or incredibly slow.
What we currently observe : This isn't some kind of lag, the player won't experience network, nor cpu lag when this occurs.

Since this isn't creating any logs, how can we help document this bug?

[Gurken Khan]

I also had this happen a few times where everything but the ambient sounds froze. But no logs...

[Atrik]

4 hours ago, Gurken Khan said:

I also had this happen a few times where everything but the ambient sounds froze. But no logs...

If you experience screen freeze/0ad freeze it might be a different issue, I believe in the case I describe the player can still control the camera for example. The above also seems to happen 1/10th of games.

[Seleucids]

You can decrease the BAD NETWORK constant in net server.cpp or some file in the source code to print out the lagger at earlier times. Also don't allow this value to be increased by lag.

[Atrik]

6 minutes ago, Seleucids said:

You can decrease the BAD NETWORK constant in net server.cpp or some file in the source code to print out the lagger at earlier times. Also don't allow this value to be increased by lag.

Could or could not help idk, it's not network related. 

[guerringuerrin]

22 hours ago, Atrik said:

A27 has a new bug that occurs, reasonably often where a player will make the game freeze or incredibly slow.
What we currently observe : This isn't some kind of lag, the player won't experience network, nor cpu lag when this occurs.

Since this isn't creating any logs, how can we help document this bug?

It looks like similar behavior of game stuttering due to PC performance
Are you referring to cases where no unit moves and no orders are executed, but the GUI and camera remain responsive? 
I’ve also noticed that in cases like this, hosts start kicking players until units begin to move again. But after the suspected bugged player rejoins, the game continues with exaggerated stuttering.

Edited Wednesday at 15:30 by guerringuerrin

[Atrik]

4 minutes ago, guerringuerrin said:

Are you referring to cases where no unit moves and no orders are executed, but the GUI and camera remain responsive? 
I’ve also noticed that in cases like this, hosts start kicking players until units begin to move again. But after the suspected bugged player rejoins, the game continues with exaggerated stuttering.

Yes that's exactly the symptoms. IIRC I've already been that player who slows the game and I didn't experience any lag, not network, nor pc. Every time this happens, the buggy player doesn't believe others when they say he is lagging ("My cpu is at 15%" "I've just run a speedtest"). But kicking him out clearly solves the issue. It really happened to a very wide variety of players, and with enough occurrences to believe it's a bug of the game rather then any external factor.

[Stan`]

29 minutes ago, Atrik said:

Yes that's exactly the symptoms. IIRC I've already been that player who slows the game and I didn't experience any lag, not network, nor pc. Every time this happens, the buggy player doesn't believe others when they say he is lagging ("My cpu is at 15%" "I've just run a speedtest"). But kicking him out clearly solves the issue. It really happened to a very wide variety of players, and with enough occurrences to believe it's a bug of the game rather then any external factor.

How do you find out which one is slowing the game ?

[Atrik]

45 minutes ago, Stan` said:

How do you find out which one is slowing the game ?

The host kick players 1by1. Then once the buggy player leave, simulation run again, once he rejoined, the game is stopped or incredibly slow (slower then x0.1).
This never happens mid game, it's always at game start, but the player can rejoin and he'll still stop the simulation.

Edited Wednesday at 16:52 by Atrik

[hyperion]

Is it never observers causing this?

Does who is hosting matter?

What OS are the players on that trigger the issue?

Is there a video that could be shared to have a better impression of what the issue looks like for involved players?

What versions does this happen with, 0.27.0 and 0.27.1?

[guerringuerrin]

1 minute ago, hyperion said:

Is it never observers causing this?

I've experienced this as host. Max Lag for observer set to -1 on my case. Didn't test kicking observers.

1 minute ago, hyperion said:

Does who is hosting matter?

I've experienced this with at least 3 different hosts that often hosts games, including me as host. 

1 minute ago, hyperion said:

What OS are the players on that trigger the issue?

Is there a video that could be shared to have a better impression of what the issue looks like for involved players?

What versions does this happen with, 0.27.0 and 0.27.1?

Idk, will ask next time

1 minute ago, hyperion said:

What versions does this happen with, 0.27.0 and 0.27.1?

I'm not sure but I think it happens with both versions. Currently using 0.27.1 but i'm not sure if everyone is using it

 

[guerringuerrin]

@hyperion
https://drive.google.com/file/d/1BUfQS7Vyi9jUcRVY7M2IW_P4VFA_JUF_/view?usp=drive_link
Sharing through drive bc file is too big
This happened today. Unfortunately, I didn’t start recording from the beginning, but the behavior is exactly what we mentioned earlier:
The game starts, the GUI is responsive, you can select units and buildings and issue commands, but nothing moves and unit production doesn’t start either. The suspected player gets kicked or leaves, and everything starts working normally.
In this case, it was reza-math, who often plays in my matches. It could be a "false positive" and just a connection issue.
But I think this is the kind of behavior we’ve been seeing more frequently lately.

@Atrikcould you confirm this is more or less what you described? 
reza-math is using Windows 10

Edited 20 hours ago by guerringuerrin

[Atrik]

11 minutes ago, guerringuerrin said:

@Atrikcould you confirm this is more or less what you described? 

Yes, also seems for-now that a27 or a27.1 makes no differences.

[erictommy]

mainlog.html from my host to @ffm2  to check.

mainlog.html

[guerringuerrin]

@hyperion
https://drive.google.com/file/d/15vGn4ddL1nNRbtZeXqI5sqKLhm7zyNAt/view?usp=sharing
This is another example. The game was originally hosted by @erictommy, and everything was going fine until several spectators joined at some point. One of them caused an OOS, but we continued the match for a few seconds until the simulation completely froze (the GUI was still responsive).
levai was one of the active players in the match — I don’t remember if he left on his own or was kicked, but once he was out, the simulation started working again. When he rejoined, the same thing happened: simulation frozen, GUI functional.
We decided to save and rehost the match, this time with me as the host. After everyone loaded in, the simulation was frozen again. Once levai left, it resumed.
So we saved and hosted again, but this time replaced levai with RangerK — who happened to be the player that, when kicked or banned, allowed the simulation to continue normally — as if there was a problem tied specifically to that slot or civilization.

What’s strange is that the simulation started failing when a spectator joined the game. Until then, everything worked fine and levai hadn’t shown any lag or other performance issues.
A preliminary hypothesis is that this bug occurs when a player or spectator finishes loading the game or map. Why do I think this? Because the only thing I can clearly identify in common between a match just starting and one already in progress when a spectator joins, is the act of loading the game/map.

[hyperion]

Thanks,

Going by the video the issue is likely the simulation freezing due to a player not sending his turn end signal within a reasonable amount of time, which is different than the game freezing. For the player causing the issue this might indeed look like a game freeze tho. That would also match @Gurken Khan report which I first thought to be unrelated but actually might be the same.

Given that in one video there are still sporadic simulation updates happening it's not a hang but something taking much more time then it should. So of most interest right now are logs / OS / hardware from players causing the issue.

Looking at the mainlog of @erictommy there are a few mods which are unlikely to cause the issue but at least in theory could, so keeping an eye on what mods are enabled wont hurt at least. That mainlog also contains warnings for compiling glsl shaders. That is reasonably suspicious as it also happens early in the game. Maybe a problematic GL driver might be the root cause. That could also explain why 0.27.x as there where major changes there to support Vulkan. Players causing the issue might want to try switching to Vulkan if possible to see if it helps. Trying to update the driver is certainly a good idea as well.

I'm not yet able to reproduce it so more hints are welcome.

[Gurken Khan]

@hyperion The times it happened to me I was on Vulkan and had played awhile. What I found curious: when I killed the game after 1/2hr or putting the laptop to sleep and waking it up again, there was one last entry in the log about loading something (after I sent the kill command). As that specific item didn't seem necessarily suspicious to me I didn't save it tho.