Game Crash - Observer Turn Desynchronization

[RolandSC2]

Hello,

Was hosting a 4v4 yesterday: all units in game froze around minute 20 as a player attempted to join as an observer. The error message was something about the player joining on turn "1710, but expected turn is 1752."

Replay attached, but I can't find the log.

Will look again for other logs/files if anyone can provide instructions.

roland_desynch_replay.zip

[elexis]

Careful, if you start 0ad some of the vial logfiles will be overwritten. I guess you already did that to find the directory of the replay.

https://trac.wildfiregames.com/wiki/GameDataPaths

It's important to know the exact error message, in case of out-of-sync also which player exactly was out of sync (and then ideally the logfiles of that one too).

Reading your recollected error message, it however seems that it wasn't an out-of-sync error, but a TurnManager error.

Especially since you said that the units froze, it sounds a lot like this bug, there will never be a new Turn executed.

Quote

LOGERROR("NotifyFinishedClientCommands: Client %d (%s) is ready for turn %d, but expected %d",

LOGERROR("NotifyFinishedClientUpdate: Client %d (%s) is ready for turn %d, but expected %d",

I expect the game to not progress until the client was disconnected, was that observed?

If the rejoining player was disconnected and the game still didn't progress, then there is something unknown here.

If the rejoining player never disconnected or if he disconnected and the game then progressed,

-> tolduso (or rather toldthemso), was just fixed before the release (and should have been fixed in previous ones): #3643

[RolandSC2]

Thanks, Elexis. I found the log files, but you are right that they have been overwritten and contain no data from the event yesterday. I can still furnish them as needed.

6 minutes ago, elexis said:

If the rejoining player was disconnected and the game still didn't progress, then there is something unknown here.

If the rejoining player never disconnected or if he disconnected and the game then progressed,

1 - The rejoining player connected and then disconnected (after "losing connection" countdown--without kick/ban), but the game still did not progress.

2 - The rejoining player attempted to rejoin again and was banned during connection. Next, all observers were banned, but the game did not progress.

[elexis]

So either

1. the described TurnManager error was irrelevant to the game not progressing and a different bug made the game stop, or
2. there is a different callstack than the previously reproducible one that might somehow break the NetServerTurnManager cleaning up (and thus not waiting anymore for the next turn of that client).

For 1: the game only progresses as long as all clients are ready for the next turn. But one client might still be connected to the game, but it's network connection is so bad that it stalls the entire game without being notified (could probably only happen under extreme conditions).

For 2: We have old logfiles in the ticket, but it's too much data without a specific indication as to what caused the error. With good knowledge of the turnmanager code one might be able to discover unreported bugs that may coincide with the bug described here.