Rejoining games as spectators instead of player in some cases

[Dunedan]

There have been reports that rejoining games isn't possible sometimes, as the rating for the player rejoining the lobby doesn't show up (https://gitea.wildfiregames.com/0ad/0ad/issues/7185). There was already some discussion regarding this topic in the following thread:

Analysis of logs for cases where that happened to players showed that in these cases the players show up as connected with two different instances of 0ad to the lobby server. As we can assume they didn't do that intentionally, it's still not clear how that can happen.

Workarounds which would fix this problem as well would be https://gitea.wildfiregames.com/0ad/0ad/issues/5320, https://gitea.wildfiregames.com/0ad/0ad/issues/4203 or patching Slixmpp to let EcheLOn keep track of all connected resources of a player, however these would just be workarounds, without addressing the root cause.

What I need for further debugging to find the root cause are:

• whenever this problem happens to you, ping me in the lobby
• whenever this problem happens to you, reply here with the following information:
  • operating system
  • kind of internet connection (landline, mobile network, satellite, ...)
  • list of installed mods
  • reason why you left the game in the first place (crash, loss of internet connectivity, ...)

[Dunedan]

I just retrieved the confirmation that this bug happens unrelated to being in a game or not. So just loosing connection to the lobby and rejoining it, might trigger it.

[Acero]

  

On 09/11/2024 at 9:02 PM, Norse_Harold said:

When you have an existing session and connect another one then the new session will not receive the rating list right away. It will only receive the rating list once another user has connected to the lobby or completed a rated match. Sometimes this takes many minutes when it is late at night and there aren't many players online.

If I understand correctly what Norse says here, the event of any new user connecting to the lobby refreshes the ratings for all players, including the ones with the bugged ratings.

If this is the case, two ideas come to my mind:

1) This problem of aborted games due to the rating bug probably happens mostly for us late night american players which usually play when the lobby is at its lowest in terms of connected players. At this time of the night it is common to not see any new player connecting for minutes or even tens of minutes at a time. On the other hand, at any other time of the day this problem is probably not as important beacuse on average you see new players connecting very often.

2) A convoluted but effective method of saving games for late night american players could be the following: always have the credentials of a second account ready, and when you or another player looses ratings: (a) leave 0ad, (b) log in with the other account in order to refresh the ratings for everyone, and (c) rejoin with your original account.

With this method any player should be able to fix the ratings bug for any other player, even if the player with the bug does not have a second account for this purpose.

[Dunedan]

1 hour ago, Acero said:

If I understand correctly what Norse says here, the event of any new user connecting to the lobby refreshes the ratings for all players, including the ones with the bugged ratings.

If this is the case, two ideas come to my mind

Ironically it refreshes only the ratings for all players not affected by this bug, because this bug seems to be caused by players (unintentionally?) being connected with multiple instances of 0 A.D. and only one instance would receive these updates. Even if all instances would receive the updates, other players joining is a frequent occurrence when somebody is affected by this bug and doesn't solve it.

1 hour ago, Acero said:

2) A convoluted but effective method of saving games for late night american players could be the following: always have the credentials of a second account ready, and when you or another player looses ratings: (a) leave 0ad, (b) log in with the other account in order to refresh the ratings for everyone, and (c) rejoin with your original account.

Registering multiple accounts violates the Terms of Use. Don't do that!

[Stan`]

9 hours ago, Dunedan said:

Ironically it refreshes only the ratings for all players not affected by this bug, because this bug seems to be caused by players (unintentionally?) being connected with multiple instances of 0 A.D. and only one instance would receive these updates. Even if all instances would receive the updates, other players joining is a frequent occurrence when somebody is affected by this bug and doesn't solve it.

I've noticed that sometimes when 0 A.D. crashes there is a zombie process lingering. Maybe this process is somehow maintaining the connexion to the lobby causing those issues.

[Dunedan]

In the time since, thanks to several people pinging me, I was able to investigate additional cases where this happened. What usually happens is:

1. player joins the lobby with connection 1 (everything is fine)
2. player joins the lobby with connection 2 (that being their active 0ad now and no rating is being shown)
3. player leaves the lobby with connection 2 (probably when closing 0ad)
4. player leaves the lobby with connection 1 (maybe when shutting down the PC)

Step 2 is usually a direct response of a player having left a game (for reasons I don't know) and wants to rejoin.

It's certainly not a timeout issue, as step 4 might happen hours later.

To me that zombie process theory sounds somewhat plausible, however I've had no player answering the questions from the first post here yet, so we don't know if and why they restarted 0ad. If the zombie process theory is correct, then that process should be visible in the list of processes and a clean restart of the PC should work around that problem. Instead of a zombie process, I could also imagine 0ad getting confused internally and managing two concurrent connections to the lobby.

[0 calories]

4 hours ago, Dunedan said:

In the time since, thanks to several people pinging me, I was able to investigate additional cases where this happened. What usually happens is:

1. player joins the lobby with connection 1 (everything is fine)
2. player joins the lobby with connection 2 (that being their active 0ad now and no rating is being shown)
3. player leaves the lobby with connection 2 (probably when closing 0ad)
4. player leaves the lobby with connection 1 (maybe when shutting down the PC)

Step 2 is usually a direct response of a player having left a game (for reasons I don't know) and wants to rejoin.

It's certainly not a timeout issue, as step 4 might happen hours later.

To me that zombie process theory sounds somewhat plausible, however I've had no player answering the questions from the first post here yet, so we don't know if and why they restarted 0ad. If the zombie process theory is correct, then that process should be visible in the list of processes and a clean restart of the PC should work around that problem. Instead of a zombie process, I could also imagine 0ad getting confused internally and managing two concurrent connections to the lobby.

I think most of the cases is when user having networking issue he will just rejoin with another provider mobile/wifi etc.. what has available to finish the game...

[Dunedan]

In the cases I've seen, the users rejoined with the same internet provider.

[Dunedan]

I just happened to catch a player while they had the problem and was able to ask some questions:

• OS: Ubuntu 24.04LTS
• 0ad installed as snap package
• no second pyrogenesis process running, as verified via "ps"
• installed mods: Feldmap and autociv, disabling them didn't help
• using WiFi at home and landline connection to the ISP

Having more of such detailed reports would be helpful, as a pattern might emerge.

[leopard]

This happened to me today: I changed my VPN after losing connection while playing, but I kept rejoining as a spectator. Could it be possible that the issue was due to the new IP allocated by the ISP? I was able to rejoin the game with the VPN I was previously using to play. One VPN was a self-hosted server in India, and the other was a third-party server located in Paris.

[Vantha]

I think the cause could indeed an IP change.

A while ago, I had a router that would get assigned a new IP address on every reboot. My connection back then was pretty weak and I had the habit of restarting it before reconnecting to a game I was disconnected from (just don't question it; it actually helped with the lag). After doing so, I - as far as I remember - pretty consistently rejoined as observer the first time (and would only be assigned to the open player slot after leaving and joining a second time - without any reboots in between).

[0 calories]

I have very same experience. 100% triggered by changed IP address - no other cases I observed.  Of course there can be other use cases when it happens but this is obvious... 

[Dunedan]

All the cases I've been able to look into so far, including the one from today, didn't involve a changed IP-address.

[real_tabasco_sauce]

do you want info from players that haven't ever had this happen to compare? In my case, I have never had a rating with my account, so that is probably why.

Edited January 6 by real_tabasco_sauce

[Dunedan]

The issue isn't related to ratings, but caused by the XMPP server seeing two distinct connections from a client. There tend to be more reports from players with rating, as in that case rejoining a game doesn't work. Rating or not is however not related to the underlying issue.

[real_tabasco_sauce]

have you had a report from someone without a rating? I wonder if ratings could not be the direct cause but one of a handful of contributing factors.

Edited January 6 by real_tabasco_sauce

[Dunedan]

No report from somebody without a rating yet, so yeah, worth keeping that in mind, even though I believe it's unlikely to be a contributing factor.

Who knows, maybe a27 will already fix this issue.

[ffm2]

I just had this recently somewhere between Christmas and now. Iirc the map was corithian_isthmus. My os is arch linux. Mods ffm_4k, ffm_visibility, autociv (with some adjustments). It could be that community-mod and feldmap were also on, but unsure. I remember in game was ffm (without rating) logged in, so I couldn't join. I then hosted a game, launched a second instance of 0ad and joined the lobby and could join the game. I searched my game records and didn't find this game (corinthian isthmus or a ffm without rating and multiplayer). This was the only time it happened to me since I play again. But it was a issue 4 years ago too.

Edit: The replay was just on a different pc.

Edited January 6 by ffm2

[sarcoma]

this happens a lot when someone losses connection suddenly (power/internet outage, ddos, etc)

the lobby keeps the player listed for a minute or so, if u join before that it gives you the nickname without rating, so u have to wait until it destroys the first conn and rejoin

[Dunedan]

25 minutes ago, sarcoma said:

the lobby keeps the player listed for a minute or so, if u join before that it gives you the nickname without rating, so u have to wait until it destroys the first conn and rejoin

The problem tracked here persists for way longer than a minute, sometimes up to an hour or so.

[guerringuerrin]

 

On 15/12/2024 at 5:38 AM, Dunedan said:

In the time since, thanks to several people pinging me, I was able to investigate additional cases where this happened. What usually happens is:

1. player joins the lobby with connection 1 (everything is fine)
2. player joins the lobby with connection 2 (that being their active 0ad now and no rating is being shown)
3. player leaves the lobby with connection 2 (probably when closing 0ad)
4. player leaves the lobby with connection 1 (maybe when shutting down the PC)

Step 2 is usually a direct response of a player having left a game (for reasons I don't know) and wants to rejoin.

It's certainly not a timeout issue, as step 4 might happen hours later.

To me that zombie process theory sounds somewhat plausible, however I've had no player answering the questions from the first post here yet, so we don't know if and why they restarted 0ad. If the zombie process theory is correct, then that process should be visible in the list of processes and a clean restart of the PC should work around that problem. Instead of a zombie process, I could also imagine 0ad getting confused internally and managing two concurrent connections to the lobby.

This has happened with some games I played it and sometimes the player restart his PC and the rating issue keeps going so it might not being related with a client-side zombie process.

Edited January 7 by guerringuerrin

[Dunedan]

I just changed some config on the lobby server. While that doesn't fix the actual issue, it might speed up the time it takes to work again for affected players. If a player is affected now and rejoins the lobby after 90 seconds, they should have a working state again. There is a slight chance this has a negative effect on players with flaky connections though, which might get disconnected more often from the lobby. Please report your experiences!

[0 calories]

2 minutes ago, Dunedan said:

I just changed some config on the lobby server. While that doesn't fix the actual issue, it might speed up the time it takes to work again for affected players. If a player is affected now and rejoins the lobby after 90 seconds, they should have a working state again. There is a slight chance this has a negative effect on players with flaky connections though, which might get disconnected more often from the lobby. Please report your experiences!

90 second is quite   high value for real time game.. so if anyone has really bad connection why keep him stay alive?  Can you test set as low as possible? What about 30 secs?

[Dunedan]

50 minutes ago, 0 calories said:

90 second is quite   high value for real time game.. so if anyone has really bad connection why keep him stay alive?

That's not related to the game itself, but to the connection to the lobby.

[0 calories]

1 hour ago, Dunedan said:

That's not related to the game itself, but to the connection to the lobby.

That was the point. Player will lose connection or got disconnected, usually he is able to rejoin quickly as I observed... -> joining to the lobby within 90 sec is very likely... user get no rating assigned by ratingbot and issue is still present.. Lowering lobby client "keep" alive to mimumim is best solution.