Stuck on Loading map page when joining Ginnungagap and Norse Harold

[Yekaterina]

 

Whenever I try to join Ginnungagap's and @Norse_Harold's game,  I get stuck on this page. Their side shows that I am in the room, but I cannot see  any buttons or the chat. There is no button to return neither, so the game is crashed. What could be causing this bug?

I've never been able to join anything hosted by Norse Harold even though he claimed that he has whitelisted me. There is always either Stun Error or this page. The issue with Ginnugagap started this afternoon.

 

 

[Yekaterina]

@Ginnungagap

[Yekaterina]

@sternstaub suggested turning off "Persist match settings". Perhaps it would help?

[sternstaub]

5 minutes ago, Helicity said:

@sternstaub suggested turning off "Persist match settings". Perhaps it would help?

It was mentioned in another forum topic, but that affected hosting only.

[Yekaterina]

I can't join Norse Harold's game since 1 year ago. Whenever I click on his host, it either says "the host is not running" or it freezes my game completely so that I had to close the program. On one occasion I clicked on his game and it crashed my entire operating system and I had to force reboot by pressing the power button. It happend on Arch KDE (0AD really hates KDE)

Maybe Norse Harold can provide some clue to what's going on, and please disable whatever mod you are using that causes freezes! If you don't like someone, you can ask them to leave or kick them, but don't crash their computer! 

 

[sternstaub]

44 minutes ago, Yekaterina said:

0AD really hates KDE

while using KDE, i sometimes had the impression that KDE really hates KDE...

[Yekaterina]

@Norse_Harold here:

[Norse_Harold]

Anyone on the list of players allowed to join my games who has trouble joining my games should join IRC so we can conduct troubleshooting. It might be a problem on my end, it might be a problem on your end.

Edited March 2, 2023 by Norse_Harold

[Norse_Harold]

Okay, Helicity and I did some testing, and we discovered that it's probably caused by the bug related to hero garrison with regicide mode. Helicity said that sternstaub recommended that he check for that bug and disable persist match settings. Merely disabling persist match settings was not enough, though.

Here is how to fix the bug related to hero garrison and regicide mode.

Except that Helicity couldn't find matchsettings.json or matchsettings.mp.json.

So, I hosted a game with a non-random map, such as Acropolis Bay. Helicity connected, and unlike when I was hosting with a random map like Mainland, he did not get disconnected. Then I enabled Regicide mode. Helicity noticed an error message that pointed toward the hero garrison bug. I enabled hero garrison, then disabled it. Then I switched to a randomly generated map (Mainland). And, Helicity did NOT get disconnected this time. Success.

Later, with Helicity's "normal settings" (with community-mod and other mods?) the problem returned. So, there is still some troubleshooting to do.

Note that there were some files in Helicity's mods/user directory, which is like having an extra stealth mod enabled that changes unexpected files. Files don't belong there for non-developer users, so we made a backup of those files and deleted them. It didn't fix the problem, but it is something else to watch out for in the future.

The next step for troubleshooting would be to narrow down a setting or mod that is a minimal way to reproduce the symptoms.

Note that there are three places where settings are stored for the game. There's default.cfg, local.cfg and user.cfg. Check in each of those paths for matchsettings.mp.json and matchsettings.json, and check in each of those config files for settings related to hero garrison. Backup any data that you remove before removing it so that we can identify the root cause(s) of the problem. If you choose to remove default.cfg then reinstalling the game will restore it to the correct state.

Another clue: Helicity said that he neither hosted a game nor played as a player of a game involving regicide. However, he did spectate a game involving regicide. Developers, is there a way for match settings to become persistent in the game other than matchsettings.mp.json?

It's strange that the problem would even happen without regicide mode enabled. Maybe there is a different root cause that induces the same symptoms as the hero garrison+regicide bug. Helicity, can you please provide a copy/paste of the console messages about the JavaScript errors related to the infinite loop, please?

Edited March 2, 2023 by Norse_Harold

[Silier]

If map settings are not resetting correctly between games or switching maps then yes, it could keep some settings from another map/game.

There is also genral dilema when messing up with settings, if player can do the change to the specific setting in the map group, then it probably should not change when switching maps, so potentially if on one map setting is locked and it is switched to map where it can be changed, it could be carried over.

[Yekaterina]

These were the errors.

[Yekaterina]

Do you think reinstalling 0AD and deleting all config and cache would help?

[gui456wSERTDYF]

Don't know if it helps, but this used to happen a lot to me "randomly" depending on the day (so maybe with some player X on some day I had this issue, the next day it would work perfectly, and so). After I changed my internet provider (not because of this) now it works much better and I still get every now an then this issue but way less often than before. Almost never I would say.

[Stan`]

@Helicity Does that happen if you disable the option as shown in this video ? 

Re-installing the game might not be enough. If you are on Windows they are in %APPDATA%\0ad\config you can type that in the address bar, or in the run command 

[Yekaterina]

1 hour ago, Stan` said:

 Does that happen if you disable the option as shown in this video ? 

I have. It happened with Norse Harold still. I will try to join Ginnungagap tonight if I see him then update you, thanks.

[Norse_Harold]

6 hours ago, Helicity said:

These were the errors.

@HelicityThe screenshot shows the errors that appeared when I enabled regicide and hero garrison mode at the same time. It's not what I was asking about.

Please start 0ad from a Terminal window, connect to a game that I'm hosting with randomly generated map like Mainland, and copy/paste the JavaScript errors from the Terminal window to a forum post.

The reason that this is necessary is because we should not assume that the root cause of the problem is regicide mode + hero garrison. There could be a different cause of an infinite loop during initialization.

[Stan`]

14 minutes ago, Norse_Harold said:

Please start 0ad from a Terminal window, connect to a game that I'm hosting with randomly generated map like Mainland, and copy/paste the JavaScript errors from the Terminal window to a forum post.

If he uses windows this won't work.

To get the "terminal log" you need to run dbgview from sysinternals in parallel (DebugView - Sysinternals | Microsoft Learn) (Or run the game in a debugger)

The mainlog.html might be enough. If you just want the javascript errors, they are all logged in interestinglog.html

[Norse_Harold]

Helicity is on Linux, not Windows.

We can reproduce the symptoms with no mods enabled, simply by setting the number of player slots to 5 or more, or toggling settings like capture the relic very rapidly with 4 slots.

There are no JavaScript errors when the problem occurrs.

When the problem occurs, the reliable data in transit increases to several thousand bytes and does not decrease. Then Helicity gets disconnected after a few seconds. I think that this is caused by packet loss, either on delivery of update packets from the host to Helicity, or on delivery of acknowledgement packets from Helicity to the host.

The problem so far only occurs when I host via a VPN. When I hosted without VPN, Helicity remained connected despite using 8 player slots, and much rapid adjustment of game settings like capture the relic and number of relics.

Helicity got disconnected from another host as a player in a 4v4 game today. People commented that he regularly has connection problems.

My assessment is that there is not a bug in 0ad, but there is a combination of network or computer problems with the players involved. Maybe VPN traffic is bursty, and maybe Helicity's connection or computer has trouble with that.

Edited March 2, 2023 by Norse_Harold

[Yekaterina]

13 hours ago, Norse_Harold said:

People commented that he regularly has connection problems.

Actually the connection problems almost disappeared since this Monday, very suddenly. However, my observation is that I struggle to maintain a good connection to most German hosts, but for hosts in other European countries the connection quality is good.

Often, if I click on a game too quickly, I get STUN error and I am asked to wait a few seconds before trying. Waiting does work in this case and I am not sure whether it is related to this.

In addition, If I drop and try to reconnect, I get the error "player name in use, please try later". Why? I am not in the game anymore but the host's player list still displays me as playing.

On the positive side, my connection is quite stable now (except to some German hosts), so don't call me "LagCity" anymore!

[Silier]

3 hours ago, Helicity said:

. Why? I am not in the game anymore but the host's player list still displays me as playing.

Exactly, because host did not unlist you, host/server thinks there is player with your name and another one is trying to join. Can be timing, maybe try what happens if you try few seconds later, if doesn't help the best would be to open ticket.

[Norse_Harold]

I remember advice about preventing packet loss when using UDP protocol. Because ISPs and Internet backbones are more likely to drop UDP packets that are larger, the advice for real-time network protocols like VoIP and gaming is to use smaller packet sizes. If I recall correctly, about 500 bytes is the maximum recommended.

Maybe some Internet backbones used by Helicity, and the 5 other known players who have trouble staying connected to VPN users, have implemented the Random Early Drop QoS algorithm, preferring to drop larger UDP packet sizes.

Is there a way to easily adjust the maximum packet size used by 0ad?

Even so, it doesn't seem like the "reliable" feature is working as expected. Note that VPNs have a lower MTU than the average Internet connection, by almost 1000 bytes.

I see a packet flag in the ENet library:

   /** packet will be fragmented using unreliable (instead of reliable) sends
     * if it exceeds the MTU */
   ENET_PACKET_FLAG_UNRELIABLE_FRAGMENT = (1 << 3),

Hopefully this is not in use by 0ad, but if it is then it might explain what's happening here.

Edit: there's only one call to enet_packet_create() that I can see, located in source/network/NetHost.cpp. Apparently all packets have the reliable flag, and do not have the unreliable fragment flag.

My new hypothesis is that the number of retry attempts is being exceeded and the connection is being forcefully reset by ENet.

Edited March 3, 2023 by Norse_Harold

[hyperion]

Quick look, enet assumes an mtu of 1400, so what's the output of

ping -M do -s 1372 [norse_harolds ip]

 

[Norse_Harold]

Someone else will have to help with that test, because if I ping myself then it just sends the packets over the loopback adapter.

The problem seems to be fragmented packets that are being dropped some time before they reach Helicity.

Two fragments:

• Packet of size 1420 bytes on wire, 1400 data bytes
• Packet of size 28 bytes on wire, 8 data bytes

These aren't being acknowledged at all by Helicity when Helicity plays with his laptop and school Internet connection.

Today he's using his dad's network, and it seems to be working.

But, I want to avoid having fragmented packets being sent out. How does ENet determine what MTU to use? Maybe it's looking at the wrong interface.

My ethernet interface has MTU 1500 bytes (maybe data).

My VPN interface has MTU 1420 bytes on wire.

ENet is using the default MTU of 1400 data bytes.

I want ENet to use a lower MTU, such as 1392 data bytes.

Edited March 4, 2023 by Norse_Harold

[Yekaterina]

I want to add: this weekend I returned home from school and I shared my father's optical fibre network. I had no trouble joining Norse Harold using my laptop neither. So the cause was partially the school WiFi.

[hyperion]

6 minutes ago, Norse_Harold said:

Someone else will have to help with that test, because if I ping myself then it just sends the packets over the loopback adapter.

Well, you could ping play0ad or whoever instead with the above arguments. In case you are the bottleneck this would suffice. 1400 should work with a vpn but yours might cut it further than strictly required.

11 minutes ago, Norse_Harold said:

My VPN interface has MTU 1420 bytes on wire.

Meaning "ping -M do -s 1392 play0ad.com" succeeds.  If so there is supposed to be no issue on your end and the enet default of 1400.

 

11 minutes ago, Norse_Harold said:

I want ENet to use a lower MTU, such as 1392 data bytes.

You shouldn't unless required but if you want, there is no config in 0ad so you have to patch the code. Just set it right after enet_host_create on the returned struct, then let it run as usual.