Release Preparation of A27.1

[Itms]

Hi everyone,

We are preparing the first "patch release" of 0 A.D., now that our workflow allows us to do that. We need your help testing it!

What is a patch release?

A patch release is a minor release of 0 A.D., bringing a few selected fixes from the development version into the latest release of the game. All fixes of the patch release are already present in the development version (the future Release 28).

There are two important aspects to a patch release:

1. Patch releases are OOS-compatible with major releases. If your friend has not heard about the patch release, or cannot install it for whatever reason, they should still be able to play multiplayer with you. This limits the kind of fixes we can include in a patch release.
2. Patch releases are prepared on the side, and do not take precedence over our work towards the next major release. We prepare and provide patch releases as our resources allow. This limits the scale of the fixes we can include in a patch release.

What is the current status of A27.1?

We have setup all the infrastructure for generating patch releases, and a first testable version is ready for you. It is still incomplete, and thus labeled as "Release Candidate Zero" (RC0).

In A27.1, we wish to include, but those are not yet in RC0:

• a fix for the flood on non-flood maps reloaded from a save
• a fix for modifying your lobby password when you have an uppercase letter in your username

In RC0 are included:

• a fix for the large performance issue observed by some players on older hardware
• several small performance improvements in the calculation of simulation hashes (related to multiplayer stuttering)
• a fix for the distorted 3D models with GPU Skinning
• fixes for crashes with Vulkan and/or with GPU Skinning
• a fix for the game crash on pressing Fn key
• a fix for a crash in Atlas when the map generation fails
• fixes for multiplayer port forwarding
• package fixes for Linux desktop environments
• various build fixes for Linux (some sent by package managers)

A27.1 will not include:

• a fix for the Britons OOS on rejoin: unfortunately it would make A27.1 incompatible with A27 (against rule 1 above)
• any modification of simulation hashes: it could remove stuttering entirely, but it would make A27.1 incompatible with A27 (against rule 1 above)
• incremental simulation hashes: those improve stuttering as well, but are too complex to backport (against rule 2 above)

How can I test it?

You can download the release bundles of A27.1-RC0 at https://releases.wildfiregames.com/rc/

Please report below any crashes or new issues (that weren't already present in A27, unless they are listed as allegedly fixed above).

Thanks for your help and support!

Edited yesterday at 17:46 by Itms
Forgot to list the Britons OOS in the incompatible fixes

[borg-]

Congratulations development team for their commitment and dedication.

[sj113]

How can I check if it installed correctly?

because after installation it still displays alpha 27.

[Lopess]

Congratulations to everyone on the team for the new update, I'm happy with this new approach!

[Seleucids]

I am currently playing on the RC. It feels much much smoother than A27 and I have experienced no random crash / stuttering so far. The minimum fps has always been very high. Good improvement! 

However, Atlas crash still persists, especially on Wayland.

 

[Itms]

1 hour ago, sj113 said:

How can I check if it installed correctly?

because after installation it still displays alpha 27.

In the bottom of the main menu, it should display (release-a27.1, 68d99) instead of (release-a27, 8c96b)

[Itms]

1 minute ago, Seleucids said:

However, Atlas crash still persists, especially on Wayland.

Is it already a known issue? If not, please open a ticket on Gitea, if yes we'll try and take a look.

[Seleucids]

5 minutes ago, Itms said:

s it already a known issue? If not, please open a ticket on Gitea, if yes we'll try and take a look.

There are many variations of the crash errors. I will put the issue onto Gitea. This RC gives a different error to the A27-0 version. 

[Seleucids]

The errors and crashes specific to bundlable-a27.1, 86c33

(Linux, KDE Plasma Wayland)

https://gitea.wildfiregames.com/0ad/0ad/issues/7952

https://gitea.wildfiregames.com/0ad/0ad/issues/7951

https://gitea.wildfiregames.com/0ad/0ad/issues/7950

[Emacz]

would it still be the same lobby?  Lobby looks pretty dead right now  almost like some people are on 27.0, others 27.1

[Dunedan]

5 hours ago, Emacz said:

would it still be the same lobby?

yes

[ffm2]

I had a OOS again. They are very rare though. Here are my logs+replay and riccis. I was observer, ricci host. Both on self compiled a27.1 versions. I was the only one OOS.

ooslog_2025-05-24.zip

[Itms]

7 hours ago, ffm2 said:

I had a OOS again. They are very rare though. Here are my logs+replay and riccis. I was observer, ricci host. Both on self compiled a27.1 versions. I was the only one OOS.

Thanks for the report, but the dump difference shows that you have a behavior difference in GUIInterface enabling range visualizations (for attacks, auras, and healers). This is definitely caused by one of your mods (I suppose it's Ricci's "rangedoverlay" mod, but if you're the one OOS, it can be one of your mods). You folks should stop abusing the compatibility checks system, or at the very least test RCs without mods.

[Seleucids]

1 hour ago, Itms said:

a behavior difference in GUIInterface enabling range visualizations

The range visualisations shouldn't affect the game state when computing simulations, right? It is a pure visual element. Therefore it shouldn't be a part of OOS. The same applies for alarms suppression queue, which shouldn't be reported as OOS if it's different for each player. 

[ffm2]

That's what I thought. The weird thing is I had ffm_4k, ffm_visibility, feldmap, so reliable tested mods. Ricci had ranged overlay. But I was the only one OOS. Maybe everyone else had it too.

[Itms]

Currently, the RangeOverlayManager is serialized, so a difference will create OOS. Take a look yourself at the difference between the two oos_dump.txt files.

[ffm2]

When I run my replay (not riccis) again with my (the same) mods I get the hash of ricci (and the others) and I am OOS. I'm sure I didn't flare at that time. I was observer and may toggled team colors. But when I toggle team colors in other replays I don't get OOS.

So I think the ranged overlay mod is not causing the OOS.

Edited 15 hours ago by ffm2

[Deicide4u]

4 hours ago, ffm2 said:

So I think the ranged overlay mod is not causing the OOS.

Regardless, testing of new software should be done in a blank-slate state, without any additional code. This is an established practice in software development, to better detect and isolate buggy code. Doing it any other way is just a waste of time, as you're all finding out now.

[ffm2]

Yes, I disabled all mods but feldmap now and will report once it shows up again.

However:

-RangeOverlayManager is not serielazed (see the Serialize function)

-I was in the game from the start (no rejoin)

-the binary dumps were identical

[ffm2]

elexis has to add this:

 

ffm was present from the beginning, did not rejoin
everyone except ffm had the same hash on turn 1560
ffm had the same issue one match later
ffm uses only 4k, feldmap and visibility mod

sim state was from 3 turns later:
oos_dump.txt
oos turn: 1560
net client turn: 1563
sim turn: 1563

binary simstate is identical!
textual simstate contains only differences in RangeOverlayManager.enabledRangeTypes local entities

but both states are 3 turns too late, and its unlikely the mechanism will ever write simstate dumps from the correct turn, because the turn lengths are so short and because multiple turns are being computed in advance as far as I know.

"Currently, the RangeOverlayManager is serialized"
False: RangeOverlayManager.prototype.Serialize = null;
The textual output showing this is a regression introduced in 9fc6c3c89769a3, pointed out in #7634 and stated to be addressed by c475cc22652528ce7014d

"is definitely caused by one of your mods" pure conjecture
ffm did not use riccis mods, ricci computed the same hash as everyone else but ffm!
riccis hash is also computed without mods

"Doing it any other way is just a waste of time, as you're all finding out now."
you can be grateful if you get an OOS report at all, players reporting that is rare, you dont get many chances to identify an OOS, so if you want to dismiss an OOS report, you better investigate it first and not disregard it easily. it's possible it's caused by a broken mod but if you dismiss it without investigating it, you might be ignoring a valid report which can cause the bug to go not unnoticed but unreported for months.

only difference in commands.txt besides the mods in the first line:

--- commands_ffm.txt    2025-05-24 12:17:26.000000000 +0200
+++ commands_ricci.txt    2025-05-24 12:21:51.000000000 +0200
-hash cc1eb515c7554393f2a2dae2ca4770e6
+hash a763d2495ddab5956941ea577d3c6f30

another bug, not only in JS GUI but also the turn manager should not allow this:
turn 3433 200
cmd -1 {"type":"stop","entities":[],"queued":false}
cmd -1 {"type":"stop","entities":[],"queued":false}

when ffm replays his own replay:
Executing turn 1560 of 5970
ERROR: Replay out of sync on turn 1560

when I replay ffms replay without mods, also:
hash MISMATCH (a763d2495ddab5956941ea577d3c6f30 != cc1eb515c7554393f2a2dae2ca4770e6)
Turn 1560 (200)...

So ffm did not rejoin, yet he was the only one to compute something different on turn 1560 but on turn 1563 the difference was gone already and he had this phenomenon on multiple matches.

I also recommend never to release 0ad with OOS being not fixed, and therefore you should not remove 0a4bfefb1e5f40d93180690e328b11e148dec0cb but include this in the re-release and to bump the mod version in mod.json from "0.27.0" to "0.27.1".
You're saying you release "0.27.1" so the mod version showing "0.27.0" seems misleading.

Bumping the version should (TM) mean that due to the mod compatibility check in the lobby, 27.1 players can play in the same a27 lobby, but they can only join 27.1 games, while the 27.0 players play in the same lobby and can join only 27.0 games.

A reason against doing it this that was brought up by sera was that you find yourselves unable to provide a release for all platforms and thus some players would be stuck to the old version for months (and thus would not be able to find players to play with).

but if that problem was real, then you'd have the same problem with releases, not only re-releases and it doesn't stop you from releasing either.

if that OOS 0a4bfefb1e5f40d93180690e328b11e148dec0cb was fixed in 27.1 and the version would have been bumped, then now you wouldnt have to investigate if it could be caused by the modifiers cache OOS.

Worse: players get used to that OOS occurring and this means they will not report OOS anymore because they think its the same OOS.

[Deicide4u]

31 minutes ago, ffm2 said:

when ffm replays his own replay:
Executing turn 1560 of 5970
ERROR: Replay out of sync on turn 1560

when I replay ffms replay without mods, also:
hash MISMATCH (a763d2495ddab5956941ea577d3c6f30 != cc1eb515c7554393f2a2dae2ca4770e6)
Turn 1560 (200)...

@ffm2 You both should just test the RC without mods, instead of constantly trying to prove that mods weren't at fault for it.

Replaying the game without mods is not the same as playing the game without mods. 

Edited 9 hours ago by Deicide4u

[ffm2]

Yes, I told you I do it from now on. I just had a OOS with only feldmap on.

[ffm2]

Here is my OOS log and the one of diagonalo. Also included are the 2 replays of us. I apologize on behalf of diagonalo that he used mods, he was not aware that we were looking for bugs. I was the only player OOS though. I was present from the setup. I was observer. I did not flare, I did not chat. There was a flare during the OOS though from Flammea which could be a hint.

Ricci was not present at all.

0ad_OOS-2025-05-25.zip

[Itms]

Hi @ffm2, thank you very much for the steady test results. It will be far easier to analyze them without mods on your side, no worries about playing with others using mods.

I had indeed missed #7635, I understand better now.

I am going to take a closer look at your latest OOS report during the week and I'll get back to you.