Fatal error while generating maps from size 1024 up

[enko]

If I try to generate random maps by the size of 1024 or bigger I receive an error message (Error generating random maps. Check application log for details.) and the loading process fails. The only exception is the "Latium" map, obviously due to the fact it's area is smaller by its quadratic form. However the "Latium" map fails at sizes of 2048 or higher as well.

To reproduce the mod add the following folders and files in the Mod-folder. The mod is attached as a zip-file.

Spoiler

MOD-FOLDER / MapsForTitans / mod.json

{
	"name": "mapsfortitans",
	"version": "0.0.2",
	"label": "Maps for Titans",
	"url": "https://play0ad.com",
	"description": "Increase map sizes.",
	"dependencies": []
}

MOD-FOLDER / MapsForTitans / simulation / data / settings / map_sizes.json

{
	"TranslatedKeys": ["Name", "Tooltip"],
	"Data":
	[
		{
			"Name": "Tiny",
			"Tooltip": "This map size has barely any room for even the most basic resources.",
			"Tiles": 128
		},
		{
			"Name": "Small",
			"Tooltip": "Small maps are suitable for 1v1 matches, as players can reach the enemy quickly while still having some room to expand.",
			"Tiles": 192
		},
		{
			"Name": "Medium",
			"Tooltip": "Medium-sized maps provide plenty of space for two players, but even eight players fit.",
			"Tiles": 256,
			"Default": true
		},
		{
			"Name": "Normal",
			"Tooltip": "This is the ideal mapsize for 4 to 8 players, because there is plenty of space to expand while the neighbors are quickly reachable.",
			"Tiles": 320
		},
		{
			"Name": "Large",
			"Tooltip": "The large mapsize is recommended when great portions of the map are covered with water or inaccessible mountains.",
			"Tiles": 384
		},
		{
			"Name": "Very Large",
			"Tooltip": "Very large maps are not recommended as it takes very long to reach the enemy and the game performance can be reduced.",
			"Tiles": 448
		},
		{
			"Name": "Giant",
			"Tooltip": "Very large maps are not recommended as it takes very long to reach the enemy and the game performance can be reduced.",
			"Tiles": 512
		},
		{
			"Name": "Titan 1024",
			"Tooltip": "Chose this if you don't want to finish map generation.",
			"Tiles": 1024
		},
		{
			"Name": "Titan 2048",
			"Tooltip": "Chose this if you don't want to finish map generation.",
			"Tiles": 2048
		}
	]
}

The error log shows this information:

Quote

Generating Latium of size 2048 and 8 players.
Creating players...
0.000s.
Creating river...
1.765s.
Creating river...
1.240s.
Painting elevation...
14.869s.
Painting terrain...
ERROR: JavaScript error: uncaught exception: out of memory
ERROR: CMapGeneratorWorker::Run: Failed to load RMS 'maps/random/latium.js'
Loading config file "config/user.cfg"
Loaded config string "gui.gamesetup.aibehavior" = "balanced"
Loaded config string "gui.gamesetup.aidifficulty" = "2"
(‌…)
Loaded config string "view.zoom.speed.modifier" = "1.05"
Loaded config string "view.zoom.speed.wheel" = "32.0"
Loaded config string "windowed" = "true"
Shutting down profiler2 GPU mode
Engine exited successfully on 2019-09-15 at 09:18:34 with 873 message(s), 2 error(s) and 0 warning(s).

The mod has been tested on 2 different computers with several Linux based operating systems (using openjdk). A lack of hardware resources is excluded.

MapsForTitans.zip

[feneur]

2 hours ago, enko said:

A lack of hardware resources is excluded.

 

2 hours ago, enko said:

ERROR: JavaScript error: uncaught exception: out of memory

I'll leave it to the programmers to say anything definitive, but those two quotes seem to conflict =) It would probably help them solve the issue/find what's the cause if you would include the systeminfo.txt file (see https://trac.wildfiregames.com/wiki/GameDataPaths for more info on how to find it).

[Stan`]

The game is 32bit so it can only use up to 4GB of RAM. I guess generating a Giant latium creates too many objects and runs out of memory. There is some work to be done in this area @elexis might know more.

[enko]

Indeed "out of memory" and "no lack of hardware resources" sounds like a conflict, but it is no hardware problem for sure. If the game can use just 4 GiB of RAM, then this sounds like the problem for me. How can I check this? The systeminfo.txt of the weaker machine is attached.

By the way, this is a great game. I find it very sad that there are so many basic technology parts not up-to-date. Of course there is the fact, that the development started long time ago. But shouldn't it be a major strategy point of development to raise up the basics to the modern machines?

system_info.txt

[smiley]

The map generator will only use 93 MB.

[enko]

Is there a way to increase this value?

[smiley]

Not unless you patch engine code and recompile it yourself.

Certain maps sized 512 takes over 6 minutes to generate. So, if your 2048 sized map doesn't run out of memory, you would still have to wait for an eternity.

Unfortunately, the 0AD map generation system has a terrible design problem. And fixing it is taking a lot longer than one first imagined.

Edited September 15, 2019 by smiley

[Stan`]

24 minutes ago, enko said:

. I find it very sad that there are so many basic technology parts not up-to-date. Of course there is the fact, that the development started long time ago. But shouldn't it be a major strategy point of development to raise up the basics to the modern machines?

That's the catch of a volunteer project ...

[enko]

22 minutes ago, smiley said:

And fixing it is taking a lot longer than one first imagined.

That is what I mean. I am afraid if the basics are not OK, than the volunteers invest a lot of time in a code which must be complete trashed, because it is more easy to write it new, in worst case.

In case of "Latium" I need 12 seconds for a 512 map and 38 seconds for a 1024 map. For a lot of maps this won't be a big problem, but these more bigger maps offer a lot of fun.

If it is able to patch, is it possible to implement this in the next release? In my opinion an open-source software should not implement unadjustable boundaries.

[smiley]

2 hours ago, enko said:

If it is able to patch, is it possible to implement this in the next release? In my opinion an open-source software should not implement unadjustable boundaries.

I just changed it to scale with the actual map size. Thanks for bringing it up.

Spoiler

Quote

is it possible to implement this in the next release?

That I cannot guarantee, I don't work on 0AD, but a fork of it. Maybe 0AD would eventually merge that in.

 

[enko]

Can you tell me where I have to implement the changes, in case I have to build it on my computer?

[smiley]

https://code.wildfiregames.com/source/0ad/browse/ps/trunk/source/graphics/MapGenerator.cpp$43

Just change that line to something higher. That won't add the scaling with the sizes part though.

Edited September 15, 2019 by smiley

[enko]

Thank you, smiley, for the code and thanks to all for the support. I will report you my results as soon as I tested it. The issue is not solved, because it is not implemented in the regular code, but I think we have a solution.

[smiley]

Not a solution unfortunately. On my end, it doesn't really run out of memory even without increasing the runtime size. But it still takes a while to generate. But like I said, its bad design. On top of that, its highly cpu unfriendly. _Lots_ of cache misses, and those are expensive.

Good enough for a workaround I guess. But why such a large size though? The largest map size in AoE2 was apparently 256

Edited September 16, 2019 by smiley

[enko]

I already figured out several issues of the game. And the main problem seems to me that the software do not use all the system resources in a proper way.

Usually, I play high economy-based, without unit-limits. Therefor it happens that the map gets too fast too small for all players. Not enough resources, not enough space. Moreover, the bigger maps invent a complete new way of playing. For example, you have to place your units and cities on strategic places on the map, because the time to travel through the whole mat would be too long. It is more close to a "open world".

I have several ideas for mods in combination with the bigger maps, but I didn't find the time to implement it, yet.

[Loki1950]

You are right of course first off we still are 32-bit app can not address more than 3.8 Gb of memory then we still are written for a single core CPU so updating for those is not a trivial task with the current code base needs a complete re-factoring and we just do not have the resources(programmers) for that we are an entirely volunteer project we can't just throw more money at it as a commercial developer would do we have to recruit them very creatively they have to want to do something.

Enjoy the Choice      

[Stan`]

One solution would be to create child processes. 

As for the threading when D14 and #3700 are in there will be not many things to thread