[Random Map] Realistic Terrain Demo

[Spahbod]

However, I don't see problems with dense wood if paths are added to connect players.

@ Hight based terrain texture placement

That's part of the idea of this map. What's the problem with that?

True. Perhaps apart from performance issues there won't be any problem with dense woods (although some biome like savannah may need different levels of density still)

The idea itself is not the problem. The problem is with our current textures. They get "tiled" when a large enough patch is created. Also height based textures don't work nicely in all of the instances.

[FeXoR]

Concerning the textures not fitting well: I'd just take different textures and always more than one for each "height level". That way it would work.

Is there any reason why not well matching textures are tied together in the random biome system in one variable? It would be easiest if we could change it there.

Otherwise I have no problems with going through all biomes and pick the textures for this map by hand.

Another thing is that the random biome variables are at least not very descriptive (even if taking into account the documentation):

Code wise it's hard to guess what "rbt12" means. The documentation says: "Other miscellaneous terrain textures" here which is not much better.

For the code I'd be much easier to read if it was something like biome.texture.grass or something.

I don't know what the original idea was so I'm not sure if it'd be better to change the random biome system or just handpick the textures/actors to use for this map.

Edited October 8, 2013 by FeXoR

[historic_bruno]

I don't know what the original idea was so I'm not sure if it'd be better to change the random biome system or just handpick the textures/actors to use for this map.

The original, never implemented idea was http://trac.wildfiregames.com/wiki/Random_Map_Scripting_Interface#BiomeReference which is at least more descriptive, but not much more flexible. Ideally, biomes would be complete packs of textures, actors, entities, and perhaps in the future, sounds, that would make sense for each biome. It should be an extensible system with minimal hard-coding, so new mods could easily add biomes. I don't know whether that would be some data (JSON) or scripts or a combination. And if we're going for realistic maps, there should be ways of specifying relationships between biomes on maps where several might be mixed or overlap. Obviously there wouldn't be a hard transition line between two biomes but a gradual transition and that's where texture blending comes into play. If random maps don't use texture priorities yet, consider that as well.

The current biome "system" should go away as soon as possible

[FeXoR]

The original, never implemented idea was http://trac.wildfiregames.com/wiki/Random_Map_Scripting_Interface#BiomeReference which is at least more descriptive, but not much more flexible. Ideally, biomes would be complete packs of textures, actors, entities, and perhaps in the future, sounds, that would make sense for each biome. It should be an extensible system with minimal hard-coding, so new mods could easily add biomes. I don't know whether that would be some data (JSON) or scripts or a combination. And if we're going for realistic maps, there should be ways of specifying relationships between biomes on maps where several might be mixed or overlap. Obviously there wouldn't be a hard transition line between two biomes but a gradual transition and that's where texture blending comes into play. If random maps don't use texture priorities yet, consider that as well.

The current biome "system" should go away as soon as possible

Thx. Some questions:

And this biome system is:

1.) A planned feature for part 1 (and the team agrees on that)?

2.) Used by Atlas and random maps (and maybe others)?

In this case we'd need something like this (or if not please let me know your plans).

@ biome transitions by texture priority

That's easy to do with the current code of this map. That would even be possible for the textures based on height.

I'm not quite sure what is meant by "texture blending" though. Does it mean textures fade into others at the border of tiles or that one tile can have multiple textures that are fade together or if something else what is it and is it implemented and is it usable/setable for random maps?

Questions and more questions ^^

Edited October 9, 2013 by FeXoR

[sanderd17]

@ biome transitions by texture priority

That's easy to do with the current code of this map. That would even be possible for the textures based on height.

I'm not quite sure what is meant by "texture blending" though. Does it mean textures fade into others at the border of tiles or that one tile can have multiple textures that are fade together or if something else what is it and is it implemented and is it usable/setable for random maps?

See http://trac.wildfiregames.com/wiki/ArtDesignDocument#TerrainTextures for examples. Every tile has a priority (I don't remember if high or low numbers are rendered on top), but in Atlas, you can influence the priorities by left or right clicking when painting. If you take f.e. a 3x3 paint square, and you click left with it, it's blended on top (the borders of your square get the new texture). If you click right, it's rendered underneath (the borders of the square keep the old texture).

Atlas does this by setting the priority to less than the minimum or more than the maximum of the surrounding tiles.

The rmgen/map.js seems to currently use a fixed number for the priority (see line 403)

[Spahbod]

Concerning the textures not fitting well: I'd just take different textures and always more than one for each "height level". That way it would work.

Unfortunately, no. The current random number generation algorithm has a too low period for this work and in the end we'll have tiled textures again. You can try that.

Is there any reason why not well matching textures are tied together in the random biome system in one variable? It would be easiest if we could change it there.

Because the current textures are picked as to be "patches" on the ground, not some standalone terrain. Also some of the textures are intended as "blending" textures and should not be used alone at all. This was done with the current philosophy of random maps (painting clumps of textures here and there).

Otherwise I have no problems with going through all biomes and pick the textures for this map by hand.

That would be a better idea because in addition to the problem mentioned above, some good textures were omitted from the current list because they did not go with the "clump" philosophy. But they could be very good textures for your system.

Another thing is that the random biome variables are at least not very descriptive (even if taking into account the documentation):

Code wise it's hard to guess what "rbt12" means. The documentation says: "Other miscellaneous terrain textures" here which is not much better.

For the code I'd be much easier to read if it was something like biome.texture.grass or something.

And this biome system is:

1.) A planned feature for part 1 (and the team agrees on that)?

2.) Used by Atlas and random maps (and maybe others)?

In this case we'd need something like this (or if not please let me know your plans).

This is on my list to be changed (to become similar to the system historic_brunno mentioned). Although speeding up some specific maps (islands, anatolian plateau, etc.) and the completion of the new clump placer are of higher priority now.

Also the documentation needs another update. the problem with biome.textures.grass is that not all biomes are going to have grass, and some will have more than one such texture.

I am thinking of a whole new library with a different approach. Instead of placing clumps of textures, trees, etc. again and again (which is very very inefficient), it will determine the status of each tile only once. In this system the main map file would only determine the water/unique_cliff shape and player placement. The rest will be done in library functions. Still I don't know if I can finish it before summer.

[FeXoR]

Thanks everyone for the input!

I guess I'll go for handpicked textures then.

Some more questions:

Is anyone working on shared library support? Is it wanted? If so, how?

Is it planned to add customizable texture fading to RMGEN? Is it wanted at all? If so, how?

Edited October 9, 2013 by FeXoR

[niektb]

Is there any progress on this? (I guess it needs an update to the latest RMGEN functions?)

I like to see to get such functionality in-game.

For example put in a heightmap and this script takes care of texturing, resources and all. Or that this script takes care of the evelation and that you (as designer) take care of the rest etc., etc.

Edit : Maybe a useful paper on generating realistic terrain:

http://oddlabs.com/download/terrain_generation.pdf

Edited March 8, 2014 by niektb

[FeXoR]

I just started to work on this again (other priorities in the last months).

My main concern is the texture blending ATM.

I fear there is no way to make randomly distributed tiles of two or more types in the same area look well (just by blending). So I might implement a function to take care of removing single (or 3 or more connected tiles) tiles surrounded by others (Seams like I ran into a percolation like problem). That is most promising I guess.

The reference of yours is a good one though I doubt we would use all of that because some of those functions are really slow and need to be applied multiple times (especially erosion).

If you are interested in erosion you could try to check out my (many, never really working) approaches here: http://www.wildfiregames.com/forum/index.php?showtopic=16233&page=5#entry274289

I intend to make a lib from all of this but I'm not qute sure which functions are worth to add there. Feel free to use them as you see fit. If you need a specific function and can't figure it out I'm willing to help.

I'm likely to finish this map (realistic terrain demo) within the free period (but you never know what gets in the way ^^).

Edited March 8, 2014 by FeXoR

[niektb]

It thought of the following map:

Schwarzwald

This is a map with a rich valley in the middle filled with a lot of resources. Players start on the hills near the edge of the map. This is a map with a high forest density (Like AOEII's Black Forest map)

The only thing I need to know is how to bias the realistic terrain generator to create a valley. Furthermore do I want to know how you made sure that the levels are playable.

I noticed BTW that a lot of other RM-scripts use a heightbased texture painting too without looking very tiled. I'll try and see if it works for me.

[FeXoR]

It thought of the following map:

Schwarzwald

This is a map with a rich valley in the middle filled with a lot of resources. Players start on the hills near the edge of the map. This is a map with a high forest density (Like AOEII's Black Forest map)

The only thing I need to know is how to bias the realistic terrain generator to create a valley. Furthermore do I want to know how you made sure that the levels are playable.

I noticed BTW that a lot of other RM-scripts use a heightbased texture painting too without looking very tiled. I'll try and see if it works for me.

You can influence the general shape of the map by giving a rough initial heightmap (e.g. 3x3) like:

initialReliefmap = [    [heightRange.max, (heightRange.max + heightRange.min) / 2, heightRange.max],    [heightRange.max, heightRange.min, heightRange.max],    [heightRange.max, (heightRange.max + heightRange.min) / 2, heightRange.max]];

and then give it to getBaseTerrainDiamondSquare like:

var myReliefmap = getBaseTerrainDiamondSquare(g_Map.size + 1, heightRange.min, heightRange.max, 0.5, initialReliefmap);

You will have to apply the reliefmap to the g_Map if you finished manipulating it:

setReliefmap(myReliefmap);

So if you use the realistic terrain demo just add the initialReliefmap code (search for initialReliefmap to see where. the other preset initial heightmaps are documented out and will form islands).

It then will look similar to this (tiny, 2 players, seed 1):

(The diagonal appearance comes from the start locations being added that way, not from the initialReliefmap. The right and left side medium height "valleys" however do.)

The getBaseTerrainDiamondSquare function doubles the size until it's bigger then the needed size and then cuts the bottom left part of the needed size. That will change in the future so the center is used rather then the bottom left. So until now on some map sizes the lake will not be in the center of the map.

Additionally keep in mind that the shape is most suitable for 2 players. You might want to make the initialReliefmap depend on the number of players here because it's not radial symmetric.

Also you should change on which height the start locations are added and add more tree covered "height levels" (textueByHeight array).

Hope I could help.

(Could someone PLZ make the post editor stop adding/removing spaces tabs and newlines insanely?)

Edited March 9, 2014 by FeXoR

[niektb]

I tried to strip off the texturing part of RealisticTerrainDemo and replacing it with the texturing part of Deep Forest. However I encountered this error:

ERROR: JavaScript error: uncaught exception: createSimpleTerrain expects string as input, received 0

The strange part is that I do not even call that function.

It goes wrong at this piece of code:

var ByHeight = [];ByHeight.push({"height": heightRange.min + 1/3 * (waterHeightAdjusted - heightRange.min)});ByHeight.push({"height": heightRange.min + 2/3 * (waterHeightAdjusted - heightRange.min)});ByHeight.push({"height": heightRange.min + 3/3 * (waterHeightAdjusted - heightRange.min)});ByHeight.push({"height": waterHeightAdjusted + 1/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 2/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 3/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 4/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 5/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 6/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 7/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 8/9 * (heightRange.max - waterHeightAdjusted)});ByHeight.push({"height": waterHeightAdjusted + 9/9 * (heightRange.max - waterHeightAdjusted)});//placing basesfor (var i=0; i < numPlayers; i++){	playerAngle[i] = (playerAngleStart + i*playerAngleAddAvrg + randFloat(0, playerAngleMaxOff))%(2*PI);	var x = round(mapCenterX + randFloat(minPlayerRadius, maxPlayerRadius)*cos(playerAngle[i]));	var z = round(mapCenterZ + randFloat(minPlayerRadius, maxPlayerRadius)*sin(playerAngle[i]));	playerStartLocX[i] = x;	playerStartLocZ[i] = z;	// Place starting entities		myReliefmap = rectangularSmooth(x, z, 25, 25, 1, myReliefmap, (ByHeight[4].height + ByHeight[5].height) / 2);	placeCivDefaultEntities(x, z, i+1, BUILDING_ANGlE, {"iberWall": false});	// Place base texture	var placer = new ClumpPlacer(2*baseRadius*baseRadius, 2/3, 1/8, 10, x, z);	var painter = [new LayeredPainter([terrainBaseBorder, terrainBase, terrainBaseCenter], [baseRadius/4, baseRadius/4]), paintClass(clPlayer)];	createArea(placer, painter);	// Place starting resources	var distToSL = 10;	var resStartAngle = playerAngle[i] + PI;	var resAddAngle = 2*PI / startingResources.length;	for (var rIndex = 0; rIndex < startingResources.length; rIndex++)	{		var angleOff = randFloat(-resAddAngle/2, resAddAngle/2);		var placeX = x + distToSL*cos(resStartAngle + rIndex*resAddAngle + angleOff);		var placeZ = z + distToSL*sin(resStartAngle + rIndex*resAddAngle + angleOff);		placeObject(placeX, placeZ, startingResources[rIndex], 0, randFloat(0, 2*PI));		addToClass(round(placeX), round(placeZ), clBaseResource);	}}

Edited March 9, 2014 by niektb

[niektb]

I was able to narrow the problem down to these lines of code:

// Place base texturevar placer = new ClumpPlacer(2*baseRadius*baseRadius, 2/3, 1/8, 10, x, z);var painter = [new LayeredPainter([terrainBaseBorder, terrainBase, terrainBaseCenter], [baseRadius/4, baseRadius/4]), paintClass(clPlayer)];createArea(placer, painter);

Any thoughts on this?

Edited March 9, 2014 by niektb

[FeXoR]

Are terrainBaseBorder, terrainBase, terrainBaseCenter, baseRadius defined in your script?

(Without the hole script it's hard to tell)

Edited March 9, 2014 by FeXoR

[niektb]

Thanks, I fixed it. Had too much quotes at the defining of terrainBase.

[niektb]

Okay, now since I'm trying to use the other texture mechanics I need to know how to select existing parts of the map by height and paint them with a class.

Paint with a class isn't too difficult. That just the following line:

paintClass([Classname]);

That little piece of code should be in a createArea call, with itself features a placer and a painter. In that painter the paintClass piece should be added.

But how do I select a height (For example all waterareas (= all places with height < waterheight).

Secondly I'm wondering whether there is a way to select tiles by slope rather than by height.

[FeXoR]

Okay, now since I'm trying to use the other texture mechanics I need to know how to select existing parts of the map by height and paint them with a class.

Paint with a class isn't too difficult. That just the following line:

paintClass([Classname]);

That little piece of code should be in a createArea call, with itself features a placer and a painter. In that painter the paintClass piece should be added.

But how do I select a height (For example all waterareas (= all places with height < waterheight).

Secondly I'm wondering whether there is a way to select tiles by slope rather than by height.

If you are generating a new map open a new topic for that. I'll answer there.

[Radagast.]

15:14 FeXoR> Philip`: What do you think whould be better suited for a replacement for the gaussian:

(cos()+1)/2 from -Pi to Pi

OR

(sin()+1)2 from 0 to 2*Pi

or a

[4th] 5th [sic.] order polynome (center 1, start 0, end 0, start derivatiion 0, end derivation 0)

?

Is this still relevant?

• Compile time optimization possible? Can compiler replace divisions by multiplication and shifting?
  • yes, because it's only constants or integers!
    • => DON'T OPTIMIZE. Compiler will (probably) do.
  • no, because it's floating point.
    • Is calculation in a frequent loop or heavily called function at runtime?
      • -> optimize.
    • Otherwise DON'T OPTIMIZE. Compiler will not do, but it's not worth it.

trigonometric functions imply floating point. So if you think the above is true. Then try to get rid of your division. Also note the 2 as 2.0 or 2.0f for float for clarity.

Then do your optimization manually:

So this

(cos()+1)/2 from -Pi to Pi

becomes:

(cos()+1) * (1.0/2.0) from -Pi to Pi

or if you prefer:

(cos()+1) * .5 from -Pi to Pi

For the others I'm not sure. It finally all depends on how well the trigonometric function is doing ...

If you'd had only integer/ie. non-floating point then division by 2 (== 2^1) might be put as

int >> 1

division by 2^2 as:

int >> 2

multiplication by 2 (== 2^1) then could be:

int << 1

by 2^2 as:

int << 2

If you are very interested in these topics or want to know what exactly the compiler is doing and why performance profiling might depend on the OS and the compile mode (Debug vs. Release, the latter producing better results with your own optimization e.g. for the floating point case) or want to know the Goldschmidt algorithm, ... what AMD currently uses, then a good place to start might be e.g. here.

Edited March 16, 2014 by Hephaestion

[Radagast.]

(Could someone PLZ make the post editor stop adding/removing spaces tabs and newlines insanely?)

As a workaround you can switch of the WYSIWYG editor, besides colour coding BBCodes or HTML yourself is quite straight forward.

[sanderd17]

AFAIK, that code was all JS. So there are no types, which means noting something as 2.0 isn't needed.

Also, the compiler can do strange things when it comes to optimisation, and you'll notice that none of your replacements will make any difference. Not even in micro benchmarks. And it's not even sure the compiler will kick in.

These aren't really optimisations that will make the RM generator faster. Instead, the placement algorithms have to be made better, but that needs template information.

[Radagast.]

Ha, good having some eagles' eyes around. Thanks for clarifying.

Also, the compiler can do strange things when it comes to optimisation, and you'll notice that none of your replacements will make any difference. Not even in micro benchmarks. And it's not even sure the compiler will kick in.

Agreed, compilers do strange things, switch optimization off if you want a rigid system. Apart from that for typed languages and floating point arithmetic which I apparently made not clear enough, the x * (1.0/2.0) multiplication instead of division x / 2.0 makes a real difference.

I just forget about JavaScript. So what I pointed out actually is completely unrelated as it can't help out then. Thanks for the hint Sander.

And excuse my confusedness, i have to pay better attention of my long beard not getting stuck in the thorn bushes too often.

Instead, the placement algorithms have to be made better, but that needs template information.

Interesting. I hope FeXoR gets this template structures soon. Does it relate to making parts of our simulation available for Random maps to use?

[FeXoR]

Hephaestion: Well, I learned something (at least for relevant for Python I use most of the time) ^^

What I was mainly asking for is a fast well fitting function to replace the gaussian with.

As a windows function I use a combination of a 2nd order polynomial and a trapeze (looks OK and is quite fast):

var scaleX = (1 - (wx / dx - 1) * (wx / dx - 1) + min(min(3 * wx / dx, 3 * (2 * dx + 1 - wx) / dx), 1)) / 2;var scaleY = (1 - (wy / dy - 1) * (wy / dy - 1) + min(min(3 * wy / dy, 3 * (2 * dy + 1 - wy) / dy), 1)) / 2;

(I could replace the "/ 2" with "* 0.5" but that's not the time consuming thing here anyways)

The template information indeed would be very nice (though mainly for wall placement and debugging).

Edited March 17, 2014 by FeXoR

[Radagast.]

wall placement

wow, yet another surprise, didn't know you planned that, too. Already analyzed China's wall? Will you make the walls run along hilltops?

I got suspicious of the min and max as they are function calls, so I googled it: They say it's a 35% saving to replace min(_, 1) with if else or even better use ternary operator. The not so good readable but speedy:

<condition< ? <met> : <not_met>

The blogpost is quite interesting, it also deals with python's slow gauss and how they replaced it:

Use Numpy for generating random numbers

Another significant improvement in speed came when I switched from the gauss() function of the built-in random library to the normal() function from numpy.random. Using a similar script to the one above, I found that the numpy version is about four times faster than random.gauss(). However, generating a random number from a uniform distribution (random.uniform) is very fast. In the source code of random, you will note that drawing one random number from a Gaussian distribution requires computing a square root, a log, a cosine, and a sine in Python. Numpy is faster because it does the math in C.

Also, can't we abschätzen a bit more? Those - 1 and + 1 look suspicious if wx and wy are meant to be non-floats. Though I don't know the details.

Dom't you think the min() and max() has to evaluate every parameter (expression) given while an if else usually often is lucky and has only to examine one expression? Another reason to replace it by if else? I'm not sure myself.

I'm afraid this would make the solution look much less elegant though.

[FeXoR]

Hephaestion:

- "If" instead of "min" would not likely be faster here because the if condition would be the same calculation.

- The "1 -" at the beginning is for the negative square function (before the "+") that should give a probability between 0 and 1. Similar the "- 1".

- The "+ 1" in the second part (after the "+") is just to make sure the probability is never 0 (would do nothing).

(ATM I go from the center to the edge of the window. Reversing that could remove some "+/- 1" but I would find that counterintuitive)

(I could get rid of the "/ 2" by just halving both parts individually but the way it is it's simple to combime multiple windows functions. I like that.)

- I don't plan walls on this map but I wrote the RMGEN wall_builder.js lib (where every wall element length is "hardcoded" again due to the lack of entity access which is bad ofc.).

Edited March 18, 2014 by FeXoR

[Radagast.]

@FeXoR:

"If" instead of "min" would not likely be faster here because the if condition would be the same calculation.

Correct. I misinterpreted the equation. It will end up in one condition. It's even possible the "min" function calls will be gone (thus overhead too) anyway after inbuilt optimization of the interpreter (if it's doing any of course).

The "1 -" at the beginning is for the negative square function (before the "+") that should give a probability between 0 and 1. Similar the "- 1".

That clarifies it all.

If it's counterintuitive then don't worry, your solution is fine. I even doubt the division by 2 could help, division by dx and dy will probably often be the bottlenecks.

Anyway, it looks like your calculations are already quite thought-through.

wrote the RMGEN wall_builder.js lib

Great! The wall builder can be improved once a more dynamic approach is possible. Would it be usable for generating hillchains too? Could emulate glaciers and their consequences then. Or perhaps generate shores/coastlines with it.

Do you think we could employ some of your path builder algorithms in the engine? e.g. if a mayor decides to build a road, then it should float naturally (as it possibly mostly did in ancient times).

Hopefully the rock and cliffs will be possible soon. I really would like to see how we could generate underwater characteristics randomly too.You finally emulate the real world. Even caves seem possible for your algorithms.