Jump to content
Palaxin

Palaxin's Heightmap Guide

Recommended Posts

Introduction

Spoiler

If you do not already know the heightmap import feature of the Atlas map editor, please read the official wiki page first before continuing here. It is especially stated there that “[c]urrently the heightmap importer doesn’t have any configuration, so if import doesn't produce the desirable result, the source image should be edited in an image editor and re-imported.”

This guide will give you a helping hand with the required image editing. In the Beginner section you will find the basics everyone needs for heightmap manipulation with easy step-to-step instructions and many illustrations. The Advanced section includes topics not everyone may use, but the necessary steps are still not too complicated for anyone who doesn't like coding. In the Expert section I will even show how to use direct hacks to the PMP file which stores the map height data. However it doesn't contain strict rules and you need to be able to write and run scripts.

I hope you enjoy this guide and strongly appreciate any constructive criticism, proposals, discussions and comments especially since this is my first guide :) At this point I would like to thank @FeXoR, @Garaf and @niektb.

Basics

Spoiler

Software

This guide comes without any liability regarding the used software. It might damage or cause instabilities to your system. My operating system is Windows 10 Pro and I will give instructions only for the programs listed below.

The minimum requirements for heightmap manipulation described in the Beginner and Advanced sections are

  • an installation of 0 A.D.
  • a web browser
  • an image editing tool, preferably GIMP

Heightmap manipulation techniques in the Expert section additionally require

  • a text editor, preferably Notepad++
  • the text and hex editor PSPad
  • the hex editor HxD
  • a working environment for running scripts and some knowledge for writing them

Heightmap types

The basis for all further processes are the original grayscale heightmaps, so they should be chosen carefully. There are three different types:

Topographic images represent the elevation of the land surface. They usually use black for ocean level and lighter gray up to white for mountains. Using topographic images alone is only useful when in your area of interest there are no to very little water areas as you will get very shallow waters.

NASA Topography.png

Bathymetric images represent the elevation of the ocean floor. They usually use white for all land areas and darker gray down to black for deep trenches.

NASA Bathymetry.png

Combined topographic and bathymetric images both depict ocean floor usually with darker gray and land surface with lighter gray. Quality images of this type are very rare so most of the time you will need to combine topography and bathymetry manually. For more information have a look at the Advanced section.

NASA Combined.png

In the further mapmaking process you may also want to use photographs from airplanes or satellites as a reference for terrain texturing. However the globe has changed a lot in the last 2,000 years so it is strongly advised to take historic information into account as well.

NASA Photograph.png

Heightmap sources

If you are looking for large scale real world data, the NASA Blue Marble image series is an excellent source. It includes topography and bathymetry heightmaps as well as satellite based composite photographs of the earth. Examples are shown above.

For topography and bathymetry you can choose between two resolutions and image formats. The 21,600*10,800 pixels PNG images have a resolution of 1/60° = 1 arc minute which means one elevation data point every 1,855 m or 6,087 ft at the equator. For more details there are 8 GeoTIFF images available with 10,800*10,800 pixels each which also cover the whole globe when assembled together in the following way:

A1 | B1 | C1 | D1
A2 | B2 | C2 | D2

Their resolution is 1/120° = 1/2 arc minute = 30 arc seconds which corresponds to 928 m or 3,044 ft at the equator. These global maps contain very much details. Even a 32*32 pixels (29.7 km or 18.4 mi) piece suits for the Tiny map size, 64*64 pixels (59.4 km or 36.9 mi) for Medium and 128*128 pixels (119 km or 73.8 mi) for Giant. For smooth terrain you can even halve that sizes. You just have to scale the piece by a factor of 4 or 8, respectively. If your image program doesn't already interpolate between the pixels, you need to add a blurring filter.

Due to image file size you should right click the link and directly save the picture as it may cause instabilities to your browser. For bathymetry, don't use the colored versions.

If you are interested in heightmaps for even smaller areas, you will find detailed data in this digital elevation data search engine. I can really recommend the 15 arc second resolution (464 m or 1,522 ft at the equator) and the 3 arc second resolution (92.8 m or 304 ft at the equator) topographic data collected by Jonathan de Ferranti. You will need to convert the data to a suitable image format, e.g. with GDAL. Have a look at Garaf's posts for a bit more information.

If you do not already have samples, you can use the ones below. Alternatively you can paint your own grayscale heightmaps in GIMP.

Topography.png     Bathymetry.png

Beginner

Spoiler

Selection and scale

First you need to crop your image to the desired area and scale it to the desired size. In the GIMP menu bar, click on Windows → Toolbox and select the elliptic symbol Ellipse Select Tool.

Toolbox Elliptic.png

After that, go to the menu bar again and open the Tool Options dialog via Windows → Dockable Dialogs → Tool Options. Check the Fixed box and select Aspect ratio in the field aside. The ratio must be 1:1. Make sure that Mode is set to Replace the current selection (the single square filled with red).

Tool Options Fixed.png     Tool Options Mode.png

Now drag your cursor over the image while holding Left Click. You can estimate the area a circle map and a square map would include. Play around until you are satisfied. Be aware that the areas next to the map border won't be accessible for units and buildings in the game. If you want to combine topography and bathymetry files as described in the Advanced section, additionally note the pixel values for Position and Size, so you can repeat the exact selection in the other image.

Drag.png     Tool Options Position Size.png

Select Image → Crop to Selection and then Image → Scale Image. Enter 128, 256 or 512 pixels for both Width and Height, depending on the map size you want. Finally select Filters → Blur → Gaussian Blur with a horizontal and vertical blur radius of about 4-8 pixels. A higher amount of pixels means your map surface will be smoother which allows better building placement.

Scale.png     Blur.png

Image → Mode → Grayscale should be marked, if it's not the case, do it now. Finally save your file with File → Export As and choose an appropriate name ending with .png so the image can be imported in Atlas.

Height range

When importing a heightmap with white and black areas, you probably will find that using the whole height range will lead to very steep slopes, giant mountains and/or giant sinkholes. Therefore you need to change the grayscale range in your image.

Open the heightmap in GIMP and make sure that Image → Mode → Grayscale is marked. Select Colors → Brightness-Contrast and click on Edit these Settings as Levels in the opening dialog. In order to see more details, expand the window to the majority of your screen. Under Input Levels you can see the distribution of different shades of gray in the image. Higher columns mean the respective shade occurs more often in the image. Now play around with the Output Levels slide control. You can change the minimum and maximum gray value and thus the minimum and maximum height in your final map. I recommend to set the minimum value to 0, because the skybox (often seen on screenshots with flat camera angles) is only present for low height ranges. Set the maximum value to about 64 (fourth of the full scale) or less.

Levels.png

Save your edited file clicking File → Export As with a new name ending with .png. Import the heightmap image in Atlas to check the result and re-edit it in GIMP until you are satisfied. For each new version, always start with the original image to minimize the loss of information.

Advanced

Spoiler

Curved surface corrections

There is an issue with the proportions on large scale heightmaps like the global maps from the NASA Blue Marble series. It can be neglected for maps depicting areas smaller than roughly about 1,000*1,000 miles or 1,600*1,600 km. The problem with earth's surface is that it can't be mapped on a flat surface without changing scale and/or proportions. There is no perfect solution, but if you are taking only a part of such a big map, there is a great potential for enhancement. Near the equator or the vertical center of the original map we don't need to do corrections, but the closer your area of interest is to the upper or lower edge, the greater the deviations of size and/or proportion are in that area.

Now, take the center of your area of interest and determine its deviation from the center of the original map in degrees. You can use a globe or Google Maps for this task. Calculate 1/cos(degrees) and note that number. In GIMP, click on Image → Scale Image and then on the chain right of the Width and Height field. Then multiply your calculated number with the height and enter the new height. The proportions in your map center should be correct again, and they will deviate more and more to the upper and lower map edge (but much less than before).

Of course you need to do this scaling before you crop your map to the final square piece, or you need to crop it again.

Merging topography and bathymetry

Often it will be necessary to combine bathymetric and topographic heightmap images. If they have the same size and depict exactly the same area, this task is quite simple. Open the bathymetry file in GIMP and copy the topography image on top of it. Open the Windows → Dockable Dialogs → Layers dialog and set Opacity to 50.0. Now you have combined the two heightmaps with the cost of halving the precision. Before their fusion they had 256 steps of gray each, now only 256 steps in total. I will show how to eliminate this information loss in the Expert section.

Now it can get pretty complicated if your heightmaps don't fit 1:1 on top of each other. You will need to scale one to the other. You can get the best estimation for the right scale by examining steep shorelines. You may also edit the images to make the shoreline more visible. For example you can use the upper slide control in Colors → Brightness-Contrast → Edit these Settings as Levels to accomplish this.

Expert

Spoiler

Here I will show you how to extract height information directly from BMP bitmap images and write them to the PMP file which stores the height data of the actual 0 A.D. map.

This is only one possible solution, and it is not the most efficient nor the most elegant. It's just working with the tools and skills available to me. I would be glad if some developer could use this as an inspiration for a better solution.

Motivation

Atlas imports 8-bit grayscale heightmaps which are encoding 256 possible heights. These heights are mapped on the whole height range supported by 0 A.D. There are 65,535 possible heights corresponding to about 712 m meaning one height step corresponds to about 1.1 cm. However, the grayscale heights produce steps of about 2.78 m. If you have a close look at imported height maps this leads to a terrace like style especially at smaller slopes. Making direct changes to the PMP file in theory allows access to the full range of 65,535 possible heights. However, you usually find 8-bit and not 16-bit grayscale heightmaps so the precision increase is not that big. In most cases it is still big enough to make the terrace steps so small that they are not noticed in-game.

A typical map in 0 A.D. only uses less than an eighth of the possible height range (because this was the old height limit, see ticket #3112), newer maps may also use about a quarter of the range. This means there are fewer than 32 discrete height values (up to 64 for newer mountainous maps) when using the standard heightmap import. If you combine topographic and bathymetric height data and write it to the PMP file, you can have a total of up to 2*256 = 512 discrete heights within the same range. This means a precision increase of about 5-15 depending on the number of shades of gray and in-game height range. The height steps shrink from 2.78 m to about 19 to 56 cm in size.

Currently you need to do a lot of extra work for a quality map after the heightmap import due to the limited precision. But if you use the methods described in the next sections, you can reduce that work to a minimum.

Extracting height data from BMP images

Usually images are stored in a binary file format so you can't simply read them with a text editor. However, uncompressed data can be accessed easily with a hex editor. I recommend to export the heightmap as grayscale BMP file in GIMP and open it with PSPad. Click on View → Hex Edit MODE if you are still in the text edit mode. Wikipedia describes how to interpret the shown data. It is important to know that the data is stored in little-endian format meaning the least significant byte is found most left. Some important bytes in the BMP file format are:

  • 11-14: 32-bit unsigned integer, offset or starting address of the actual bitmap image data
  • 19-22: 32-bit integer, bitmap width in pixels
  • 23-26: 32-bit integer, bitmap height in pixels

If the last number is positive, we have a bottom-up bitmap where the data starts in the lowest line and ends in the highest. This is important for our use since the PMP file also stores the data bottom-up. Otherwise we would have a top-down bitmap.

We would expect that the data block has a size of the pixel width*height since GIMP exports to 8-bit grayscale. However, if the width is no multiple of 4, the rest is filled up with zeros. This is important because the PMP map size is (mapsize*16 + 1), so there are three empty bytes following the end of each pixel line. You need a 129*129 pixels image for the Tiny map size, 257*257 pixels for Medium and 513*513 pixels for Giant.

Script calculations

With the right scripting languages you can directly readout data from binary files. However I'm a JScript noob. No it's not JavaScript, but a scripting language invented by Microsoft. I didn't choose it, but had to learn it for my Bachelor thesis and it's the only scripting language I'm comfortable with at the moment. The bad thing about JScript is that it only supports the readout of text files. If you need a text file like me, copy the hexadecimal code into Notepad++ and save it in text format. This procedure is done for both the topography and the bathymetry file.

With the knowledge about BMP files from the previous section, the script extracts the 8-bit height values of topography and bathymetry, chooses one of them and converts them to 16-bit values considering some boundary conditions. These include the minimum and maximum height of the output and the number of additional steps inserted between land and ocean. This is an useful feature for maps with very shallow waters as units won't just walk through the ocean. The calculated data is then written to a new text file. You can have a look at my script bmp-to-pmp-conversion.js.

Writing height data to PMP files

Our wiki contains all information you need to know about the PMP file format which stores the height and terrain texture information of the 0 A.D. maps.

The easiest way to get an empty PMP file for your needs is to open the Atlas editor, generate a new random map Wall Demo of the desired map size, remove all structures and lift all the terrain to the maximum height. This way the data section will be clearer visible when you open the file with HxD. This hex editor supports the selection of huge file parts with Shift + Left Click. However, unlike PSPad, HxD inserts whitespaces when you copy hex code to a text file, so I'm using both editors.

Now copy the processed data of the script with Notepad++ (or with a hex editor if you didn't use text files) and paste it in the right place in the PMP file. You should skip the first 16 bytes and select all data until the F-sequence stops before pasting the new data. If no This operation changes the file size message is shown you probably haven't done much wrong yet. Save your edited file and open the map again in Atlas and voilà – there it is!

Changelog

Spoiler

version 0.5 - 06/26/2016

  • Introduction: also thank FeXoR and niektb, remove outdated picture announcement
  • Basics - Heightmap sources - 2nd and following paragraphs:
    add information about GeoTIFF images and resolution in degrees, arc minutes/seconds, (kilo)meters and miles/feet
    link/recommend high resolution data collected by Jonathan de Ferranti
  • Advanced - Curved surface corrections - 2nd paragraph: link Google Maps
  • Expert - Motivation - 2nd paragraph: add information about old height limit and correct some numbers
  • Changelog: added as a new spoiler
  • wording and formatting improvements

version 0.4 - 06/06/2016

  • Introduction: thank Garaf
  • Basics - Heightmap sources: link DEM search engine, GDAL and Garaf's posts about its usage

version 0.3 - 06/05/2016

  • Pro: rename to Expert

version 0.2 - 05/28/2016

  • Basics: insert pictures, simplify instructions by removing unnecessary steps

version 0.1 - 05/28/2016

  • initial commit

 

Edited by Palaxin
  • Like 5

Share this post


Link to post
Share on other sites

@Palaxin Very nice!

I added a link in the Heightmap Import section of the Atlas Manual.

If you want you can extend that section yourself if you feel some informations are missing there ;)

EDIT: I wonder though if anyone could make a living with that "Pro" hints ... for that's what "professional" means AFAIK ^^

 

  • Like 2

Share this post


Link to post
Share on other sites

Added illustrations to the Basics and Beginner sections :)

4 hours ago, FeXoR said:

EDIT: I wonder though if anyone could make a living with that "Pro" hints ... for that's what "professional" means AFAIK ^^

You suggest to rename it to something else? E.g. Advanced II? Or a more content related caption like .pmp file hacks or scripting or something similar? BTW thanks for linking the guide ;)

Edited by Palaxin

Share this post


Link to post
Share on other sites

I'd go with "Advanced with Scripting" I guess ;)

Edited by feneur
Scriting -> Scripting

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×