Atlas-Tools, a package combining Mercator and post-Atlas

[Grautvornix]

Mercator and post-Atlas are now one package with one version number: Atlas Tools 0.20.0. It holds everything of Mercator 0.17.0 and post-Atlas 1.19.0, the last separate releases. They stay two programs, because the two jobs are different, but what you install is now what was tested together and they share the same codebase. The tools are also closeley linked so that you can start editing in mercator and then handover to post-Atlas for verification or conversion, and back.

• Mercator is a tool to create maps either from scratch or from digital elevation models, sculpt the terrain add features and textures. It however does not replace Atlas, the games's map editor. Mercator's sole purpose is to ease some of the pains the powerful but complex atlas tool creates to users like myself. In fact, Mercator allows handing over its data to the actual Atlas tool for refinement and can then take the map back for the final touch creating a mod or an installation.
• post-Atlas is a map checking and conversion tool that can be used to process maps and map collections in differnt formats an convert them into R28 and R29 game format. I also verifies map propserties and provides support to correct issues. post-Atlas also works with mercator and Atlas allowing to handover to these tools and take the result back afterwards. Maps can be tested in-game (headless) to verify there is no blocking issue left. 

The package is still delivered as python souce code (see Readme.md for installation instructions) and should run on Windows 11, Linux and MacOS. While I can only test it on Windows currently, @lilypoppy has kindly tested it extensively on Linux and @Asher used it on MacOS. I am keen to receive any feedback, criticism, ideas, suggestions, error reports etc. Please let me know!

As per @Asher's suggestion, here are also specfiles that can be used with pyinstaller to create two executables (each ~95MB). Create them from the target root folder you chose when extracting the zip using these commands:   

pyinstaller Mercator.spec
pyinstaller Post-Atlas.spec

or, even better, use @Asher's modified build script  (build.sh for bash environments and build.bat for windows commandline, both located in the root folder of this zip) that also installs all necessary python modules if needed. Prerequisite: You need an installed python environment (> 3.10) an an installed version of the O.A.D. game.

IMPORTANT NOTE: Attached below are two versions of the same package (see changelog for the difference) :

• Atlas-Tools 0.24.0  has a build script/pyinstaller spec files that pack the two tools plus their supporting libraries into two large executable files (Mercator: 97 MB, Post-Atlas 96 MB) into a folder named "/dist" . They start very slowly as these packages are unpacked each time the exe is started. benefit: each is is a self-sufficient exe-file. The two executables must stay next to each other in the same folder.
   
• Atlas-Tools 0.25.0 has a build script/pyinstaller spec files that deliver two folders within a parent folder named "/dist". The two subfolders are mercator and post-atlas. Each subfolder has two items: another subfolder with the already unpacked support files (done once during install) plus a much smaller executable file (16 MB) that is starting faster. The two folders must stay next to each other in the same master folder.

The condition "have to stay next to each other" is due to the handover mechanism between the two tools, allowing to continue your worlkflow in the repective other tool and then hand back to the first one, both working on the same map object.

IMPORTANT: For those without access to Python, I am including a pre-packaged version 0.25.0 for the Windows 11-Platform (basically the content of the /dist folder after running build.bat). Ic ould be nice to have versionfor Mac OS and Linux as well ( @Asher @lilypoppy possibly you could help out ?). Download the zip and unpack into your destination folder. You will have everything ready to start editing in the subfolders named Mercatpor and Post-Atlas. Just run the executables there.

Happy Mapping!

 

 

 

README.md atlas-tools-0.24.0.zip

CHANGELOG.md atlas-tools-0.25.0.zip

65638838_Atlas-tools0_25.0-prepackagedforwindows.zip

Edited 34 minutes ago by Grautvornix
Provided version 0.24.0 along with version 0.25.0 and a prepackaged pair of executables.

[Asher]

build.sh Just made a build script to make this into an executable. (Should work across systems). Maybe you'd want to add this @Grautvornix? Also, does it just need pyqt6 and scipy to also be installed, or are there more?

[lilypoppy]

@Grautvornix @Asher

[Grautvornix]

Hi @Asher thanks for the quick reaction -and the good idea!

The tool needs PyQt6, Pillow, numpy, scipy, and pyqtgraph/PyOpenGL.

Your script is great,  but there are actually two tools in the package (Mercator and post-Atlas) and they need to have their specific environment.

My intention was to keep it open as editable python code so anyone can inspect and modify it as needed. But you are certainly right - not everyone has python installed or would like to look into the code. Here are the two pyinstaller spec files  Mercator.spec and post-Atlas.spec . 

A plain pyinstaller --onefile Mercator.py only follows Python imports. It misses non-code resources that get loaded relative to the installed files — docs/ (Help, User Manual, Design notes, Credits), CREDITS.md/LICENSE, and most of all  mercator/world/*.json.gz (the real-elevation-data feature). 

Attached are obsolete pyinstaller spec files that can be called by

pyinstaller Mercator.spec
pyinstaller PostAtlas.spec

New files are in the current distribution (cf. initial post of this thread)

This would work as a standalone double-click exe, however: 

• The hand-over between the two tools will not work as built:

 Check or convert in post-Atlas... and Convert > Edit in Mercator launch the sibling tool via sys.executable -m <module> (atlas_ui/other_tool.py). Inside a frozen exe, sys.executable is that exe, which doesn’t understand -m — and worse, the check for “is the other tool installed” (importlib.util.find_spec) will always say no, since each exe only bundles its own tool. So the button offering the hand-over won’t even appear. That’s a significant regression for frozen builds specifically. I ask my coding buddy (Claude) to find a solution for this as I believe the linking o fhte two tools is an important aspect. @lilypoppy suggested to create a single tool but that would make the GUI even more complex than it is today, I am afraid. But he does have point here.

• Size:

Mercator.exe is ~140 MB, Post-Atlas.exe ~116 MB  on windows. They are bundling PyQt6 + NumPy + SciPy + Pillow + pyqtgraph/PyOpenGL for the 3D views). Versus the ~6 MB release zip, this is a very different distribution. This would clearly call for a single exe only.

 

 

@Asher @lilypoppy Claude just implemented a fix in version 0.23.0, see initial post of this thread. So the only remaining issue is size. Still unsure if we should dare going for a single tool. Please see the latest update on the initial post of this thread.

 

Here are only old/buggy versions. There are new ones in the distribution on top of this thread.

Mercator.spec PostAtlas.spec

Edited 1 hour ago by Grautvornix
added the pyinstaller calls, updated the specs

[Asher]

Do you have icons for these, @Grautvornix? And here is the update build script build.zip (I included the files @Grautvornix posted above)

Edited 21 hours ago by Asher
me must work on grammers

[Asher]

build.zip (remove --onefile from build script so it doesn't take as long to run the program)

[Grautvornix]

11 hours ago, Asher said:

build.zip (remove --onefile from build script so it doesn't take as long to run the program)

Cool! But does that script work?  (shouldn't it be e.g. "pyinstaller mercator.spec" instead of "pyinstaller mercator.py" ?) Embarassingly stupid remark, apologies for that!. 

11 hours ago, Asher said:

Do you have icons for these, @Grautvornix? And here is the update build script build.zip (I include the files @Grautvornix posted above)

Now that is a challenge! No, I don't have any yet, possibly something like the attached (still ugly with the tongue hanging out) 

Edited 9 hours ago by Grautvornix

[Asher]

I saw that you cannot use the -w flag with using the .spec files @Grautvornix, so I think just building from the .py files will be ok. Was there a specific reason you did the .spec files? Here is an update version of the build script: build-1.0.1.zip

[lilypoppy]

I'm going to test an existing mod to observe its behavior; indeed, I've noticed that the Atlas Tools Kit seems designed for mods built with the most recent versions. So I'd like to check whether it still works with an older mod or if it needs adjustments. I also have my own mod (made up of 20 files);

@Grautvornix how will the output process work with my mod ? 

[Grautvornix]

10 hours ago, Asher said:

I saw that you cannot use the -w flag with using the .spec files @Grautvornix, so I think just building from the .py files will be ok. Was there a specific reason you did the .spec files? Here is an update version of the build script: build-1.0.1.zip

Oh, you are right, of course! The reason I asked Claude to provide spec files was that I wanted ensure all documents and the additonal data set for mercator's world map (forms a possible basis for selecting terrain tiles from public DEM sources) are included. the result is ratehr complex and unpleasant to read while your script is straightforward and understandable. I prefer yours actually.

[Asher]

42 minutes ago, Grautvornix said:

Oh, you are right, of course! The reason I asked Claude to provide spec files was that I wanted ensure all documents and the additonal data set for mercator's world map (forms a possible basis for selecting terrain tiles from public DEM sources) are included. the result is ratehr complex and unpleasant to read while your script is straightforward and understandable. I prefer yours actually.

Is the .spec file like the build file, so everytime you build it, it runs that?

[Grautvornix]

11 hours ago, lilypoppy said:

I'm going to test an existing mod to observe its behavior; indeed, I've noticed that the Atlas Tools Kit seems designed for mods built with the most recent versions. So I'd like to check whether it still works with an older mod or if it needs adjustments. I also have my own mod (made up of 20 files);

@Grautvornix how will the output process work with my mod ? 

Good plan! The intent  of post-Atlas was to allow batch editing of map packages (well still editing one-by-one but you can step through the maps) and outputting everything into a mod. 

[Grautvornix]

5 hours ago, Asher said:

Is the .spec file like the build file, so everytime you build it, it runs that?

It was supposed to also include the worldmap data for selecting the digital elevaiton area for terrain building from scratch. It also removes (now, after bug fixing) teh not needed part of the libraries so the exe goes down from 142 MB to 98 MB for mercator and for post-Atlas from 116 to 97 MB. For me the startup process of the exe files with the "onefile" option is still way too long. This is due to the need to unpack everything during each startup. I will instead now change that to "onedir" i.e. an exe plus a folder with runtime libs. These must stay next to each other at all times.

[Asher]

Here are pre-built app bundles for macos: macos.zip