Grautvornix Posted Friday at 22:44 Share Posted Friday at 22:44 (edited) Mercator and post-Atlas are now one package with one version number: Atlas Tools 0.20.0. It holds everything of Mercator 0.17.0 and post-Atlas 1.19.0, the last separate releases. They stay two programs, because the two jobs are different, but what you install is now what was tested together and they share the same codebase. The tools are also closeley linked so that you can start editing in mercator and then handover to post-Atlas for verification or conversion, and back. Mercator is a tool to create maps either from scratch or from digital elevation models, sculpt the terrain add features and textures. It however does not replace Atlas, the games's map editor. Mercator's sole purpose is to ease some of the pains the powerful but complex atlas tool creates to users like myself. In fact, Mercator allows handing over its data to the actual Atlas tool for refinement and can then take the map back for the final touch creating a mod or an installation. post-Atlas is a map checking and conversion tool that can be used to process maps and map collections in differnt formats an convert them into R28 and R29 game format. I also verifies map propserties and provides support to correct issues. post-Atlas also works with mercator and Atlas allowing to handover to these tools and take the result back afterwards. Maps can be tested in-game (headless) to verify there is no blocking issue left. The package is still delivered as python souce code (see Readme.md for installation instructions) and should run on Windows 11, Linux and MacOS. While I can only test it on Windows currently, @lilypoppy has kindly tested it extensively on Linux and @Asher used it on MacOS. I am keen to receive any feedback, criticism, ideas, suggestions, error reports etc. Please let me know! As per @Asher's suggestion, here are also specfiles that can be used with pyinstaller to create two executables (each ~95MB). Create them from the target root folder you chose when extracting the zip using these commands: pyinstaller Mercator.spec pyinstaller Post-Atlas.spec or, even better, use @Asher's modified build script (build.sh for bash environments and build.bat for windows commandline, both located in the root folder of this zip) that also installs all necessary python modules if needed. Prerequisite: You need an installed python environment (> 3.10) an an installed version of the O.A.D. game. IMPORTANT NOTE: Attached below are two versions of the same package (see changelog for the difference) : Atlas-Tools 0.24.0 has a build script/pyinstaller spec files that pack the two tools plus their supporting libraries into two large executable files (Mercator: 97 MB, Post-Atlas 96 MB) into a folder named "/dist" . They start very slowly as these packages are unpacked each time the exe is started. benefit: each is is a self-sufficient exe-file. The two executables must stay next to each other in the same folder. Atlas-Tools 0.25.0 has a build script/pyinstaller spec files that deliver two folders within a parent folder named "/dist". The two subfolders are mercator and post-atlas. Each subfolder has two items: another subfolder with the already unpacked support files (done once during install) plus a much smaller executable file (16 MB) that is starting faster. The two folders must stay next to each other in the same master folder. The condition "have to stay next to each other" is due to the handover mechanism between the two tools, allowing to continue your worlkflow in the repective other tool and then hand back to the first one, both working on the same map object. IMPORTANT: For those without access to Python, I am including a pre-packaged version 0.25.0 for the Windows 11-Platform (basically the content of the /dist folder after running build.bat). A MacOS versions is also avaialble belwo for download (thanks @Asher !). Download the zip for your OS Atlas-tools0_25.0-prepackagedforwindows.zipAtlas-tools0_25.0-prepackagedforwindows.zipand unpack into your destination folder. You will have everything ready to start editing in the subfolders named Mercatpor and Post-Atlas. Just run the executables there. Happy Mapping! README.md atlas-tools-0.24.0.zip CHANGELOG.md atlas-tools-0.25.0.zip Atlas-tools-prepackaged_for_macos.zip Atlas-tools-pre-packaged_for_windows.zip Edited Saturday at 21:34 by Grautvornix Provided version 0.24.0 along with version 0.25.0 and a prepackaged pair of executables. 2 Link to comment Share on other sites More sharing options...
Asher Posted Friday at 23:01 Share Posted Friday at 23:01 build.sh Just made a build script to make this into an executable. (Should work across systems). Maybe you'd want to add this @Grautvornix? Also, does it just need pyqt6 and scipy to also be installed, or are there more? 1 Link to comment Share on other sites More sharing options...
lilypoppy Posted Friday at 23:37 Share Posted Friday at 23:37 @Grautvornix @Asher 2 Link to comment Share on other sites More sharing options...
Grautvornix Posted Friday at 23:43 Author Share Posted Friday at 23:43 (edited) Hi @Asher thanks for the quick reaction -and the good idea! The tool needs PyQt6, Pillow, numpy, scipy, and pyqtgraph/PyOpenGL. Your script is great, but there are actually two tools in the package (Mercator and post-Atlas) and they need to have their specific environment. My intention was to keep it open as editable python code so anyone can inspect and modify it as needed. But you are certainly right - not everyone has python installed or would like to look into the code. Here are the two pyinstaller spec files Mercator.spec and post-Atlas.spec . A plain pyinstaller --onefile Mercator.py only follows Python imports. It misses non-code resources that get loaded relative to the installed files — docs/ (Help, User Manual, Design notes, Credits), CREDITS.md/LICENSE, and most of all mercator/world/*.json.gz (the real-elevation-data feature). Attached are obsolete pyinstaller spec files that can be called by pyinstaller Mercator.spec pyinstaller PostAtlas.spec New files are in the current distribution (cf. initial post of this thread) This would work as a standalone double-click exe, however: The hand-over between the two tools will not work as built: Check or convert in post-Atlas... and Convert > Edit in Mercator launch the sibling tool via sys.executable -m <module> (atlas_ui/other_tool.py). Inside a frozen exe, sys.executable is that exe, which doesn’t understand -m — and worse, the check for “is the other tool installed” (importlib.util.find_spec) will always say no, since each exe only bundles its own tool. So the button offering the hand-over won’t even appear. That’s a significant regression for frozen builds specifically. I ask my coding buddy (Claude) to find a solution for this as I believe the linking o fhte two tools is an important aspect. @lilypoppy suggested to create a single tool but that would make the GUI even more complex than it is today, I am afraid. But he does have point here. Size: Mercator.exe is ~140 MB, Post-Atlas.exe ~116 MB on windows. They are bundling PyQt6 + NumPy + SciPy + Pillow + pyqtgraph/PyOpenGL for the 3D views). Versus the ~6 MB release zip, this is a very different distribution. This would clearly call for a single exe only. @Asher @lilypoppy Claude just implemented a fix in version 0.23.0, see initial post of this thread. So the only remaining issue is size. Still unsure if we should dare going for a single tool. Please see the latest update on the initial post of this thread. Here are only old/buggy versions. There are new ones in the distribution on top of this thread. Mercator.spec PostAtlas.spec Edited Saturday at 20:26 by Grautvornix added the pyinstaller calls, updated the specs Link to comment Share on other sites More sharing options...
Asher Posted Saturday at 00:13 Share Posted Saturday at 00:13 (edited) Do you have icons for these, @Grautvornix? And here is the update build script build.zip (I included the files @Grautvornix posted above) Edited Saturday at 00:15 by Asher me must work on grammers Link to comment Share on other sites More sharing options...
Asher Posted Saturday at 00:29 Share Posted Saturday at 00:29 build.zip (remove --onefile from build script so it doesn't take as long to run the program) Link to comment Share on other sites More sharing options...
Grautvornix Posted Saturday at 00:42 Author Share Posted Saturday at 00:42 (edited) 11 hours ago, Asher said: build.zip (remove --onefile from build script so it doesn't take as long to run the program) Cool! But does that script work? (shouldn't it be e.g. "pyinstaller mercator.spec" instead of "pyinstaller mercator.py" ?) Embarassingly stupid remark, apologies for that!. 11 hours ago, Asher said: Do you have icons for these, @Grautvornix? And here is the update build script build.zip (I include the files @Grautvornix posted above) Now that is a challenge! No, I don't have any yet, possibly something like the attached (still ugly with the tongue hanging out) Edited Saturday at 12:00 by Grautvornix 1 Link to comment Share on other sites More sharing options...
Asher Posted Saturday at 00:53 Share Posted Saturday at 00:53 I saw that you cannot use the -w flag with using the .spec files @Grautvornix, so I think just building from the .py files will be ok. Was there a specific reason you did the .spec files? Here is an update version of the build script: build-1.0.1.zip Link to comment Share on other sites More sharing options...
lilypoppy Posted Saturday at 01:17 Share Posted Saturday at 01:17 I'm going to test an existing mod to observe its behavior; indeed, I've noticed that the Atlas Tools Kit seems designed for mods built with the most recent versions. So I'd like to check whether it still works with an older mod or if it needs adjustments. I also have my own mod (made up of 20 files); @Grautvornix how will the output process work with my mod ? Link to comment Share on other sites More sharing options...
Grautvornix Posted Saturday at 11:48 Author Share Posted Saturday at 11:48 10 hours ago, Asher said: I saw that you cannot use the -w flag with using the .spec files @Grautvornix, so I think just building from the .py files will be ok. Was there a specific reason you did the .spec files? Here is an update version of the build script: build-1.0.1.zip Oh, you are right, of course! The reason I asked Claude to provide spec files was that I wanted ensure all documents and the additonal data set for mercator's world map (forms a possible basis for selecting terrain tiles from public DEM sources) are included. the result is ratehr complex and unpleasant to read while your script is straightforward and understandable. I prefer yours actually. 1 Link to comment Share on other sites More sharing options...
Asher Posted Saturday at 12:35 Share Posted Saturday at 12:35 42 minutes ago, Grautvornix said: Oh, you are right, of course! The reason I asked Claude to provide spec files was that I wanted ensure all documents and the additonal data set for mercator's world map (forms a possible basis for selecting terrain tiles from public DEM sources) are included. the result is ratehr complex and unpleasant to read while your script is straightforward and understandable. I prefer yours actually. Is the .spec file like the build file, so everytime you build it, it runs that? Link to comment Share on other sites More sharing options...
Grautvornix Posted Saturday at 13:09 Author Share Posted Saturday at 13:09 11 hours ago, lilypoppy said: I'm going to test an existing mod to observe its behavior; indeed, I've noticed that the Atlas Tools Kit seems designed for mods built with the most recent versions. So I'd like to check whether it still works with an older mod or if it needs adjustments. I also have my own mod (made up of 20 files); @Grautvornix how will the output process work with my mod ? Good plan! The intent of post-Atlas was to allow batch editing of map packages (well still editing one-by-one but you can step through the maps) and outputting everything into a mod. Link to comment Share on other sites More sharing options...
Grautvornix Posted Saturday at 18:22 Author Share Posted Saturday at 18:22 5 hours ago, Asher said: Is the .spec file like the build file, so everytime you build it, it runs that? It was supposed to also include the worldmap data for selecting the digital elevaiton area for terrain building from scratch. It also removes (now, after bug fixing) teh not needed part of the libraries so the exe goes down from 142 MB to 98 MB for mercator and for post-Atlas from 116 to 97 MB. For me the startup process of the exe files with the "onefile" option is still way too long. This is due to the need to unpack everything during each startup. I will instead now change that to "onedir" i.e. an exe plus a folder with runtime libs. These must stay next to each other at all times. Link to comment Share on other sites More sharing options...
Asher Posted Saturday at 21:17 Share Posted Saturday at 21:17 Here are pre-built app bundles for macos: macos.zip 1 Link to comment Share on other sites More sharing options...
lilypoppy Posted 2 hours ago Share Posted 2 hours ago @Grautvornix @Asher Hi, the test doesn't seem to have worked: the build failed with externally-managed-environment (PEP 668) and pyinstaller: command not found, even though the script still printed "Success!". :~/Téléchargements/atlas-tools-0.25.0$ bash build.sh atlas-tools build script build script version 1.2.0 =============================================== Collecting libs... error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. error: externally-managed-environment × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-Debian-packaged Python package, create a virtual environment using python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make sure you have python3-full installed. If you wish to install a non-Debian packaged Python application, it may be easiest to use pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed. See /usr/share/doc/python3.12/README.venv for more information. note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. hint: See PEP 668 for the detailed specification. =============================================== Building Mercator... build.sh: ligne 13: pyinstaller : commande introuvable Done =============================================== Building Post-Atlas... build.sh: ligne 17: pyinstaller : commande introuvable Done =============================================== Success! The executables can be found under the dist folder 1 Link to comment Share on other sites More sharing options...
Grautvornix Posted 1 hour ago Author Share Posted 1 hour ago @lilypoppy Oops! Sorry for the trouble! there are actually two faults, both in build.sh, not in the specs: The cause. Debian and Ubuntu refuse pip install into the system Python (PEP 668). So nothing was installed, pyinstaller did not exist, and no build ran. The misleading end. The script carried on after every failure and printed “Success!”. Working on a better script! Will come out shortly. Link to comment Share on other sites More sharing options...
Grautvornix Posted 39 minutes ago Author Share Posted 39 minutes ago Here is the new build script and also new spec files (bug identified: On Linux the program is a file called Mercator, with no .exe, and PyInstaller writes it into its work folder, where the spec had already made a folder of that name. Both specs now work in build/<name>/, which is PyInstaller’s own layout. I do not know why this did not stop @Asher’s macOS build, since macOS has no .exe either. Will update all spec-files, both build scripts and deliver a new release 0.26.0 very soon (will be available from the initial post on this thread). For @lilypoppy and all Ubuntu or Debian users, the whole preparation of the build should be this one line: sudo apt install python3-venv binutils libpython3.12 Attached are the new build.sh and the two new pyinstaller spec files. Things to know: The tool also needs the usual desktop libraries added (glib, OpenGL, xkbcommon and so on). A normal Linux desktop has them; a bare server would not. The Linux builds are large: about 269 MB each, against roughly 98 MB on Windows. The icon (icon.ico) is ignored on Linux; PyInstaller only supports it on Windows and macOS. The new release will 0.26 contain this: [fix] The new build scripts (build.sh and build.bat as well as mercator.spec and post-atlas.spec) and a slightly revised icon [new] It integrates the findings tab from post-atlas into mercator so to allow fixing any findings already in the tool itself without having to switch between tools anymore when creating a new map. This option is still there, however. The additional workflow and overall GUI is not yet as simple and as clear as I would like it to be. Will continue working on it. Post-Atlas.spec build.sh Mercator.spec Link to comment Share on other sites More sharing options...
Asher Posted 27 minutes ago Share Posted 27 minutes ago 11 minutes ago, Grautvornix said: Here is the new build script and also new spec files (bug identified: On Linux the program is a file called Mercator, with no .exe, and PyInstaller writes it into its work folder, where the spec had already made a folder of that name. Both specs now work in build/<name>/, which is PyInstaller’s own layout. I do not know why this did not stop @Asher’s macOS build, since macOS has no .exe either. Will update all spec-files, both build scripts and deliver a new release 0.26.0 very soon (will be available from the initial post on this thread). The reason is pyinstaller makes the executable for whatever os you are using. I think I saw somewhere in the terminal output that it was converting exe to something so mac could use it Link to comment Share on other sites More sharing options...
Asher Posted 22 minutes ago Share Posted 22 minutes ago 2 hours ago, lilypoppy said: × This environment is externally managed ╰─> To install Python packages system-wide, try apt install python3-xyz, where xyz is the package you are trying to install. This is the problem @lilypoppy. I was getting this on linux too, I think I used --break-system-packages or something. This is how ai summarized it: Quote The --break-system-packages flag allows pip to install Python packages directly into the system-wide Python environment, bypassing the Externally Managed Environment restriction enforced by PEP 668 Link to comment Share on other sites More sharing options...
lilypoppy Posted 10 minutes ago Share Posted 10 minutes ago @Asher Thanks ! @Grautvornix this version is better but I have this : 39875 INFO: Graph cross-reference written to /home/felix/Téléchargements/atlas-tools-0.25.0/build/Mercator/xref-Mercator.html 39925 INFO: checking PYZ 39925 INFO: Building PYZ because PYZ-00.toc is non existent 39925 INFO: Building PYZ (ZlibArchive) /home/felix/Téléchargements/atlas-tools-0.25.0/build/PYZ-00.pyz 41085 INFO: Building PYZ (ZlibArchive) /home/felix/Téléchargements/atlas-tools-0.25.0/build/PYZ-00.pyz completed successfully. 41115 WARNING: Ignoring icon; supported only on Windows and macOS! 41126 INFO: checking PKG 41126 INFO: Building PKG because PKG-00.toc is non existent 41126 INFO: Building PKG (CArchive) Mercator.pkg 41144 INFO: Building PKG (CArchive) Mercator.pkg completed successfully. 41145 INFO: Bootloader /home/felix/Téléchargements/atlas-tools-0.25.0/.build-venv/lib/python3.12/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run 41145 INFO: checking EXE 41145 INFO: Building EXE because EXE-00.toc is non existent 41145 INFO: Building EXE from EXE-00.toc ERROR: The output directory "/home/felix/Téléchargements/atlas-tools-0.25.0/build/Mercator" is not empty. Please remove all its contents or use the -y option (remove output directory without confirmation). Link to comment Share on other sites More sharing options...
lilypoppy Posted 7 minutes ago Share Posted 7 minutes ago @Grautvornix Also note that I get the same error, even if I clear the build folder! Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now