Leaderboard

Introduction This article describes different aspects of the scripting integration in our game engine. It starts explaining the basics like the difference between a scripting language and a native programming language which should be comprehensible for everyone with a basic knowledge of computers. In the last paragraph about the ongoing Spidermonkey upgrade, the article goes into more detail and is targeted more at programmers and scripters who are already a bit more familiar with the topic. What is scripting? Basically scripting uses another layer between the operating system (Windows, Mac OS X, Linux) and the code being executed. This layer is a program that reads the script code and executes the commands and is called "interpreter", "script engine" or "script runtime". You don't have to compile scripts before they can be executed by an interpreter. ScriptsNative code Native code on the other hand needs to be compiled before it can be executed. Compiling generates binary machine code that is specific to an operating system and even specific to a certain type of processor. For example you can't run the machine code generated for Linux or Mac OS X on a Windows system, but script code is the same on all three platforms and can run on all of them if there's an interpreter program available. An internet browser like Firefox contains a Javascript interpreter for example. You can display websites containing Javascript on all three operating systems, because Firefox contains the interpreter for those scripts. If you copy the .exe file of Firefox from Windows to Linux you won't be able to run it because it's binary machine code created specifically for Windows systems. Why scripting? With scripting languages you generally have less control over the low level parts of the program like memory access. First that might sound bad, but it also avoids a lot of potential errors the programmer could make. Bugs in code that accesses the memory directly usually can cause crashes or buffer overflows which are very severe problems and often also security issues. Interpreted scripts have an additional level of security because the scripts can only do what the interpreter allows them. Usually if scripts contain errors, an error message is printed to the screen and something in the application might not work as expected. It will be much less likely that an attacker can compromise the user's system if he finds an error in a script compared to finding an error in native code. From the security point of view, it would be very bad if not completely insane to download and execute untrusted native code. In addition to that it would also be quite difficult because the operating system and processor architecture need to match. Some scripting languages on the other hand are designed for that. Javascript can be used for websites and every time you access an unknown website containing Javascript, you execute untrusted script code in your browser. So let's sum it up. Advantages of scriptingMore or less "untrusted" code can be executed with little security concerns. In our case this can be used for automatic downloads of random map scripts, user defined mission scripts, user defined AIs or even GUI mods.Less low-level functionality makes scripts less vulnerable to security issues.Scripts generally allow faster iteration in the development process.Scripts are easier for modders because they only need a text editor instead of a lot of development toolsIn our case we attracted some developers with Javascript knowledge who probably wouldn't have joined us if we only used C++. This is probably different for other projects and not a general advantage of scripting though.Disadvantages of scripting The additional layer added by the interpreter and other characteristics of scripts make them run slower than native code.Maintaining a script interface and including a script interpreter in an application adds additional complexity and takes time.Where do we use scripting? Basically the idea is to use scripting wherever it's possible and makes sense performance wise. It doesn't really matter if it takes a few nanoseconds more until the game decides which piece of music to put next into the playlist. It can even do that before the previous piece of music stops playing, so it won't make a difference at all. On the other hand, it makes a difference for the player if pathfinding of hundreds of units on a map with a million of terrain tiles (little squares of terrain) runs in native code or in a slower script. If you order a unit to move somewhere and it takes half a second until it reacts, that's bad. The difficulty here is to find the right balance and to design sensible interfaces between native code an scripts. GUI (Graphical User Interface) Basically the whole GUI can be modified by changing XML files and Javascript scripts that are linked to actions in the GUI. The GUI system itself is written in C++ code, but XML files and scripts define how graphics are arranged and what happens when you click on a button. The Javascript code behind buttons might call native C++ function that are exposed to scripts though. Random map scripts Random map scripts are written in Javascript which makes them easy to share because they are meant to be shared and will probably even be downloaded automatically from other players in the future. Random map scripts aren't very time critical. The player shouldn't wait five minutes until a game starts, but it doesn't matter if it takes a few seconds longer. There was an idea recently that changed our view about that a bit though. It would be nice to generate previews of random maps before starting the game. This changes the situation a bit because the player might want to generate multiple maps before he's happy with the result. Now there are ongoing discussions about providing some commonly used functionality through native functions exposed to scripts. AI (Artifical intelligence) The AI is where using scripts is currently most controversial. The AI does a lot of calculations and works with a lot of data which runs too slow with the current scripts. Some people want to switch it to native C++ code completely, but that alone won't solve the problem completely. The better approach is probably switching the most performance critical AI functions to C++ and thinking about how they could be optimized at the same time. The less performance critical code can stay in the scripts. Of course this description of the solution is simplified and there are a lot of other aspects to consider which makes it more difficult than it might sound first. Gameplay logic Gameplay logic includes everything that defines how the game plays like that there are units and these units have proerties such as "health" or "walk speed". Other examples of gameplay logic are technology research, Battle dection (for playing battle music), the implementation of trade, or how building works (using more builders makes building faster, only some units are builders etc.). We use a system called the "simulation" for implementing gameplay logic and associated functionality. The simulation consists of components ("Health" or "Builder" for example). Components can be written either in C++ or in Javascript. In this area we are quite flexible to switch between a scripted or a native code implementation. Scripting integration Scripting language I've already mentioned that we have chosen Javascript as our scripting language. There were a few other candidates like Lua and I can't really tell why we have favoured Javascript. It's a very feature rich and flexible scripting language that is broadly supported and known by many people. While flexibility is its strength it's probably also its weakness. This flexibility makes it a bit slower and more difficult to be optimized by the scripting engine. It has also some quirks that modders and programmers need to know about in order to avoid trouble. In practice it's hard to compare performance to other scrip languages because benchmarks usually differ a lot from the code that is actually used in real world applications. Scripting engine There are multiple different scripting engines available for Javascript. These are all designed for maximum performance and there's an ongoing competition between companies like Apple, Google, Microsoft and Mozilla about who has the fastest web browser and the fastest Javascript engine. These engines go way beyond running Javascript in a script interpreter as described earlier in this article. They gather information about the script code and optimize it while it's running. They even create platform specific assembly code, which is quite similar to what a C++ compiler generates. This is a very complex task and all these companies put a lot of man-years and money into their scripting engines. Those engines we could use are mainly Google's V8 or Mozilla's Spidermonkey. Other engines are not compatible with our licenses, only work on a specific platform or aren't available as standalone libraries. We have chosen Spidermonkey years back and it would be a lot of work to change to another library today if we wanted that for some reason. Here the issue with benchmarks is basically the same. There are benchmarks available but we had to learn the hard way that those benchmarks don't really reflect the performance in our game and we have to conclude that there's no easy way of telling if V8 would be faster or slower than Spidermonkey in our case. More on that later. Spidermonkey upgrade Javascript is based on the ECMA standard, but the API (Application programming interface) isn't standardized accross different scripting engines and not even from one version of a scripting engine to the other. This means that changing to another script engine or updating to newer versions is a very time consuming task. We are currently using Spidermonkey v1.8.5 which is the version used in Firefox 4. We have seen benchmarks showing major improvements since Firefox 4 and red a lot of technical articles describing new features solely designed to improve this performance even further. There was the addition of type inference in Firefox 9, the new JIT compiler called Ion Monkey was added in Firefox 18 and a new baseline compiler was added in Firefox 23. We somehow expected that these benchmarking results would be a bit exagerated and that the results would be less spectacular in real world applications like 0 A.D.. Still I was very disappointed after spending dozens of hours adapting to the new version just to notice that the performance in our case is even below the level of version 1.8.5. I've spent even more time and got help from a Mozilla developer but even though we were able to fix some performance related issues in Spidermonkey, we haven't yet managed to get back to the performance before the upgrade. It might be possible that the big changes made recently in Spidermonkey still need some more time until they are polished enough to improve the performance for normal applications and scripts which are not as well tested by the Mozilla developers as the benchmarks. On the other hand we have measured quite a lot of different code parts which more or less all showed the same result as v1.8.5, so I don't expect a huge speedup anyway. The following graph shows the duration of simulation turns in milliseconds. A simulation turn happens all 200ms by default and updates all the simulation components and does AI calculations. Basically it shows the performance as the game progresses going from the left side of the graph to the right. Lower values are better. Check the detailed description below the graph for details! Y-Axis: Duration in milliseconds X-Axis: Turn number divided by 20 This was measured using a non-visual replay which only runs the simulation code and executes AI calculations. It's described in our wiki if you want to do such measurements yourself. Rendering overhead or network performance is not involved because no visuals are displayed and everything runs locally. The game here was a 2vs2 AI game with Aegis bots on the map Oasis 4. You can use this command to watch the AIs fight in the setup that was used for the measurements (but the real measurements were done using the non-visual replay). ./pyrogenesis -quickstart -autostart="Oasis 04" -autostart-ai=1:aegis -autostart-ai=2:aegis -autostart-ai=3:aegis -autostart-ai=4:aegisNot everything on this graph runs in JS. There's the pathfinding and Simulation components that are written in C++, but it runs on the same version of our code and the only difference between the two graphs is the Spidermonkey version. Measuring is a difficult topic and also this measurement has a factor that makes it not 100% accurate. The new Spidermonkey changes the iteration order of some types of loops under some circumstances which causes the AI to behave slightly different than with v1.8.5. For example if it looks for an idle worker, it might pick another one and send him/her to gather wood. There seems to be no easy way to change this back to the old behavior for the measurements without affecting the performance negatively. You notice four things when looking at that graph, The longer the game runs, the slower it gets. This is mainly due to more units being produced which means it takes longer to filter the units by specific criteria, more units move and need to find paths to somewhere, more units are involved in range calculations etc. It's definitely too slow. Later the simulation update takes close to or even more than 200ms which means it will run in each frame. If something that takes more than 200ms runs in each frame, you won't get more than 5 FPS (1000[ms] / 200[ms] = 5 [FPS]). As I already said, the new Spidermonkey is slower. Somewhere after 6000 turns (300 * 20) the performance drastically reduces with the new Spidermonkey. I still have to figure out why this happens. Until now I've looked more at the peaks that happen between turn 0 and 6000. One positive exception regarding performance is the random map loading performance which is now much faster with the new Spidermonkey for those configurations I tested! The exact command used for this measurement was (and I added some code to print the duration): ./pyrogenesis -autostart=alpine_lakes -autostart-random=123 -autostart-size=256 Conclusion regarding performance The conclusion for 0 A.D. is, that we have to redesign our current AI interface a bit. Long running performance intensive tasks need to run in native C++ code because only this way we have the fine grained control we need for optimization and only this way we get the performance we need. The challenge we face will be how the commonly used performance intensive tasks can be separated and moved to a C++ API without loosing the flexibility of scripting in our AI. Even though the performance aspect of the upgrade was a failure, it allowed us to see how far Javascript can get performance-wise and we don't have the expection anymore that it could suddenly run twice as fast with a new version of the script engine. We hope that the competition for better bench-marketing results between the major browser manufacturers will stop better sooner than later. For us a stable API would be much better than some new features aimed for performance improvements. Why the update is still required: We know that Javascript performance most likely won't improve a lot and can design our code accordingly.The Javascript library needs to stay up to date for security reason. This is not a real issue at the moment, but will become more important once we release the first stable version of 0 A.D..Linux distributions don't like to bundle thirth-party libraries with applications and they won't provide Spidermonkey v1.8.5 packages forever.It's important to know about several multi-threading related aspects that changed since v1.8.5 in order to design multi-threading sensibly for the parts that use scripting.The new version fixes bugs, and adds new features.A cleanup of our scripting related code was necessary. We used way to many different approaches which makes adapting to API changes harder, results in duplicated code and makes bugs more likely to occur. These issues were mostly grown historically.The new Spidermonkey is more strict about Javascript syntax and prints more warnings or errors that point developers or modders to potentially buggy code. We have already discovered some issues due to the new warnings.If we need any features or bug fixes in Spidermonkey, we need to have an updated version because Mozilla won't do any changes on v1.8.5.The next steps There's a work in progress patch for integrating the current Spidermonkey 27 development version into 0 A.D. This patch still needs a lot of work before it's ready and I'm going to commit independent parts of it step by step as I already did with some patches. I'm not yet sure which version we will exactly use. We should aim for an "ESR" release which is supported longer and for which a standalone library will be packaged by most Linux distributions. The problem ist that the next ESR version will be released somewhere around July 2014 which is too late. On the other hand it would be a waste to revert the API adaptions from ESR24 to v27 that are already included in the current patch. Links / Resources Trac ticket for the Spidermonkey upgrade with a list of related tickets: #1886Forum thread: Spidermonkey upgradeBugzilla meta-bug for performance fixes in Spidermonkey: Bug 897962

Yes. but they were not nations. Modern people are too attached to national identity i.m.o.

Bienvenido al forum! Buen trabajo con blender. Yo también uso blender, pero solamente para ficción científica, así que tengo poco en deviant art

Quick solution: don't build walls when playing against AI, the AI can't handle it (it creates the same situation as a naval map). Wraitii is looking into fixing those issues. But it will take a long time. I mean, hiding the issues is easy, solving the fact that AI is getting stuck when it's facing a wall is more difficult.

The primary suggestion I make does not consist of that many difficulties. Allow me to elaborate. For the Romans, the Marian reforms would only require one unit, which exists in game. Obviously a previous one could be introduced if possible, but even that is unnecessary. The Spartan reforms would be quite simple I should think. The Perioikoi could possibly become hoplite peltast hybrids at some point while the Spartiate would change from a hoplite to a pikeman. (Another unit which exists.) The Athenian reform would be similarly simple. (The champions are post-reform so they could be unlocked or upgraded upon research while peltasts require only a few changes which are not that hard. (consider the possibility of using Macedonian skirmisher assets)) The Macedonians would be pretty easy to do also since it requires simply using assets of the generic Hellenes. That leaves it to the Ptolemies and the Seleucids, and I am having a difficult time envisioning much anything for them other than Romanization as Sighvatr mentioned.

There is no October update thread. The last work I did was the first week of October where I fixed memory leaks in the new lobby, added VS2012 support for premake4 and some bug-fixes on megapatch (no new features). And the end of September I fell quite ill and was out of action for 2 weeks; actually I've yet to fully recover - I should probably give myself more rest. Since there has yet to be any discussion on resuming paid development, I'm remaining in stand-by.

If Romans get their reforms, then I would like to see romanized infantry for the Ptolemies, Mythos.

FYI, "Myst" makers Cyan Worlds are starting a new ambitious game project, Obduction, and they just launched a Kickstarter, which meets an impressive success : http://www.kickstarter.com/projects/cyaninc/obduction Their goal is set at 1,100,000 !

I had them in my hard drive since some time. I have commited it now. However, I downscaled the textures to 512x512. 1024 seemed like too big for a texture just for one structure. Preview:

Parthian horse armor (bronze): Parthian Phrygian cap:

Rare Parthian unit: Parthian light cavalry with thureos: Parthian infantry:

I found this http://weekly.ahram.org.eg/2002/568/bo8.htm http://www.discoveringegypt.com/rebuildIsis.htm Enter and see the video Trajan's Kiosk, a hypaethral temple, is one of the largest Ancient Egyptian monuments standing today at the island of Agilkia, which was constructed by the Roman Emperor, Trajan.[1] It was originally built at the island of Philae (near the lower Aswan Dam) but transported to Agilika in the 1960s by UNESCO to save it from being enveloped by the rising waters of the Nile due to the construction of the Aswan High Dam.[2] This 15-x-20 metre kiosk is 15.85 metres high; its function was likely "to shelter the bark of Isis at the eastern banks" of Philae island.[3] Its four by five columns each carry "different, lavishly structured composite capitals that are topped by 2.10-metre-high piers" and were originally "intended to be sculpted into Bes piers, similar to the birthhouses of Philae, Armant, and Dendera though this decoration was never completed.[4] The structure is today roofless,[5] but sockets within the structure's architraves suggest that its roof, which was made of timber, was indeed constructed in ancient times.[6] Three 12.50-metre-long, presumably triangulated trusses, "which were inserted into a ledge at the back of stone architecture, carried the slightly vaulted roof."[7] This building represents an example of the unusual combination of wood and stone in the same architectural structure for an Egyptian temple.[8] http://en.wikipedia.org/wiki/Philae http://en.wikipedia.org/wiki/Trajan%27s_Kiosk

Very interesting. Nevertheless, that same source also includes most of Britain as part of the "Celtic World", so it seems that although they think it is very questionable, it is the best hypothesis avaliable.

The stone is what passed the centuries but did it really have an open roof or was it covered by wood or something else?

I am not actually opposed at all to the idea of introducing reforms to the Romans in game. It would introduce an intriguing aspect to the game which could overlap over to other nations. To that extent then, the Macedonians could commence the game with hoplites and then alter them to pikemen, the Spartans could perhaps do the same later, the Athenians could have a similar mechanic with upgrading their peltasts to ridiculously powerful javelin/hoplite hybrids. Even the Persians could have a technology for changing their infantry to a hoplite form.

Hello Gallaecio. Thanks for the information... Hello Mega Mania. Thanks. Nice to meet you too!

personally, i still think it should be Stonehenge that's the Brythonic Wonder i'd recommend making the horse larger so that it's more easily seen, but i otherwise agree with the idea that the hill is physically cut off; that's really the only problem i ever had with the horse as a wonder, since there was nothing to realistically stop units from walking on it

well technicaly, there are two Celtic factions: the Gauls represent the Celts of France, and the Britons represent those of the British Isles. it's no surprise that "Celt" and "Celtic" would crop up for both of them, but there's no civ that's just called "Celts". there used to be, but now they're divided into the Brythonic and Gaulish varieties