Entity–component–system (ECS)

[dmzerocold]

Hello,

is it possible to implement ECS for 0A.D ? 

https://en.wikipedia.org/wiki/Entity_component_system

Can it fix the lag issue (AI pathfinding )?

 

[Feldfeld]

Doesn't 0 A.D. already use it ?

[Stan`]

We use an ECS. Each of the object in the game have a bunch of components. A health component, a UnitAI component a sound Component etc. The title of the video is a bit misleading. What they do is optimize the scene on the fly using LODs (Which we don't have and are also Components (One could add that component to every entity and switch meshes on the fly depending on the zoom level)) GL optimizations (which we can't use because of the old opengl version) Multithreading which we only partially have. And finally they optimize stuff for platforms.

[jrevel]

There is not only that, Unity uses components from the beginning, but the ECS system is very recent. The difference with the new system is that it is implemented using the Data Oriented paradigm. It is mostly a bunch of good practices that go against a lot of bad habits that OOP tend to give, and that drive performance down. For example, virtual functions is not good to use everywhere, it has a performance cost, a tiny one, but that scales when used for every function every frame. It is quite hard to learn, because it needs to shift our mindset, but it gives a lot of advantages : better performance, but also simpler code with no side effect, and way easier to multithread... And it makes also lodding easier.

This book explains it perfectly (but is quite technical though) http://www.dataorienteddesign.com/dodbook/

The problem of that paradigm is that it needs to shift the way we think about entities, and would need a lot of refactoring to implement. But it is possible to isolate portions of code that are slow and optimise it, so it can be done step by step.

[Stan`]

6 minutes ago, jrevel said:

There is not only that, Unity uses components from the beginning, but the ECS system is very recent. The difference with the new system is that it is implemented using the Data Oriented paradigm. It is mostly a bunch of good practices that go against a lot of bad habits that OOP tend to give, and that drive performance down. For example, virtual functions is not good to use everywhere, it has a performance cost, a tiny one, but that scales when used for every function every frame. It is quite hard to learn, because it needs to shift our mindset, but it gives a lot of advantages : better performance, but also simpler code with no side effect, and way easier to multithread... And it makes also lodding easier.

This book explains it perfectly (but is quite technical though) http://www.dataorienteddesign.com/dodbook/

The problem of that paradigm is that it needs to shift the way we think about entities, and would need a lot of refactoring to implement. But it is possible to isolate portions of code that are slow and optimise it, so it can be done step by step.

Well we have a lot of improvements to make  

[elexis]

https://trac.wildfiregames.com/wiki/SimulationArchitecture

https://trac.wildfiregames.com/wiki/Entity

https://trac.wildfiregames.com/wiki/SimulationRequirements

Edit: The performance they get is amazing. Even with logarithmic cost data structures, level-of-detail variants and the well maintained unity renderer I have trouble explaining myself how they can run this scene on a mobile phone.

[jrevel]

Applied to an RTS game...

 

[Stan`]

• So 96% Multithreading → We don't have
• Navigation Meshes → We don't have
• Lods → We don't have
• Advanced Particles → We don't have.
• Instancing → We don't have 

[nani]

• No Javascript -> We do have -> Slow ->

[WhiteTreePaladin]

2 hours ago, stanislas69 said:

• So 96% Multithreading → We don't have
• Navigation Meshes → We don't have
• Lods → We don't have
• Advanced Particles → We don't have.
• Instancing → We don't have 

Well, that means that we can potentially add some of that. If we already had all of those, then there wouldn't be as much room for improvement. Multithreading and LoD are some things that we have wanted for awhile. Do we not already have instancing? I assumed that common static models like trees and structures are instanced, right...?

1 minute ago, nani said:

• No Javascript -> We do have -> Slow ->

Yeah, originally the scripting language was going to be Python, but JavaScript was chosen instead. JavaScript has gotten a lot of attention and performance improvements over the years, so I don't know that Python would have been any better. We could port some things back to C++ if there is a clear reason to do so.

[Sundiata]

Isn't instancing the holy grail of performance improvements in graphics rendering? Basically anything that is rendered more than 2 or 3 times... Especially for trees, grasses and all other flora, as well as things like rocks, boulders, cliffs, which add a lot of character to maps, but can't be used to their full potential right now because of performance. Even things like farms, and houses, walls (basically any geometry that's rendered more than a few times).

I barely know what I'm talking about, but in blender for example, instancing makes the impossible possible. Like rendering a high poly forests on a laptop. The performance improvement is enormous. Can't be understated.  

I also read this piece about instancing huge numbers of animated meshes (with their own textures) a while ago, making me curious to what extent even units could be instanced?  https://forum.unity.com/threads/experiments-with-instancing-and-other-methods-to-render-massive-numbers-of-skinned-meshes.447749/

[Stan`]

3 hours ago, WhiteTreePaladin said:

Yeah, originally the scripting language was going to be Python, but JavaScript was chosen instead. JavaScript has gotten a lot of attention and performance improvements over the years, so I don't know that Python would have been any better. We could port some things back to C++ if there is a clear reason to do so.

Well LUA is usually the performance choice for scripting. Do you happen to have a link to a thread where we explain why we did choose one over another ? I get that question a lot and I don't have a solid answer.

3 hours ago, nani said:

No Javascript -> We do have -> Slow -> 

Js is not slow by itself. It's the interfacing that slows things down. Also having multiple ha contexts/threads would help. Currently only the AI and maybe the aGUI are separated.

@Sundiata well it does help a lot for static meshes grass trees props. However low poly count LODs and some other tricks might help more.

The issue with units is that they are moving so their geometry is different due to variations their textures and props are different so it might not be instanced. But combined with loss and the old mesh one could reduce the performance strain by changing the actor depending on the zoom.

[Guest]

Since we are talking about old stuff, there might be some reason to consider salvaging all the history articles written back then. Here is the first thing I could find. Late 2004. (I suppose all of these were lost in a server migration or something?. Long live the internet archive!)

Quote

 

Great Celtic Battles By Paul Basar aka Paal_101 As a people the Celts were warlike and their many conflicts with various peoples were often bloody and vicious. The Romans proved to be their nemesis, although they fought the Dacians, Greeks, Germans, and each other. They were incredibly brave, terrified only by their ferocious Germanic neighbors. Although these opponents were formidable, the Romans proved to be their undoing and in particular Julius Caesar, who conquered the continental Gauls and unsuccessfully invaded Britain twice. By the time Rome had conquered Britain millions of Celts had died at the hands of the legions and many more had been sold as slaves. Yet through it all the fighting capacity of the Celts has remained legendary to this day, as their list of battles well attests. Battle of the Allia River – 390 BC* Brennus, a chieftain of the Senones tribe from northern Italy, moves with an army of 30,000 men towards Rome in a massive raid. Marching to meet them is a force of roughly 10,000 men from Rome’s two standing legions. Meeting on the banks of the Allia River, a tributary of the Tiber, the Celts smash the Roman flank, driving many of them into the stream. With the legions defeated, the Senones charge towards Rome. Sack of Rome – 390 BC Three days after the Battle of the Allia River, the Senones arrive in Rome only to find it practically deserted. Defeating the paltry force left to oppose them, the Gauls discover the Roman senators in their official seats. After one strikes a Celtic warrior with his staff, the senators are slaughtered and the city is systematically destroyed except for the Capitoline Hill. Extracting huge tribute from the Romans, Brennus utters the famous quote “Vae victis” before returning to the north. Third Battle of Thermopylae – 279 BC* Another Brennos leads Celts of central Europe down into Macedonia and Greece. The Hellenes are terrified of the brutal warriors and make a stand to stop them at Thermopylae, already famous for several previous battles. The 40,000 Gauls are faced by a large army of 25,000 Greeks in the Pass who hold their position for one day against repeated assaults. Brennos diverts cavalry to attack Aetolia, causing that contingent of the Greek army to leave for home. Local Greeks are disgusted and show the Celts the same route used by the Persians in 480 BC. This time the Greeks manage to hold of the Gallic attack and escape onboard Athenian ships. Sack of Delphi – 279 BC Taking full advantage of his victory, Brennos leads his troops to the Greek temples at Delphi. The local Phocians attack the Gauls advancing along the narrow, winding mountain road with missile weapons, but the Celts reach Delphi and take it, looting it of precious objects. Among those killed is the Pythia, sacred soothsayer of Apollo. Brennos is wounded and the Celts are harried all along their journey north, with their leader dying at some point during the march. The Celts are driven back, but leave much turmoil despite being severely beaten at the Battle of Spercheios as they retreat. Attalos Defeats the Galatians - 240 BC Northern Greeks, trying to destabilize the rule of Antiochus I, in 278 BC transport an army of Gauls to Asia Minor. Instead they settle to form the nation of Galatia in the west of modern Turkey. Renowned for their ferocity, the Galatians are recruited by Antiochus Hierax as mercenaries in a bid to exert power. Attalos from the Asian Greek city of Pergamon defeats them in a decisive battle, allowing the victorious general to claim the title of king, completely severing the territory from Seleucid control. Battle of Telamon – 225 BC Cis-Alpine Gauls gather to attack the Romans, charging south into Etruria in Italy with 70,000 men. They are met by a Roman army of roughly 30,000 men under the command of the two consuls for that year. By skillful maneuvering, the Celts are surrounded before the battle begins. While some of their infantry fight back-to-back against the Romans on either side, the Gallic cavalry is defeated despite the death of one of the consuls. Utterly crushed, the Gauls suffer 40,000 casualties. Battle of Clastidium - 222 BC Trying to draw off the legions attacking Acerrae in northern Italy an army of 30,000 Gauls attack Casteggio, a Roman supply base. Roman troops move off to attack the Gauls and meet them at Clastidium. Charging at the smaller Roman force, the Celts are unable to break through the lines and their leader Britomartus is struck down by the 60-year-old consul. This is the last great purely Celtic battle between the Gauls and Romans in Italy. Battle of Lake Trasimene – 217 BC Gauls in the employ of the Carthaginians ambush a Roman force of 25,000 men along the shores of Lake Trasimene. Making up a large part of the mercenary Punic army, the Celts surge forward form hidden positions along the lakeside road to trap the Romans against the water. In brutal fighting the Gauls are responsible for many Roman casualties, including the consul Flaminius. Roughly 15,000 Romans are killed, in exchange for roughly 2000 Carthaginian troops. Battle of Cannae – 216 BC With 90,000 Roman troops coming at the roughly half as big Carthaginian army of Hannibal. The roughly 25,000 Celts in his army are used as cannon fodder, holding back the massive Roman force in the center while the rest of the Carthaginian army surrounds the enemy. The Gauls suffer high casualties, but the Romans lose a staggering 50,000 men. Battle of Arausio – 105 BC ** Migrating tribes from Jutland known as the Teutones and Cimbri move south through Gaul towards Roman controlled territory. Included among the small groups of other tribes accompanying the horde is a Celtic tribe known as the Tigurini. The Celto-Germanic invaders crush a Roman army sent to stop them at Arausio, modern day Orange. Roman losses are significant, causing great consternation in Italy. Battle of Aquae Sextiae – 102 BC ** After traveling through parts of Spain and southwest Gaul, the Ambrones and Teutones are confronted by the great Roman general Marius at Aquae Sextiae. The Cimbri had separated from the former tribes, leaving the roughly 100,000 strong Teutonic-caravan of men, women, and children up against a Roman army of 32,000 men. The Ambrones are defeated at a river crossing. Teutonic reinforcements arrive and attack the Romans, only to be held off and then ambushed as they pull back. Roman troops completely destroy the Teutonic horde. The Cimbri fair no better, being resoundingly crushed at Vercellae in 101 BC. Defeat of the Helvetii – 58 BC Pushed out of their homelands by the fierce Germanic Suebi, the Gallic Helvetii tribe finds itself in the situation of asking permission from Rome to cross modern day Switzerland, in addition to Gaul, in search of new land. Julius Caesar commands this territory and refuses due to concerns over other tribes attempting the same thing. In response he builds a rampart between Lake Geneva and the mountains to stop the Helvetii. The Gauls make their way around the fortifications through the mountains only to be set upon by Caesar’s five legions. The escaping Celts are across the Arar River, but the Romans quickly bridge it. The Helvetii turn and attack, but are beaten. After their surrender, the Helvetii are escorted back to their ancestral homeland. Battle of the Sambre – 57 BC *** After badly defeating the Belgae Gaul federation during his conquest of Gaul, Caesar is able to make peace with all tribes but the Nervii. They have advanced notice of the Roman marching route along the Sambre River. Hiding in the trees on the opposite side of the shallow river the roughly 40,000 Celts wait until the six vanguard legions of the Roman army begin to entrench themselves for the night. Swarming across the river the Belgae Gauls slam into the Roman forces before they have the time to don their helmets. The surprise attack is so effective that Caesar himself jumps into the thick of the fighting. But the superior Roman organization and officers combined with the arrival of the two rearguard legions allow Caesar to crush the Nervii for good. Defeat of the Veneti –56 BC After capturing Roman officers for political ransom, the Gallic Veneti, living along the western coast, are attacked by Julius Caesar. Retreating to their tide-surrounded, precipice-mounted fortresses the Gauls use their strong navy to remove their people from any strongholds that were about to fall to the Romans. Building a fleet from scratch, the Romans destroy the superior vessels of the Veneti by burning their sails and immobilizing them. This leaves their citadels vulnerable to eventual capture. As to Roman policy, the Veneti are sold en masse into slavery. Battle on Deal Beach – 55 BC Having cleared the only significant Celtic naval threat, Caesar launches a reconnaissance raid with two legions to Britain. Ostensibly to punish the Benovaci Britons for supporting their Gallic allies, the expedition sails along the cliffs of Dover only to find massed formations of Celts on the bluffs. Discovering a beach roughly 7 miles away only finds the Britons waiting for them, large numbers of infantry, horsemen, and chariots arrayed on the sand. As the Romans land the Britons fight furiously but are frightened by the Roman galleys and the determined attack of the legions. They are driven off, but Caesar returns to Gaul. Second Invasion of Britain – 54 BC Caesar returns in force with five legions and 800 ships, the largest amphibious invasion until World War II. The Britons are unable to slow the Roman advance inland, uniting under the command of Cassivellaunus. The Celts wait for the Romans, who make it across the Thames to attack the Britons. Again Caesar defeats a Celtic army in all out combat, but returns to Gaul soon after. Seige of Alesia - 52 BC Vercingetorix of the Arverni from central Gaul collects a large army in revolt against the Romans. After suffering several defeats he withdraws his force to the strong hill fort of Alesia. Julius Caesar marches after him and digs massive siege works completely around the fortress thanks to the manpower of his 10 legions. The Gauls heavily outnumber the Romans, having 95,000 men within the city itself, while another 250,000 Celts arrive to lift the siege. Despite having a ratio of 7:1 in their favor the Gauls are unable to break through the Roman lines from either side. When a major attempt to escape the Roman defenses fails, the relief army flees and the residents of Alesia surrender. These include Vercingetorix, who is executed in Rome five years later. Battle of Medway – 43 BC Roughly 90 years after the attacks by Julius Caesar, the Britons face a Roman invasion force made up of 40,000 men. Rallied by the Catuvellaunian chieftain Caratacus to defend the city of Camulodunum, 80,000 Celts array on the far bank of the Medway River to prevent the Romans from marching further. An attack by German auxiliaries disables the Celtic cavalry and draws the Briton’s attention. While fighting the auxiliaries, the Britons do not notice a Roman legion crossing the river at a fordable point. The Celts find themselves under attack from these troops on their rear, while another legion follows the first. The next day the Celts flee, forfeiting one of the most decisive battles in English history. This brings most of Britain under Roman control. Battle of Caer Caradock – 50 AD Fleeing to Wales, Caratacus continues his defense against the Romans in an increasingly lost cause. Fighting with the Ordovice and Silure tribes, the Celtic leader is attacked by Roman forces near his fortress of Caer Caradock. The Britons sit in a well-defended position between two hills and across a river from the 20,000 Romans. Despite the Celts’ best efforts the Romans break through their line and the Britons flee to safety. Caratacus escapes to northern England but is turned over to the Romans and taken before Emperor Claudius. Storming of Anglesey – 60 AD The stronghold of the druids, the island of Anglesey faces the might of a Roman force sent to exterminate the rebels who have gathered there. The druids themselves are targeted for their fostering of rebellion in Britain. Celtic warriors line along the beaches, while druids and women stand in between the formations shouting and cursing the Romans. Roman heavy artillery set up on the mainland side of the strait that separats them from the island open up as the legions and cavalry cross the water in boats or by swimming. Once the Romans make the island shore the Celts cannot hold them back and are slaughtered by the thousands. The druids are burned alive within their sacred oak groves. This attack lives on in the minds of British Celts for many years thanks to its ferocity and the blow it strikes against Celtic culture. Battle of Watling Street – 61 AD Whilst the Romans are attacking Anglesey the Iceni tribe rise up in rebellion under their queen, Boudicca. She destroys three Roman towns and part of a legion before she turns to face the main Roman force in a grove near the route of Watling Street. She leads 60,000 Celts against 11,000 Romans bottled in a defile surrounded by treed hills. Charging straight into this narrowing space, the Celts are struck down by Roman pila and then engaged hand-to-hand. Forced back upon the wagons loaded with their watching families, the Britons are entangled and unable to move. The army is slaughtered almost to a man. Boudicca dies soon after, possibly by suicide. Battle of Mons Graupius – 84 AD The Caledonians of northern Britain face a Roman army sent to bring them to heel. Defending their grain supply, the 60,000 Caledonians stand at the base of a hill called Mons Graupius by the Romans, who number 20,000. The Romans quickly attack after trading javelins with the Celts, who are defeated with a loss of 10,000 lives compared to the Romans 360. This battle marks the end of major resistance in England. Scottish Raids – 367 AD The Celtic Scots raid the Roman-controlled northern British coast from their bases in Ireland. At this time the Saxons and Picts attack Roman towns and positions throughout the island. It is so bad that the Roman governor flees to the mainland, order not being restored until 369 AD. The Scots, Welsh, Cornish, Irish, Manx, and Bretons are the only Celtic cultures to survive to the present day. *Concerning the name Brennus/Brennos, these are not from the same man, rather it is believed to be a title derived from the Celtic word for king: brenin. ** There is much debated over the ethnicity of the Cimbri and Teutones. Although the Teutone’s name was eventually used to describe Germanic peoples in general and both tribes are usually considered as such. Strangely enough the word “Teutone” is originally Celtic and both peoples spoke Celtic. The safest statement is to say that they were a mixture of Celtic and Germanic stock, similar to the Belgae Gauls. Whatever their origin, they picked up Gallic warriors and migrants as they went along, including the Tigurini. ***The Belgae Gauls were similar to the Cimbri and Teutones in that they were a combination of Celts and Germans. Ethnologically and culturally close to the Germans, they were nonetheless users of a hybrid Celtic-Germanic language.

 

Edited April 20, 2019 by Guest