Template folder structure.

[Freagarach]

LS (@Nescio, @wowgetoffyourcellphone, @bb_, @wraitii),

in light of D2774 and D2254 I've had some thoughts regarding the folder structure of the templates.

• Currently all "basic" templates are in the "template" folder, which ought to stay that way. But all templates for the factions are in the "templates/units" folder which can get quite cludged. It seems nice to put those in a folder for each civ like is already done in @Nescio's 0abc mod. Idem for structures.
• The "other" folder, which only contains templates inherited from structures may be moved to the "templates/structures" folder.
• The "gaia" folder also is quite packed. May it be an idea to move fauna under "templates/gaia/fauna", flora under "templates/gaia/flora" and geology under "templates/gaia/geology"? One could, however, also argue that fauna are just units and should be under "templates/units/gaia". (Perhaps the whole "gaia" folder is redundant?)

There may be more ways to improve but this is what I could come up with now ^^ Please add stuff as appropriate

[wowgetoffyourcellphone]

For civ folders, do we want 'athenians' or 'athen'? 'Athenians' mirrors the actor structure, while 'athen' mirrors other things.

 

templates

• campaigns
• gaia
  • fauna
  • flora
  • geology
  • ruins
  • treasure
• other
• rubble
• skirmish
  • structures
  • units
• special
  • filter
  • formations
  • player
• structures
  • athen
  • brit
  • cart
  • gaul
  • han
  • iber
  • kush
  • mace
  • maur
  • merc
  • pers
  • ptol
  • rome
  • sele
  • spart
• territory_pulls
• trigger
• units
  • athen
  • brit
  • cart
  • gaul
  • han
  • iber
  • kush
  • mace
  • maur
  • merc
  • pers
  • ptol
  • rome
  • sele
  • spart

 

Fruthermore, will we remove redundancy in the file names?

templates/units/athenians/athen_infantry_hoplite_b

becomes:

templates/units/athenians/infantry_hoplite_b

 

Edited August 7, 2020 by wowgetoffyourcellphone

[Stan`]

1 minute ago, wowgetoffyourcellphone said:

Fruthermore, will we remove redundancy in the file names?

Art file conventions says we should ^^

[wowgetoffyourcellphone]

6 minutes ago, Stan` said:

Art file conventions says we should ^^

As I have 2000 files to edit I'll need to know. lol

[Nescio]

3 hours ago, Freagarach said:

I've had some thoughts regarding the folder structure of the templates.

[...]

There may be more ways to improve but this is what I could come up with now ^^ Please add stuff as appropriate

Thank you! I've been saying this for years.

First a bit of context: three years ago @fatherbushido wrote a ticket ( https://trac.wildfiregames.com/ticket/4770 ) for cleaning up the other/ folder and asked me to work on it. @elexis also participated in that discussion and committed most of the patches:

• D989 / rP21093 (special ruins in gaia to gaia/ruins/) + rP21094 (ruins from other/ to gaia/ruins/ ) + rP21095 (treasures from other/ to gaia/treasure/).
• D1002 / rP20403 (civ structures from other/ to structures/).
• D1007 / rP20473 (plane from other/ to units/).

I also wrote patches for creating subfolders for fauna and flora (D1009) and geology (D1010), but those failed to generate interest and are now outdated and need to be redone from scratch. All that was in 2017.

More recently, in 2019, some further clean-up was done:

• D1798 / rP22266 (territory_pull from other/ to special/).
• D1881 / rP22797 (wallset_palisade from other/ to structures/).
• D2083 / rP22657 (new template_territory_pull.xml and corresponding territory_pulls/ folder).
• D2234 / rP22982 (palisades from other/ to structures/) + rP22983 (a few forgotten map fixes).

What's currently left under other/ are some gaia structures.

In my opinion there ought to be a correspondence between `template_*` files and folders, i.e. everything that inherits from `template_unit.xml` belongs under units/, every file inheriting from `template_structure.xml` going to structures/, and gaia/ containing only the files that inherit from `template_gaia.xml`, as is already the case for e.g. rubble and territory pulls. Effectively this would mean the following:

• Flying birds inherit from `template_bird.xml`, not from `template_gaia.xml`, therefore should be moved to a new birds/ folder: https://code.wildfiregames.com/D2254
• campaigns/ contains three structures and three units; that folder can be deprecated and those files moved to structures/ and units/ respectively: https://code.wildfiregames.com/D2925
• other/ can be deprecated and its files moved to structures/.
• skirmish/ can be deprecated too:

  skirmish/structures/*.xml → structures/skirmish/*.xml
  skirmish/units/*.xml → units/skirmish/*.xml

• finally, animals inherit from `template_unit.xml`, not from `template_gaia.xml`, and thus ought to be moved to units/animals/.

@Freagarach, if you're up to it, I suppose I could write a few more patches.

4 hours ago, Freagarach said:

The "gaia" folder also is quite packed. May it be an idea to move fauna under "templates/gaia/fauna", flora under "templates/gaia/flora" and geology under "templates/gaia/geology"? One could, however, also argue that fauna are just units and should be under "templates/units/gaia". (Perhaps the whole "gaia" folder is redundant?)

While I proposed introducing fauna/, flora/, and geology/ folders in the past (see above), I've changed my mind. Flora and geology are descriptive terms based on visual appearance (i.e. art). What matters for simulation is not how things look like, but what is their function in game. In my opinion the `template_gaia_fauna*` and `template_gaia_geology*` files ought to be split in fruit, tree, rock, and ore, i.e. one file per resource subtype, as is already the case for fish, ruins, and treasure. Once that's done corresponding subfolders can be introduced under gaia/.

And no, the gaia/ folder is certainly not redundant! (Just poorly named: resources would probably be more appropiate, though that isn't perfect either.)

[Nescio]

4 hours ago, Freagarach said:

Currently all "basic" templates are in the "template" folder, which ought to stay that way. But all templates for the factions are in the "templates/units" folder which can get quite cludged. It seems nice to put those in a folder for each civ like is already done in @Nescio's 0abc mod. Idem for structures.

4 hours ago, wowgetoffyourcellphone said:

For civ folders, do we want 'athenians' or 'athen'? 'Athenians' mirrors the actor structure, while 'athen' mirrors other things.

Fruthermore, will we remove redundancy in the file names?

templates/units/athenians/athen_infantry_hoplite_b

becomes:

templates/units/athenians/infantry_hoplite_b

The civ code; the full name doesn't work for the simulation folder. So basically:

{civ}_*.xml → {civ}/*.xml

See e.g. https://github.com/0abc/0abc-a23/tree/master/simulation/templates/units/athen

The great advantage is that you would end up with a few dozen folders with a few dozen files each, rather than one folder with over a thousand files. This would make things much more manageable in the long term, and also more accommodating for future additions.

4 hours ago, wowgetoffyourcellphone said:

As I have 2000 files to edit I'll need to know. lol

While it involves many files, the moves are simple and straightforward. Most of the file corrections can be done by script (sed). 

[wowgetoffyourcellphone]

Well, just tell me what you decide.

[Loki1950]

3 hours ago, Nescio said:

Most of the file corrections can be done by script (sed)

On the OS's that have it

Enjoy the Choice

[GunChleoc]

Install Git for Windows ad you'll have sed on Windows too, in the Git Bash.

Alternatively, Notepad++ can also do a regex search/replace on all files in a folder.

Edited August 7, 2020 by GunChleoc

[Freagarach]

Thank you for the history @Nescio!

On 8/7/2020 at 11:03 AM, wowgetoffyourcellphone said:

As I have 2000 files to edit I'll need to know. lol

I can try a PR again, like with the javelin*.

On 8/7/2020 at 3:25 PM, Nescio said:

if you're up to it, I suppose I could write a few more patches.

I'll kick off with the, IMHO, least controversial of changes:

{civ}_*.xml → {civ}/*.xml

since I wrote a script for that while practicing my bash ^^ Then we can also gather more feedback in the diff (D2952).

I agree about the "other"-folder, but am not sure about "campaign" and "skirmish".

The Gaia-folder may also be controversial, perhaps we should wait with that one for another team members opinion (a few are on vacation now ^^).

Edited August 14, 2020 by Freagarach
Typo, add diff.

[Nescio]

2 minutes ago, Freagarach said:

I'll kick off with the, IMHO, least controversial of changes:

{civ}_*.xml → {civ}/*.xml

since I wrote a script for that while practicing my bash ^^ Than we can also gather more feedback in there.

That's the least controversial?

[Freagarach]

13 hours ago, Nescio said:

That's the least controversial?

To me, it is (Or perhaps the "other"-folder.)

On 8/7/2020 at 3:25 PM, Nescio said:

one file per resource subtype

But what if one can gather two resources from one object, e.g. fruit and/or wood from a apple tree?

On 8/7/2020 at 3:25 PM, Nescio said:

finally, animals inherit from `template_unit.xml`, not from `template_gaia.xml`, and thus ought to be moved to units/animals/

On 8/7/2020 at 3:25 PM, Nescio said:

And no, the gaia/ folder is certainly not redundant! (Just poorly named: resources would probably be more appropiate, though that isn't perfect either.)

It seems we need something of a definition for that folder.

[wowgetoffyourcellphone]

I thought this restructuring was to reduce the sheer number of entities in each folder and help with organization. lol

Edited August 15, 2020 by wowgetoffyourcellphone

[Nescio]

1 hour ago, wowgetoffyourcellphone said:

I thought this restructuring was to reduce the sheer number of entities in each folder and help with organization. lol

Yes, that too, but the purpose is also to have clear and unambiguous folder names, e.g. currently most structures are under structures/, but some others are under campaigns/, other/, and skirmish/.

1 hour ago, Freagarach said:

To me, it is (Or perhaps the "other"-folder.)

It is a straightforward change, yes, but it also involves by far the largest number of files and affects the most mods.

1 hour ago, Freagarach said:

But what if one can gather two resources from one object, e.g. fruit and/or wood from a apple tree?

Right now that's not possible, is it? And if there is a code change to make it possible, how would it work in game? Can both resources be gathered simultaneously? Then how do workers know which one they ought to gather? Or can one be gathered only after the other is exhausted? If so, then it could already be done with <SpawnEntityOnDeath>, and that would mean they're two separate entities, one fruit and one wood tree, right?

1 hour ago, Freagarach said:

It seems we need something of a definition for that folder.

In my opinion the most consistent option would be:

birds/           ↔ template_bird.xml
gaia/            ↔ template_gaia.xml
rubble/          ↔ template_rubble.xml
structures/      ↔ template_structure.xml
territory_pulls/ ↔ template_territory_pull.xml
triggers/        ↔ template_trigger_point.xml
units/           ↔ template_unit.xml

 

[wowgetoffyourcellphone]

Just now, Nescio said:

Yes, that too, but the purpose is also to have clear and unambiguous folder names, e.g. currently most structures are under structures/, but some others are under campaigns/, other/, and skirmish/.

Indeed, but you don't need 20 more folders to do that. Case in point:

birds/           ↔ template_bird.xml
gaia/            ↔ template_gaia.xml

What is this? Why can't birds just be part of the gaia/fauna folder? 

[Nescio]

Just now, wowgetoffyourcellphone said:

Indeed, but you don't need 20 more folders to do that. Case in point:

birds/           ↔ template_bird.xml
gaia/            ↔ template_gaia.xml

What is this? Why can't birds just be part of the gaia/fauna folder? 

To be clear, birds inherit from template_bird.xml (which already exist), not from template_gaia.xml (they never did), and they don't have <Health>, <Identity>, or <ResourceSupply> nodes, and can't be killed or selected in game; birds just fly around and are completely different and independent from other entities. Right now there is only one, the erroneously named gaia/fauna_hawk.xml (it's actually a buzzard).

The chicken, peacock, etc. inherit from template_unit_fauna.xml and should thus be grouped alongside the other animals, as is already the case.

[wowgetoffyourcellphone]

The end-user is going to expect birds to be lumped with other animals is all I'm saying. We're not just organizing these things for us, or just for Nescio, but for the end-user who isn't anal about template classifications. But perhaps there could be some kind of "eyecandy" or "beautification" folder for templates like birds that don't effect the simulation. Map maker can search for "eye candy" at that stage of map design and see the birds. 

Edited August 15, 2020 by wowgetoffyourcellphone

[Nescio]

1 minute ago, wowgetoffyourcellphone said:

The end-user is going to expect birds to be lumped with other animals is all I'm saying. We're not just organizing these things for us, or just for Nescio, but for the end-user who isn't anal about template classifications. But perhaps there could be some kind of "eyecandy" or "beautification" folder for templates like birds that don't effect the simulation. Map maker can search for "eye candy" at that stage of map design and see the birds. 

The end-users are players, and for them file names etc. are completely irrelevant.

However, for future additions, modifying the game, and making maps it helps if file names are reflective of what they are. This is already the case for e.g. pyramids:

gaia/ruins/pyramid_great.xml
gaia/ruins/pyramid_minor.xml
structures/kush_pyramid_large.xml
structures/kush_pyramid_small.xml

The folder paths make it clear the former two have a different function than the latter two. Likewise, it would make sense to have fruit and wood trees in separate gaia subfolders, to make it less likely map makers mistakingly pick one functionally different from what they intend. And if at some point there are e.g. different geese, then

birds/goose.xml
gaia/fauna/goose.xml

would make it clear which one flies around and does nothing, and which one can be killed to supply food like other fauna.

[wowgetoffyourcellphone]

You do you bro. 

[Freagarach]

Templates were moved to their civ-specific folders (r24216). One can use https://code.wildfiregames.com/P225, or a adaptation thereof, to fix the majority of changes.

[Nescio]

Currently I'm contemplating inserting a new `template_unit_soldier.xml`, which would serve as a shared parent for infantry, cavalry, elephants, champions, and heroes; e.g.:

Spoiler

<?xml version="1.0" encoding="utf-8"?>
<Entity>
  <Attack>
    <Capture>
      <AttackName>Capture</AttackName>
      <Capture>2.5</Capture>
      <MaxRange>4</MaxRange>
      <RepeatTime>1000</RepeatTime>
      <RestrictedClasses datatype="tokens">Field Palisade Wall</RestrictedClasses>
    </Capture>
  </Attack>
  <Identity>
    <GenericName>Soldier</GenericName>
    <Classes datatype="tokens">Organic Human ConquestCritical</Classes>
    <VisibleClasses datatype="tokens">Soldier</VisibleClasses>
  </Identity>
  <ResourceGatherer>
    <MaxDistance>2.0</MaxDistance>
    <BaseSpeed>1.0</BaseSpeed>
    <Rates>
      <treasure>1</treasure>
    </Rates>
    <Capacities>
      <food>10</food>
      <wood>10</wood>
      <stone>10</stone>
      <metal>10</metal>
    </Capacities>
  </ResourceGatherer>
  <Sound>
    <SoundGroups>
      <order_attack>voice/{lang}/civ/civ_{phenotype}_attack.xml</order_attack>
      <order_garrison>voice/{lang}/civ/civ_{phenotype}_garrison.xml</order_garrison>
      <order_gather>voice/{lang}/civ/civ_{phenotype}_gather.xml</order_gather>
      <order_walk>voice/{lang}/civ/civ_{phenotype}_walk.xml</order_walk>
      <select>voice/{lang}/civ/civ_{phenotype}_select.xml</select>
    </SoundGroups>
  </Sound>
</Entity>

 

The resulting template tree would be:

Spoiler

template_unit.xml
template_unit_catafalque.xml
template_unit_dog.xml
template_unit_fauna*
template_unit_ship*
template_unit_siege*
template_unit_soldier*
template_unit_support*

 

The advantage is that it would allow for cleaner templates with less duplicate lines.

The disadvantage is that as a consequence, dozens of generic templates have to be renamed and parents adjusted in dozens specific units/* files, though that could be done by script.

Thoughts?

[Freagarach]

To me the idea sounds sane. But given that only four templates would get cleaner, I'm not sure it is worth it right now.
If you would provide a script that requires very limited manual effort we obviously cannot refuse ;)

[wraitii]

(also this would prevent siege & possibly ships from collecting treasures... Which may or may not be desirable).

I don't really feel strongly either way beyond that. It makes sense, I can also see it as being un-necessary indirection. The dog template should maybe inherit from Soldier?

[Nescio]

1 hour ago, Freagarach said:

To me the idea sounds sane. But given that only four templates would get cleaner, I'm not sure it is worth it right now.

The new template would have five children, though it's not inconceivable more will be added in the future (e.g. camels). More importantly, it makes `template_unit.xml` and its other children cleaner, thereby also improving mod support: if someone would want to introduce templates for hot air balloons, zeppelins, tanks, helicopters, spacecraft, dragons, etc., then fewer lines would have to be disabled or replaced.

I'll write a patch to better show you what I mean (and yes, I'll provide a sed script).

1 hour ago, wraitii said:

It makes sense, I can also see it as being un-necessary indirection.

Generally I prefer having fewer files and shorter file names. In the past I've written patches for deprecating `template_entity_full.xml`, `template_entity_quasi.xml`, and `template_unit_mechanical.xml`. I also think `template_structure_civic.xml`, `template_structure_defensive.xml`, etc. are not very useful and that the about two dozen `template_unit_fauna_*` templates should be reduced to two or three at most ( https://code.wildfiregames.com/D2953 ).

However, in this particular case, I think the benefits of having a soldier template level outweigh the drawbacks, especially in the long run.

1 hour ago, wraitii said:

The dog template should maybe inherit from Soldier?

No. Dogs are not soldiers and its contents are actually quite similar to dogs inheriting from `template_unit_fauna.xml`; the most important difference is the UnitAI.

I actually contemplated a `template_unit_human.xml` instead of soldier, which could then also serve as a parent for support units too, but I think that's unnecessary and a soldier template is cleaner.

[wowgetoffyourcellphone]

I can't wait!