Jump to content

Moving Components Schemas to XML files

Stan`
 Share

Recommended Posts

Stan` Consul

Stan`

Posted
Posted

I've been recently annoyed by having to edit indented strings in the ingame components and I was wondering if it wouldn't have been nice to have them as separate xml files instead. here is an example for the current resource supply component. 

diff --git a/binaries/data/mods/public/simulation/components/ResourceSupply.js b/binaries/data/mods/public/simulation/components/ResourceSupply.js
index a015a190ee..1bc2ee9a5d 100644
--- a/binaries/data/mods/public/simulation/components/ResourceSupply.js
+++ b/binaries/data/mods/public/simulation/components/ResourceSupply.js
@@ -1,89 +1,7 @@
 function ResourceSupply() {}
 
-ResourceSupply.prototype.Schema =
-	"<a:help>Provides a supply of one particular type of resource.</a:help>" +
-	"<a:example>" +
-		"<Amount>1000</Amount>" +
-		"<MaxAmount>1500</MaxAmount>" +
-		"<Type>food.meat</Type>" +
-		"<KillBeforeGather>false</KillBeforeGather>" +
-		"<MaxGatherers>25</MaxGatherers>" +
-		"<DiminishingReturns>0.8</DiminishingReturns>" +
-		"<Change>" +
-			"<AnyName>" +
-				"<Constraint>Alive</Constraint>" +
-				"<Value>2</Value>" +
-				"<Interval>1000</Interval>" +
-			"</AnyName>" +
-			"<Growth>" +
-				"<Constraint>Alive</Constraint>" +
-				"<Value>2</Value>" +
-				"<Interval>1000</Interval>" +
-			"</Growth>" +
-			"<Decay>" +
-				"<Constraint>Dead</Constraint>" +
-				"<Value>-1</Value>" +
-				"<Interval>1000</Interval>" +
-				"<Delay>2000</Delay>" +
-				"<Limit>500</Limit>" +
-			"</Decay>" +
-		"</Change>" +
-	"</a:example>" +
-	"<element name='KillBeforeGather' a:help='Whether this entity must be killed (health reduced to 0) before its resources can be gathered'>" +
-		"<data type='boolean'/>" +
-	"</element>" +
-	"<element name='Amount' a:help='Amount of resources available from this entity'>" +
-		"<choice><data type='nonNegativeInteger'/><value>Infinity</value></choice>" +
-	"</element>" +
-	"<optional>" +
-		"<element name='MaxAmount' a:help='The max amount of resource the entity can reach when growing'>" +
-			"<ref name='nonNegativeDecimal'/>" +
-		"</element>" +
-	"</optional>" +
-	"<element name='Type' a:help='Type and Subtype of resource available from this entity'>" +
-		Resources.BuildChoicesSchema(true, true) +
-	"</element>" +
-	"<element name='MaxGatherers' a:help='Amount of gatherers who can gather resources from this entity at the same time'>" +
-		"<data type='nonNegativeInteger'/>" +
-	"</element>" +
-	"<optional>" +
-		"<element name='DiminishingReturns' a:help='The relative rate of any new gatherer compared to the previous one (geometric sequence). Leave the element out for no diminishing returns.'>" +
-			"<ref name='positiveDecimal'/>" +
-		"</element>" +
-	"</optional>" +
-	"<optional>" +
-		"<element name='Change' a:help='Optional element containing all the modifications that affects a resource supply'>" +
-			"<zeroOrMore>" +
-				"<element a:help='Optional element defining whether and how a resource supply regenerates or decays'>" +
-					"<anyName/>" +
-					"<interleave>" +
-						"<optional>" +
-							"<element name='Constraint' a:help='Specifies the health constraint for the change to be active'>" +
-								"<choice>" +
-									"<value>Alive</value>" +
-									"<value>Dead</value>" +
-								"</choice>" +
-							"</element>" +
-						"</optional>" +
-						"<element name='Delay' a:help='Delay in milliseconds before the object starts growing or decaying'>" +
-							"<ref name='nonNegativeDecimal'/>" +
-						"</element>" +
-						"<element name='Value' a:help='The amount of resource added per interval'>" +
-							"<data type='decimal'/>" +
-						"</element>" +
-						"<element name='Interval' a:help='The interval in milliseconds'>" +
-							"<data type='positiveInteger'/>" +
-						"</element>" +
-						"<optional>" +
-							"<element name='Limit' a:help='The upper or bottom limit of the value after which the change has no effect'>" +
-								"<data type='nonNegativeInteger'/>" +
-							"</element>" +
-						"</optional>" +
-					"</interleave>" +
-				"</element>" +
-			"</zeroOrMore>" +
-		"</element>" +
-	"</optional>";
+ResourceSupply.prototype.Schema = Engine.ReadFile("simulation/components/schemas/ResourceSupply.xml").replace("Resources.BuildChoicesSchema(true, true)", Resources.BuildChoicesSchema(true, true));
+
 
 ResourceSupply.prototype.Init = function()
 {
diff --git a/source/ps/scripting/JSInterface_VFS.cpp b/source/ps/scripting/JSInterface_VFS.cpp
index ec1a1d0572..71840a513b 100644
--- a/source/ps/scripting/JSInterface_VFS.cpp
+++ b/source/ps/scripting/JSInterface_VFS.cpp
@@ -268,6 +268,7 @@ void JSI_VFS::RegisterScriptFunctions_Simulation(const ScriptInterface& scriptIn
 	scriptInterface.RegisterFunction<JS::Value, std::wstring, std::wstring, bool, &Script_ListDirectoryFiles_Simulation>("ListDirectoryFiles");
 	scriptInterface.RegisterFunction<bool, std::wstring, Script_FileExists_Simulation>("FileExists");
 	scriptInterface.RegisterFunction<JS::Value, std::wstring, &Script_ReadJSONFile_Simulation>("ReadJSONFile");
+	scriptInterface.RegisterFunction<JS::Value, std::wstring, & JSI_VFS::ReadFile>("ReadFile");
 }
 
 void JSI_VFS::RegisterScriptFunctions_Maps(const ScriptInterface& scriptInterface)

And the side file in simulation/components/schemas 

<?xml version="1.0" encoding="UTF-8"?>
<a:help>Provides a supply of one particular type of resource.</a:help>
<a:example>
    <Amount>1000</Amount>
    <MaxAmount>1500</MaxAmount>
    <Type>food.meat</Type>
    <KillBeforeGather>false</KillBeforeGather>
    <MaxGatherers>25</MaxGatherers>
    <DiminishingReturns>0.8</DiminishingReturns>
    <Change>
        <AnyName>
            <Constraint>Alive</Constraint>
            <Value>2</Value>
            <Interval>1000</Interval>
        </AnyName>
        <Growth>
            <Constraint>Alive</Constraint>
            <Value>2</Value>
            <Interval>1000</Interval>
        </Growth>
        <Decay>
            <Constraint>Dead</Constraint>
            <Value>-1</Value>
            <Interval>1000</Interval>
            <Delay>2000</Delay>
            <Limit>500</Limit>
        </Decay>
    </Change>
</a:example>
<element name="KillBeforeGather" a:help="Whether this entity must be killed (health reduced to 0) before its resources can be gathered">
    <data type="boolean"/>
</element>
<element name="Amount" a:help="Amount of resources available from this entity">
    <choice>
        <data type="nonNegativeInteger"/>
        <value>Infinity</value>
    </choice>
</element>
<optional>
    <element name="MaxAmount" a:help="The max amount of resource the entity can reach when growing">
        <ref name="nonNegativeDecimal"/>
    </element>
</optional>
<element name="Type" a:help="Type and Subtype of resource available from this entity">
		Resources.BuildChoicesSchema(true, true)
</element>
<element name="MaxGatherers" a:help="Amount of gatherers who can gather resources from this entity at the same time">
    <data type="nonNegativeInteger"/>
</element>
<optional>
    <element name="DiminishingReturns" a:help="The relative rate of any new gatherer compared to the previous one (geometric sequence). Leave the element out for no diminishing returns.">
        <ref name="positiveDecimal"/>
    </element>
</optional>
<optional>
    <element name="Change" a:help="Optional element containing all the modifications that affects a resource supply">
        <zeroOrMore>
            <element a:help="Optional element defining whether and how a resource supply regenerates or decays">
                <anyName/>
                <interleave>
                    <optional>
                        <element name="Constraint" a:help="Specifies the health constraint for the change to be active">
                            <choice>
                                <value>Alive</value>
                                <value>Dead</value>
                            </choice>
                        </element>
                    </optional>
                    <element name="Delay" a:help="Delay in milliseconds before the object starts growing or decaying">
                        <ref name="nonNegativeDecimal"/>
                    </element>
                    <element name="Value" a:help="The amount of resource added per interval">
                        <data type="decimal"/>
                    </element>
                    <element name="Interval" a:help="The interval in milliseconds">
                        <data type="positiveInteger"/>
                    </element>
                    <optional>
                        <element name="Limit" a:help="The upper or bottom limit of the value after which the change has no effect">
                            <data type="nonNegativeInteger"/>
                        </element>
                    </optional>
                </interleave>
            </element>
        </zeroOrMore>
    </element>
</optional>

 

Link to comment
Share on other sites
Guest

Guest

Posted
Posted (edited)

Component schemas being a property of that component has it’s advantages I suppose. Also, what about the js code which has to be run? Or the unmodifiable C++ component schemas?

Although, I guess one can find some reason to do that too.

Edited by Guest
Link to comment
Share on other sites
Stan` Consul

Stan`

Posted
  • Author
Posted
44 minutes ago, (-_-) said:

Component schemas being a property of that component has it’s advantages I suppose. 

That's not changing. Still a property. It's just initiated by being run.

44 minutes ago, (-_-) said:

Also, what about the js code which has to be run?

That's more troublesome but you can rely on stuff like the hack above or just append the code. It's a string after all

45 minutes ago, (-_-) said:

Or the unmodifiable C++ component schemas?

The engine schemas could be along the js ones. After all they are both components and this way easier to access for modders.

 

Link to comment
Share on other sites
Guest

Guest

Posted
Posted (edited)
5 minutes ago, stanislas69 said:

The engine schemas could be along the js ones. After all they are both components and this way easier to access for modders.

Engine schemas should not be accesible to mods. The component schema should be kept in sync with the component itself.

Ack the other points.

Edited by Guest
Link to comment
Share on other sites
Guest

Guest

Posted
Posted (edited)
8 hours ago, stanislas69 said:

Well CmpTimer is an engine component :)

I assume it was a System component. I am defining engine components as the C++ ones.

Edited by Guest
Link to comment
Share on other sites
  • 3 weeks later...
Guest

Guest

Posted
Posted
9 hours ago, Pudim said:

Better to use Json, it's faster and more compact

Thats irrelevant when these files are not parsed. The file is just a string. So, it doesn’t matter really.

Link to comment
Share on other sites
elexis Primus Pilus

elexis

Posted
Posted

The only arguable benefit of having the Schema in the JS Component implementation is to to have less user interaction when comparing the Schema with the implementation? We also have the JS and C++ interface files.

The benefits of moving the Schema to an XML file each:

  • Mixing multiple languages in a single file is an anti-pattern. In webdevelopment one can easily mix html, css, javascript , php and mysql in the same file. While there is less user interaction between searching lines in the same file, it may become even more fragmented;  and the encoding of one language in the other language adds some overhead. For the same reason it's also better to keep GUI page JS in JS files and use the XML files to specify only how the page will appear and be accessible to JS.
  • Syntax highlighting in XML / JS supporting editors will be correct
  • Syntax checks by the Phabricator bot or in the IDE of the developer will inspect correctly
  • The Schema file could be read by external scripts. Parsing XML is much easier than JS or even C++.
  • One doesn't always have to compile C++ code again in order to changing some minor property of the Schema of a C++ component. It reminds me of the time when special templates were hardcoded in C++. (Special templates are more variable than Schemas though.)
  • The componentmanager implementaiton might or might not become cleaner. Those global JS Engine.RegisterComponentType calls at the end of every JS Simulation component file break a bit the nice pattern of these JS files only specifying one prototype and doing nothing else.

Is the Schema actually used currently anywhere? I can't find any call to GetSchema() in *.js, *.cpp. Why the heck does the Pyrogenesis simulation not use the Schema currently to validate entity templates upon entity construction? Seems like a glaring use case. And SimulationDocs.h is disabled? And CCmpTemplateManager::Init also looks incomplete: 

	virtual void Init(const CParamNode& UNUSED(paramNode))
	{
		m_DisableValidation = false;

		m_Validator.LoadGrammar(GetSimContext().GetComponentManager().GenerateSchema());
		// TODO: handle errors loading the grammar here?
		// TODO: support hotloading changes to the grammar
	}

About the word "engine components" -> #5366 works with JS components being engine components as well and should thus be moved to the pyrogenesis engine mod instead of the 0ad gamecontent mod. The only difference between the two types is that one of them has to be fast C++ and the other one can be slow but comfortable JS. C++ even hardcodes references to the JS components in the C++ / JS interfaces and via the C++ components that refer to the JS components through these interfaces. At least I don't find any other distinction between these JS simulation components and the C++ simulation components that would qualify them as gamecontent:

Spoiler 

./AIInterface.js
./AIProxy.js
./AlertRaiser.js
./Armour.js
./AttackDetection.js
./Attack.js
./AuraManager.js
./Auras.js
./Barter.js
./BattleDetection.js
./Builder.js
./BuildingAI.js
./BuildRestrictions.js
./Capturable.js
./CeasefireManager.js
./Cost.js
./Damage.js
./DeathDamage.js
./EndGameManager.js
./EntityLimits.js
./Fogging.js
./FormationAttack.js
./Formation.js
./Foundation.js
./Garrisonable.js
./GarrisonHolder.js
./Gate.js
./Guard.js
./GuiInterface.js
./Heal.js
./Health.js
./Identity.js
./Looter.js
./Loot.js
./Market.js
./Mirage.js
./MotionBall.js
./Pack.js
./Player.js
./PlayerManager.js
./ProductionQueue.js
./Promotion.js
./RallyPoint.js
./RangeOverlayManager.js
./Repairable.js
./ResourceDropsite.js
./ResourceGatherer.js
./ResourceSupply.js
./ResourceTrickle.js
./Settlement.js
./SkirmishReplacer.js
./Sound.js
./StatisticsTracker.js
./StatusBars.js
./TechnologyManager.js
./TerritoryDecay.js
./TerritoryDecayManager.js
./Timer.js
./Trader.js
./TrainingRestrictions.js
./Trigger.js
./TriggerPoint.js
./UnitAI.js
./UnitMotionFlying.js
./Upgrade.js
./ValueModificationManager.js
./Visibility.js
./VisionSharing.js
./WallPiece.js
./WallSet.js
./Wonder.js
Quote

./CCmpAIManager.cpp
./CCmpCinemaManager.cpp
./CCmpCommandQueue.cpp
./CCmpDecay.cpp
./CCmpFootprint.cpp
./CCmpMinimap.cpp
./CCmpMotionBall.cpp
./CCmpObstruction.cpp
./CCmpObstructionManager.cpp
./CCmpOverlayRenderer.cpp
./CCmpOwnership.cpp
./CCmpParticleManager.cpp
./CCmpPathfinder_Common.h
./CCmpPathfinder.cpp
./CCmpPathfinder_Vertex.cpp
./CCmpPosition.cpp
./CCmpProjectileManager.cpp
./CCmpRallyPointRenderer.cpp
./CCmpRangeManager.cpp
./CCmpRangeOverlayRenderer.cpp
./CCmpSelectable.cpp
./CCmpSoundManager.cpp
./CCmpTemplateManager.cpp
./CCmpTerrain.cpp
./CCmpTerritoryInfluence.cpp
./CCmpTerritoryManager.cpp
./CCmpTest.cpp
./CCmpUnitMotion.cpp
./CCmpUnitRenderer.cpp
./CCmpVision.cpp
./CCmpVisualActor.cpp
./CCmpWaterManager.cpp

 

Notice there is a gotcha about using XML files for the Schema, since some JS files (resources and damage types) create the Schema programmatically using JS. With XML one would have to have one XML file per resource per affected simulation component if one wants to support enabling multiple mods that add new resources each.

The large XML file you posted above appears nicer than encoding that in JS. Another random example: CCmpPosition.cpp probably wouldn't look worse when that XML is in a separate file. So seems possibly nice, perhaps one can find some more specific applications to see if it's worth the refactoring and the additional userinteraction.

(JSON might be more compact, but simulation components used XML to gain validation. Then I wonder why validation isn't reasonable for aura and tech templates if its reasonable for entity templates. Weren't auras introduced as JSON, then converted to XML by merging them into the entity templates, but that failed because different entity templates should refer to the same aura types? But perhaps that could have also been solved by having auras as separate validatable XML files? Actually, meh.)

Link to comment
Share on other sites
Stan` Consul

Stan`

Posted
  • Author
Posted
1 hour ago, elexis said:

The only arguable benefit of having the Schema in the JS Component implementation is to to have less user interaction when comparing the Schema with the implementation?

I'm not sure that's even an argument. I'd rather be comparing two XML files than a JavaScript file and an XML file. Luckily enough our schemas are defined at the top of the file.

1 hour ago, elexis said:

[...] Arguments in favor.

Moving seems like a pretty good idea then.

1 hour ago, elexis said:

Is the Schema actually used currently anywhere? I can't find any call to GetSchema() in *.js, *.cpp. Why the heck does the Pyrogenesis simulation not use the Schema currently to validate entity templates upon entity construction? Seems like a glaring use case. And SimulationDocs.h is disabled? And CCmpTemplateManager::Init also looks incomplete:

I'm actually not sure. However, if one defines a broken Schema in JavaScript files, (I guess CPP too but I haven't tried) you'll get an error when you start Atlas.

1 hour ago, elexis said:

Notice there is a gotcha about using XML files for the Schema, since some JS files (resources and damage types) create the Schema programmatically using JS. With XML one would have to have one XML file per resource per affected simulation component if one wants to support enabling multiple mods that add new resources each.

I actually found a work around for that. Maybe it's dirty, but with some refinement it could work properly. 

ResourceSupply.prototype.Schema = Engine.ReadFile("simulation/components/schemas/ResourceSupply.xml").replace("Resources.BuildChoicesSchema(true, true)", Resources.BuildChoicesSchema(true, true));

Changing parameters here sounds a bit tricky,  but I guess one could come up with a way to define a special string which would cover different options.

JavascriptObject;JavascriptFunction;Param1;Param2; etc and make a parsing function for it in utils. https://stackoverflow.com/questions/4116608/pass-unknown-number-of-arguments-into-javascript-function

 

 

Link to comment
Share on other sites
elexis Primus Pilus

elexis

Posted
Posted
45 minutes ago, stanislas69 said:
2 hours ago, elexis said:

[...] Arguments in favor. 

Moving seems like a pretty good idea then. 

I could imagine some people critizing that the implementation is moving things around for no reason. So the question is whether the provided arguments are too hypothetical or defeating arguments. One can often increase the sample size in such situations, then the consequences accumulate, making the advantages or disadvantages more grave, giving one a more clear picture which alternative is preferable. 

ResourceSupply.prototype.Schema = Engine.ReadFile("simulation/components/schemas/ResourceSupply.xml").replace("Resources.BuildChoicesSchema(true, true)", Resources.BuildChoicesSchema(true, true));

Those should be *.xsd files, because they contains the Schema to validate XML files, if Im not mistaken.

The most simple solution would probably be to remove the Schema code entirely from the component code (JS or C++) and have the component manager instead load the Schema from <componentname>.xsd.

I don't know if that approach could work with the resource type definitions, or whether one can just skip validating these, or whether one could do something fancy with including another xsd file.

Link to comment
Share on other sites
Stan` Consul

Stan`

Posted
  • Author
Posted
2 hours ago, elexis said:

I could imagine some people critizing that the implementation is moving things around for no reason. So the question is whether the provided arguments are too hypothetical or defeating arguments. One can often increase the sample size in such situations, then the consequences accumulate, making the advantages or disadvantages more grave, giving one a more clear picture which alternative is preferable.

Sure. The question being would it save time in the future to modders and game programmers that they are separated. I think it would be pretty straightforward to look into the schemas for XML schemas, but some people might disagree. But the thing is the only thing they could oppose is that they like to have mixed code types, which as you stated is an antipattern.

2 hours ago, elexis said:

Those should be *.xsd files, because they contains the Schema to validate XML files, if Im not mistaken.

In theory yes, however XSD would come with other requirements  https://www.w3schools.com/xml/schema_example.asp 

Also, that would mean we'd have to add it to every template block like Identity which seems a bit unproductive.

Link to comment
Share on other sites
Guest

Guest

Posted
Posted (edited)
7 hours ago, stanislas69 said:

JavascriptObject;JavascriptFunction;Param1;Param2; etc and make a parsing function for it in utils. https://stackoverflow.com/questions/4116608/pass-unknown-number-of-arguments-into-javascript-function

Mozilla recommends using rest parameters. (If anyone cares about those kind of things)

Edited by Guest
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
 Share
Go to topic listing
×
×
  • Create New...