Looking back on the balancing strategy

[hyperion]

5 hours ago, Stan` said:

Do you mean something like this http://docs.wildfiregames.com/templatesanalyzer/ ?

No, statistics on which civ players choose, whether win loss/ratio fits statistical expectations, length of games and other collectable data, then trying to make sense of all the data. Complement it with questionnaires as to what people like about civs and what not. The goal should be to have all civs be somewhat attractive for whatever reasons, if it's just the sound track that makes a civ popular that is fine as well If those "we need ultimate balance for competitive games" people have a couple interesting civs which work for them to choose from that is good enough, they just have to have the possibility of a filter which is currently lacking.

5 hours ago, Stan` said:

I'd like that. The design document is actually versioned and changes can be made to it. https://docs.wildfiregames.com/design/ But I need people to step up and take the lead.

Contribute redirects to https://code.wildfiregames.com/source/design/ which returns an error

---

@maroder

A design document should be about principles, not stats or similar details.

What it could describe without thinking much about it:

• How to update the design document (process)
• If I want to add a civ, what do I have to fulfill
• If I want to add a map, what do I have to make sure
• If I want to add a model, what requirements are there
• If I want to write a new ui, what must I make sure of
• How to bring historical facts to the users attention
• How should city building aspect work
• How should fights work on a meta level
• What is territory meant to achieve
• many more

For instance the removal of kennels without the backing of a design document shouldn't have happened IMHO.

[LetswaveaBook]

To be honest, I think the call to for a design document isn´t rooted in practical purposes. I would consider the hope that a design document provides misleading. You might think that things get done once it is clearly put in a design document, but issues don´t magically solve itself when there is a design document.

I expect any design document to quickly get stale. So rather than saying this is the design document, It would be probably better to set goals for the next 3 alphas and count how much has been achieved to reach these goals.

There was some talk about faction differentiation, but what have the discussion since A24 delivered? Hoplite tradition and moving possibly kush pyramids to p1. You can´t say we should achieve this goal and just wait until it is achieved.

[AIEND]

我觉得公开一些的讨论对平衡总是有更多好处，人们关注一些议题是因为希望参与进去，如果没有发言权，玩家就会认为这是与自己无关的事物而忽视掉。

[Lion.Kanzen]

2 minutes ago, AIEND said:

我觉得公开一些的讨论对平衡总是有更多好处，人们关注一些议题是因为希望参与进去，如果没有发言权，玩家就会认为这是与自己无关的事物而忽视掉。

-Translation-

"I think a more open discussion is always better for balance, people pay attention to some topics because they want to participate in them, if they don't have a say, players will think it's something that doesn't concern them and ignore it".

-DeepL translation by the way.

Edited April 20, 2022 by Lion.Kanzen

[azayrahmad]

I'm no longer really active in this forum, but I do check it out occasionally from time to time. It's sad to see that we are still in a deadlock of balancing debate.

11 hours ago, hyperion said:

A design document should be about principles, not stats or similar details.

I agree with this statement. To elaborate my argument:

In Civilizations section of the design document, currently we have historical overview and then it jumps straight to detailed description of units/buildings, some even do away with overview altogether.

What I suggest is the overview of what the player expect when they select certain civilization. What differentiate it from other civilizations. Add some historical based justification as necessary. Something like Rise of Nations, but less technical and more abstraction:

e.g.

• Athenian: They gather silver faster (because Laureion mines), they research faster (because philosophy).
• Romans: They have strong infantry (because Legions), they expand faster (because Roman empire).
• Mauryan: They have mobile gathering (because ?), strong archer (because longbows).
• etc.

Avoid exact numbers and percentage, focus on general advantages/disadvantages or strengths/weaknesses of each civs. Only after these established, we can then go to units/structure/bonus description. For each description, there must be a reference to this overview.

The finalized design document should be able to answer questions like:

• why certain unit have certain stats/why does this civ have certain bonuses
• on the other hand, why this civ doesn't have that unit or structure
• what is the difference of gameplay between civ A & civ B
• I'd like to rush/turtle, which civs are suitable for me
• why does this unit too weak/strong
• etc.

After that, the balancing discussion can continue. Refer to design document established above before making any changes. When proposing any changes, ensure that it doesn't break any of the established design first before we talk about the relation with other civs.

I understand that we are not making this design document from scratch, as we already have the game, so cheating i.e make the design based on the finished game is alright. What I want to stress is that we need to make sure that people know the general intended design of each civ. When people suggest changes, there must be some degree of bias (favorite civs, preferred playstyle, favorite RTS games beside 0AD, etc) and I hope the established design document could be considered before proposing something.

[alre]

the purpose of a design document is to fix things and avoid always rediscussing them. the document is not enough in itself, a real leadership is necessary to make people respect the document. we actually already have one, the problem is noone cares (the base principles of 0AD design could still be taken as valid, and they say some very clear things about micromanagement, but I remember seing micro role in the game being questioned many many times, and noone ever pointing out the design principles).

[Stan`]

17 hours ago, hyperion said:

Contribute redirects to https://code.wildfiregames.com/source/design/ which returns an error

Thanks fixed. Git update broke it it seems.

17 hours ago, hyperion said:

No, statistics on which civ players choose, whether win loss/ratio fits statistical expectations, length of games and other collectable data, then trying to make sense of all the data. Complement it with questionnaires as to what people like about civs and what not. The goal should be to have all civs be somewhat attractive for whatever reasons, if it's just the sound track that makes a civ popular that is fine as well

That sounds like a GDPR nightmare.

 

[azayrahmad]

4 hours ago, alre said:

the purpose of a design document is to fix things and avoid always rediscussing them. the document is not enough in itself, a real leadership is necessary to make people respect the document. we actually already have one, the problem is noone cares (the base principles of 0AD design could still be taken as valid, and they say some very clear things about micromanagement, but I remember seing micro role in the game being questioned many many times, and noone ever pointing out the design principles).

I think there are some sort of authority are needed to enforce the design. There are code reviewers that could ensure all approved changes are according to the guideline provided by the design.

Unfortunately, as is the point of my comment, the design document itself is not yet clearly defined in term of balancing. Yes there are some general advice on the document that says we have to reduce micromanagement, but I think this is too subjective to be able to define a line.

By defining it clearly (what is micro, what are the behavior constitutes as micro and which could be tolerated, what should be eliminated to enforce this) it should help people to understand it more clearly. We should be able to quote a specific part of design document to stop any prolonged debates.

The last time I was active here is around perhaps A24, where people complain about nerfed Roman. But there is no statement in design document that says Roman should or should not be weaker or stronger than other civs.

[Gurken Khan]

56 minutes ago, azayrahmad said:

there are some general advice on the document that says we have to reduce micromanagement

Interesting. One time I pointed out that something could be done about the worker elephant's helpfulness, the reply I got was 'it was decided to have it skill-based', meaning microed.

[Nullus]

I've posted my proposal for a general design document at https://wildfiregames.com/forum/topic/78095-design-document-proposal/

[Grapjas]

I think being open sourced is both a blessing and a curse. In the end you need someONE (or at least a very small group like 3 ppl, this way you always have either min 2 opposed or in favor) taking the lead and put down the hammer on what gets into the game. The more people that will join the discussions, the harder it will be to agree to something and it will develop into endless debates (just look at politcs, lol). But it's a difficult task indeed for one person to carry that weight. And you will lose and gain people along the way regardless.

I agree with @azayrahmad that a proper design document needs to be in place and enforced, for people that are willing to contribute. But when it comes new ideas, it's best to have one person (or the small team) in the lead making those final calls imo.

[ChronA]

13 hours ago, Grapjas said:

I think being open sourced is both a blessing and a curse. In the end you need someONE (or at least a very small group like 3 ppl, this way you always have either min 2 opposed or in favor) taking the lead and put down the hammer on what gets into the game. The more people that will join the discussions, the harder it will be to agree to something and it will develop into endless debates (just look at politcs, lol). But it's a difficult task indeed for one person to carry that weight. And you will lose and gain people along the way regardless.

This.
Consensus governance inevitably tends toward conservatism and policy gridlock (sometimes punctuated by episodes of violent identitarianism). If you can't get everyone to agree to do something, then the one thing you can agree to do is nothing. The Romans understood this, which is why they permitted the office of Dictator to be instituted during moments of crisis. Maybe this project needs a Balance Dictator?

If you are looking for a policy proposal, here's what I would do if I were Princeps: First, a counsel of the most active developers should be convened to discuss candidates and appoint the balance dictator. Whoever they pick will then get 2 years to enact any balance overhaul plan they think is necessary, without any obligation to consult with the council of active developers or anyone else. To do this the dictator would for their 2 year office exercise a non-negotiable discretion to summarily approve or block any changes to the 0 AD development codebase and design documents, without any binding responsibility to defend or explain their reasons.

At the end of their 2 year office, the active developers or the community at large would vote whether to revert the dictator's balance contributions, and/or whether to elect a new dictator or extend the current dictator's office for another term. If no one can agree to enact such a plan (or any reasonable alternative), or if the dictator or other parties violate the terms of the concord, that'd be viewed that as strong evidence that the current design is effectively locked in. In that case the active developers should adopt a binding resolution to spend the next two years excising any problem/unfinished features from the work and release it at the end of that period... as 0 AD Beta 1!

But that's just my 2 cents. There are obvious risks to investing too much authority in one person. Even if the person with absolute power is a perfect saint, you risk alienating anyone anyone with a good faith difference of opinion about the direction of the project. But at the end of the day you are not trying to run a country here, you are trying to make a world-class video game. People don't need to be happy with the development process for the project to be successful, just the end product.

P.S. "I love democracy..." /Palpatine.gif

[wowgetoffyourcellphone]

Not a bad plan. 2 years seems excessive. Maybe 2 alphas. 1 alpha to enact some changes, and 1 more to fix/tweak these changes.

[Lion.Kanzen]

35 minutes ago, ChronA said:

But that's just my 2 cents. There are obvious risks to investing too much authority in one person. Even if the person with absolute power is a perfect saint

You should always expect the worst of the human being.

[Grapjas]

7 hours ago, Lion.Kanzen said:

You should always expect the worst of the human being.

I disagree, but you should be wary, sure. If you always assume the worst thats prejudice, and with prejudice you will bring out the worst surely. I wholeheartedly hate generalisation.

[Lion.Kanzen]

4 hours ago, Grapjas said:

I disagree, but you should be wary, sure. If you always assume the worst thats prejudice, and with prejudice you will bring out the worst surely. I wholeheartedly hate generalisation.

Isn't hate. if you lived in a country like mine, human beings are corrupt.

That is why dictatorships happen. Power( political) corrupts much more.

[Gurken Khan]

15 minutes ago, Lion.Kanzen said:

Isn't hate. if you lived in a country like mine, human beings are corrupt.

Corruption is a problem all over the world. (New Zealand again places pretty well.) But I don't think you should always expect the worst of people working on a FOSS project, especially if they already have for years.

 

13 hours ago, ChronA said:

without any binding responsibility to defend or explain their reasons.

Whatever authority might be installed, they definitely should consult and explain.

[Lion.Kanzen]

39 minutes ago, Gurken Khan said:

Corruption is a problem all over the world. (New Zealand again places pretty well.) But I don't think you should always expect the worst of people working on a FOSS project, especially if they already have for years.

I am speaking in general.

Believe it or not here among the team, people who are no longer around, there has been toxicity.

[Grapjas]

1 hour ago, Lion.Kanzen said:

if you lived in a country like mine, human beings are corrupt.

There are corrupt human beings, but not all human beings are corrupt, is my point. But we're getting off topic i guess ( my fault ) and i honestly have nothing to add.

Edited April 25, 2022 by Grapjas

[ChronA]

1 hour ago, Gurken Khan said:

Whatever authority might be installed, they definitely should consult and explain.

The whole point of that proposal was to not have to consult and explain. Consulting and explaining take up valuable energy and time that should be focused on development. Consultation also gives the impression that the consulter has some degree of moral obligation to listen to the consulted.

We are all just arguing about the best courses of action on the basis of our own prejudices without any definitive evidence to back us up. Why not give someone a chance to shoot their shot at revising the game and then just judge the result on its actual merits?

To do that though, I think you need to give some strong (verging on excessive) guarantees that the contributor will be able to work without interference and their work will be judged fairly. Otherwise no-one in their right mind will be willing to take on the burden.

That's also why I suggest a 2 year period. Yes it is a lot longer than such an endeavor should ever take to develop, but you also need to provide time for players get used to a new patch and really feel out its subtleties before asking them to pass judgement. Maybe you could go as low as 18 months: a generous 8 months to develop the main overhaul patch, a tight 1.5 months for the community to playtest and give feedback on the new alpha, then another tight 2.5 months turnaround to fine tune the balance for another alpha release, then finally 6 months to play the result before the community makes their decision whether to persevere onward with on the new vision or to revert back to before it started.

[smiley]

11 minutes ago, ChronA said:

That's also why I suggest a 2 year period.

Two years is too long without minor releases. Minor releases aren't really a thing since even small bug fixes can make versions incompatible. Minor releases should be compatible, else its a major release. There isn't a way to ship those changes without an actual release.

[Gurken Khan]

1 hour ago, ChronA said:

We are all just arguing about the best courses of action on the basis of our own prejudices without any definitive evidence to back us up.

AFAIK all researchers working on decision making include for example the point 'evaluating alternatives'. Naturally more perspectives and ideas are brought up when several individuals are involved. ( https://en.wikipedia.org/wiki/Decision-making#Steps ) I find this idea of one single person making all decisions without any communication not only objectionable, but also totally unrealistic for a project like this; would they take on the complete workload and do everything themselves?

Furthermore I doubt that any term based on a fixed number of months will do any good; it should be based on releases or milestones imho. Devs could become unavailable for any reasons, new devs could appear, each having an effect on the progress; likewise unexpected problems, as we are seeing right now.

[Mastoras]

In terms of balancing @hyperion is on the right track, we need good quality data to decide what action needs to be taken.

I would be happy to assist, to the level that my skills allow it.

We need 1 of two things (or both), at least to start measuring the top level metrics:

1. Simulate some games without the graphics part, I think there is something like that already implemented?
   1. Run say 10000 games, 1v1 all civs against all civs with same AI.
   2. I understand this can be resources intensive, but considering the fact that the work can be distributed across different members, it should be doable.
2. Even better, since we already we have a lot of games in multiplayer, once a game is finished, to report not only winner, but also civs. Or the whole game log file even better. The server can collect the data and these data can be used for balancing. We can later go deeper in unit stats, and specific features. These statistics in later phase can be incorporated in the lobby as well for everyone to access easily.

In the end balance is about the advantage or disadvantage that a civ provides to the player that is translated to victory or loss. So this is what needs to be tracked. We don't need to know which specific advantage is offering the edge, all we need to know is which civs have an "edge" and then apply any type of nerf.

We could use both:

In an ideal situation, we do balancing based on point 2 (real player matches). Before every new release we "run" point 1 (simulations between the ai), to make sure any new features and changes or even balancing actions have not "broken" the balance.

Extra Thoughts:

I think first we can focus in implementing the above for 1v1 games and then expand to more complicated scenarios like ffa and tg that team bonuses play a role.

Land and Water: The drawback here is that the current state of rated 1v1 is mostly land, so it will not "balance" properly island maps. Maybe we can create 2 categories of civ stas. Land games, and water games, cause there are big differences between civs for the 2 of those. In general Civs must be balanced in both land alone and water alone, we shouldn't have civs that are weak in land and strong in water or vice versa. It sounds nice, but in terms of balancing I think all civs should be equally capable in all scenarios, maybe it does not make a lot of historical/accurate sense, but it makes for proper-fun gameplay.

For example Athenian champion training from ships is quite OP, especially with a25 where the ships are shredding organic units, but in general they are considered weak civ for 1v1, because they are almost always land maps. This whole water power, land power is a big discussion, all I want to say is we need to keep it in mind if we implement a system like the one above.

But we know which civs need nerfing/buffing.

After-all we have people here that play 100s of games per release. Well, 25 releases show that we struggle to find a good balance and we always end up with strong and weak civs. But even so, I can accept the fact that some members have the experience to say which civs are stronger,weaker. Even in this case at least point 1 should be done and "run" before release and compared with previous release, so we know that the implemented balance "fix" will improve the game and not transfer imbalance from one civ to another.

Next action:

Create a component that takes in the saved file from a game and generates statistics for civs, segmented per relaease. We can even create a thread for people to upload their saved games so we can feed them to the calculator manually.

Edited May 16, 2022 by Mastoras

[ChronA]

AI vs AI testing is not very useful for RTS balancing purposes because it is very difficult to make an AI that plays optimally AND in a human like way. A huge part of the effectiveness of a unit is determined by 1. how it can be micromanaged to avoid threats, 2. how fast it can move between strategically important locations compared to opponent compositions, (i.e. if your force can threaten multiple locations at once, that is a force multiplier,) and 3. how effective it is across the totality of optimal compositions (i.e. a unit you don't usually need to replace when your opponent changes strategies is more valuable than one that frequently becomes obsolete). AI is good at testing precisely none of these factors.

[Freagarach]

5 hours ago, Mastoras said:

Run say 10000 games, 1v1 all civs against all civs with same AI.

More like a trillion games with machine learning, as @ChronA kind of points out.