I made some multiplayer rankings. This is a help when setting up games, to decide if teams are fair.
First, these rankings depend on replays (specifically, metadata.json files). If you'd like to improve or expand these ratings, zip up all your a21 replays and send them to me! Check https://trac.wildfiregames.com/wiki/GameDataPaths to find where they are on your computer.
I have revised these rankings several times as I get more data and improved methods. See the spoiler below for the rest of the original post, with the original rankings.
And here's the updated list with all the changes and extra data so far. This updated list was generated by the following process:
Start with replays: mine, Feldfeld's, mapkoc's, Hannibal_Barca's, and temple's.
Filter out single player games, games with AI players, pizza games, survival of the fittest, games with a matchID that was already processed, games under 200 seconds long
Map each player name to a canonical player name, according to a list of known aliases. For example, "Please" is mapped to "JC".
Score players in each multiplayer game according to 5*(enemy units killed value) + (resources harvested) + 2 * (trade income)
Each team on a multiplayer game is converted into a set of 1v1 matches between players on the same team, the winner of the match being the player with the higher custom score. However, if two players on the same team have custom scores within 20% of each other, each player is recorded as beating the other.
1v1 games are also included.
Feed these matches into a WHR (Whole History Rating) algorithm, with rating variance set to 10 per day. This gives every player a rating.
Determine player "Strengths" such that the sum of strengths on a team predicts which team wins. It is assumed/required that higher rated players have higher strengths. A set of strengths that minimizes prediction errors is found through random search, and then normalized to a 0-5 scale. Note that these strengths are not very strong predictors; just rough estimates.
Strength categories are displayed as headings, e.g. those under the ===== 4 - 5 ===== heading have a strength between 4 and 5.
Players are listed in order of decreasing rating.
Players are indented a number of spaces equal to 10 - the number of multiplayer games they played. If a player is indented, distrust their ranking.
Players with 3 or fewer games are followed by a list of players they have scored higher than, so you can see whether their rank is reasonable (often not).
A bar is printed at the left margin to prevent line wraps from confusing the rankings.
Players who have never outscored anyone are excluded from the list, to save space.
Known aliases are printed at the end.