Mentula Posted Friday at 12:49 Report Share Posted Friday at 12:49 (edited) Hello everyone! I hereby present a 0 A.D. mod aimed at evaluating the rating of players. Official mod page on GitLab here. Introduction Before diving into the description, let me introduce the problem this mod aims to solve. In 0 A.D., the ELO system is used to rank players in the lobby. This is good; but is it representative of the players' skills? As you know, the rating system in 0 A.D. only takes into account 1v1 rated games. Team games do not contribute to the ELO score of a player, as well as 1v1 unrated games. Also, the scoring system only takes into account the outcome of a game (victory/defeat) and not the "performance" during the game. Can we do better? This mod uses statistics. It extracts data from all the replays of games you (the mod user) have played. So, if you have played 20 games (1v1s, team games, other..) with a player in the lobby whose name is (for example) strangeJokes, the mod will assign a rating to strangeJokes based on the 20 games you've played with them. The rating system The functioning of the rating system is described in detail at the official mod page on GitLab, but in short what it does is: it considers the average performance of the player during the entire game (and not only at game's end). the rating assigned to a player is a percentage: for example, a player with a rating of 5.00 performs a 5% better than other players on average, while a player with a rating of -5.00 performs a 5% worse than other players on average. you can customize the rating system by giving more importance to military, economy, exploration or other factors to the aim of calculating ratings. Keep in mind that this mod is based on statistics; data are taken from your (the mod user) replays. Statistics might not be fully representative of reality; therefore, a player's rating could be inaccurate, especially if you have played few games with that player. The more you play with a player, the more accurate the rating of that player is. Mod installation Spoiler Download the attached zip file and extract it. Copy the LocalRatings folder into the mods folder, tipically located at: Linux: ~/.local/share/0ad/mods/ macOS: ~/Library/Application\ Support/0ad/mods/ Windows: ~\Documents\My Games\0ad\mods\ Launch 0 A.D. and open the Settings > Mod Selection menu. Select the LocalRatings mod, Enable it and Save Configuration. Explanatory pictures The main page: it lists all players with their rating and the number of games you have played with them. Spoiler The Options > Score Weights page: where you can customize the rating system based on the factors you consider more important. Spoiler The Options > Match Filters page: where you can select the games relevant for the rating calculation (for example, you might want to ignore games with duration less that 5 minutes). Spoiler Contribute The public repository is at this page. Everybody is very welcome to contribute, suggest, fork or simply give feedback. Have fun! LocalRatings-v0.25.3.zip Edited Sunday at 01:57 by Mentula fixed bug from previous version attachment 9 2 Quote Link to comment Share on other sites More sharing options...
seeh Posted Saturday at 12:49 Report Share Posted Saturday at 12:49 (edited) thank you soo much. BTW my LocalRating is -8 at the moment its ok for me https://youtu.be/r4kzFStUQQk but if i took more replays (from a backup HDD) its about -10 nice so i see progress ( then i at rank 364 of 629 players) Edited Saturday at 13:07 by seeh 1 Quote Link to comment Share on other sites More sharing options...
thephilosopher Posted Saturday at 13:10 Report Share Posted Saturday at 13:10 (edited) This is pretty cool. FYI: After installing and starting it up, I had to click "Rebuild List" before I was able to get actual ratings. Once I clicked that, it worked fine. If we're posting our own ratings for comparison, my rating here with this mod was 3.43. My lobby rating is 1286. Edited Saturday at 13:12 by thephilosopher 1 Quote Link to comment Share on other sites More sharing options...
seeh Posted Saturday at 13:21 Report Share Posted Saturday at 13:21 It would be great if you could limit the time. E.g. the last x months. Or the last 50 games. Then the games where you played really badly would no longer be included Quote Link to comment Share on other sites More sharing options...
seeh Posted Saturday at 13:36 Report Share Posted Saturday at 13:36 (edited) how about if we sync all our replays by using https://syncthing.net/ ? Edited Saturday at 13:36 by seeh Quote Link to comment Share on other sites More sharing options...
seeh Posted Saturday at 13:45 Report Share Posted Saturday at 13:45 (edited) seems some games are not into that list.example ValihrAnt is not in the list (but i find many replays with ValihrAnt). did i makes a different if a replay is recorded with boonGui? sorry i found him: Edited Saturday at 14:06 by seeh Quote Link to comment Share on other sites More sharing options...
Mentula Posted Saturday at 13:46 Author Report Share Posted Saturday at 13:46 17 minutes ago, seeh said: It would be great if you could limit the time. E.g. the last x months. Or the last 50 games. Then the games where you played really badly would no longer be included The main problem with restricting the sample of replays to a limited amount is statistics. The more replays you have, the more accurate the rating of a player is, therefore it is disadvisable to rely on data extracted from a small (though recent) sample. Quote Link to comment Share on other sites More sharing options...
seeh Posted Saturday at 13:50 Report Share Posted Saturday at 13:50 (edited) 5 minutes ago, Mentula said: disadvisable to rely on data extracted from a small (though recent) sample sure but if you have played many years then you have data. then i not very interested to calculate the first years. as a workaround we could remove games out of the replay folder. or a other idea: maybe you could waigt in calculation games stronger that not very old. so it not needet to be configed Edited Saturday at 13:52 by seeh Quote Link to comment Share on other sites More sharing options...
Sevda Posted Saturday at 14:19 Report Share Posted Saturday at 14:19 @Mentula I would recommend adding an uncertainty label after the main rating, calculated based on how many games they have played with you. This will prevent players with just 1 fluked match getting ridiculously high ratings. For example, leGrosRobert is at the first place although he is arguably not the best player in your list. Furthermore, 2 of Yekaterina's smurf accounts are way better than you and weirdJokes, although I doubt whether she is actually that talented. Quote Link to comment Share on other sites More sharing options...
sarcoma Posted Saturday at 20:12 Report Share Posted Saturday at 20:12 valihrant #180 <- noob 1 Quote Link to comment Share on other sites More sharing options...
Player of 0AD Posted Saturday at 21:55 Report Share Posted Saturday at 21:55 If someone will play only 1v1 vs much weaker players, the player can get easily a very high "rating". So you can think that its rather like a win rate than like a rating. Just saying 1 Quote Link to comment Share on other sites More sharing options...
Sevda Posted Saturday at 23:21 Report Share Posted Saturday at 23:21 Furthermore, you can farm ratings easily using virtual machines (although I haven't been able to work out the maths behind the rating system), so I am inclined to trust Mentula's ratings mod more than Vanilla Lobby's ratings evaluation. Quote Link to comment Share on other sites More sharing options...
Sevda Posted Saturday at 23:51 Report Share Posted Saturday at 23:51 @Mentula using your default settings, this is the leaderboard for me: And I find myself with no rating Why is it that so many players have a score of 0? 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted Sunday at 00:05 Author Report Share Posted Sunday at 00:05 6 minutes ago, Sevda said: @Mentula using your default settings, this is the leaderboard for me: And I find myself with no rating Why is it that so many players have a score of 0? Hum that is a bit strange, it could be a bug. It's actually the first time I see it and at the moment it's hard to find the cause. @Sevda I'll send you a private message to investigate (maybe tomorrow, now it's late in my timezone). Thanks for reporting! Quote Link to comment Share on other sites More sharing options...
Mentula Posted Sunday at 00:39 Author Report Share Posted Sunday at 00:39 2 hours ago, Player of 0AD said: If someone will play only 1v1 vs much weaker players, the player can get easily a very high "rating". So you can think that its rather like a win rate than like a rating. Just saying I personally disagree with this interpretation of the rating. First of all, it's true that someone who mostly plays against weaker players gets a higher rating, no doubt. However, I disagree on the interpretation of the rating as a win rate (in any context: 1v1s, TGs, ...). The rating assigned by the LocalRatings mod (the default one, as well as any user-customized rating using different weights) is very distant from representing a win rate. A win rate of 20% means that the player wins 1 out of 5 games, which is not good; on the other hand, a rating (as in the LocalRatings mod) of 20% means that the player's graph in the Summary chart is 20% better than the average graph, which in other words means that the player performs very well (and therefore presumably has a high win rate). In my experience with this mod, a player with a rating of 20% is a strong player; I wouldn't say the same of a player with a 20% win rate. However, let me clarify one thing again: this mod is based on statistics. In statistics any of us can imagine a "limit case" (it could be, for example, a player who only plays with much weaker players). On the bright side... this mod is based on statistics! This means that, generally, a player who plays with different types of players will experience more reliability in the ratings data. 56 minutes ago, Sevda said: Furthermore, you can farm ratings easily using virtual machines (although I haven't been able to work out the maths behind the rating system), so I am inclined to trust Mentula's ratings mod more than Vanilla Lobby's ratings evaluation. This comment actually gives me the possibility of clarifying one thing. Assigning a rating to a player can be something very arbitrary and will never make all of us agree. So, I tend to see the rating of a player as the player's performance over the average, or, if you prefer, their contribution to the game. But this is just my interpretation of it. The lobby's rating evaluation is a system only takes into account the win rate, whereas the LocalRatings mod takes into account scores. In this sense, they don't conflict, they just represent two different things. 2 Quote Link to comment Share on other sites More sharing options...
thephilosopher Posted Sunday at 01:24 Report Share Posted Sunday at 01:24 1 hour ago, Mentula said: Hum that is a bit strange, it could be a bug. It's actually the first time I see it and at the moment it's hard to find the cause. @Sevda I'll send you a private message to investigate (maybe tomorrow, now it's late in my timezone). Thanks for reporting! Sevda's issue is the same issue I had at first. I solved it by clicking "Rebuild List," and then all the scores populated. 1 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted Sunday at 02:06 Author Report Share Posted Sunday at 02:06 @Sevda @thephilosopher many thanks for reporting. Bug is now fixed; the updated version v-0.25.3 of the mod is now attached to the initial post of this thread. If you have the older version v-0.25.2 installed no problem: just open the mod options menu once and everything will work. 2 Quote Link to comment Share on other sites More sharing options...
Player of 0AD Posted Sunday at 10:43 Report Share Posted Sunday at 10:43 My ratings, based on resources (creating a resource gives a point, sending resources to allies too, destroying gives a point, capturing resources which are buildings gives almost 2 points (not exactly 2 because you often cant keep the building), basic loot is considered (its hard to consider the loot which results in units carrying resources)) 1 Quote Link to comment Share on other sites More sharing options...
seeh Posted Sunday at 12:29 Report Share Posted Sunday at 12:29 (edited) 1 hour ago, Player of 0AD said: based on resources BTW using my (newer from this year) localReplays your ranked #1 and not #17 BTW your played vs me also Edited Sunday at 12:41 by seeh 1 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.