sarcoma Posted June 1, 2022 Report Share Posted June 1, 2022 Can we start sharing weights? Spoiler Whats the difference between some categories (value/umber)? I had to filter players with 5+ games to avoid some randos but this is still not right. Spoiler 2 Quote Link to comment Share on other sites More sharing options...
Mentula Posted June 1, 2022 Author Report Share Posted June 1, 2022 7 hours ago, sarcoma said: Whats the difference between some categories (value/umber)? @sarcoma let us consider a specific example: "Enemy units killed (value)" vs "Enemy units killed (number)". If you set a weight of 1 to "Enemy units killed (value)", the contribution of this parameter to the rating corresponds to the total value of enemy unit killed, that is, the cost to produce them. For example, with a weight of 1, a player gets 50 points for killing an enemy citizen woman, 100 for a base infantry unit, 150 for a base cavalry unit and so on... On the other hand, if you set a weight of 1 to "Enemy units killed (number)", the system will assign one point for each enemy unit killed, disregarding the type of unit and the cost to produce it. For example, with a weight of 1, a player gets 1 point for killing and enemy citizen woman, 1 for a base infantry unit, 1 for a base cavalry unit... To continue with this example, if you set a weight of 1 to both "Enemy units killed (value)" and "Enemy units killed (number), you will get 51 points for killing and enemy citizen woman, 101 for a base infantry unit, 151 for a base cavalry etc... So probably you might want to consider one of the two parameters only and set the other to 0, but this up to your rating design. The mod's default weight for "Enemy units killed (value)" is 0.1 because in 0 A.D. each enemy unit killed increases a player's score by 10% of the unit's value. For the same reason, the default weight for "Enemy units killed (number)" is 0 because in 0 A.D. a player's score is not affected by the number of enemy units killed (but only by their value). The LocalRatings default weights are the ones that 0 A.D. uses to calculate the total score of a player. 1 Quote Link to comment Share on other sites More sharing options...
Yekaterina Posted June 4, 2022 Report Share Posted June 4, 2022 @Mentula could you explain to me exactly how the ratings are calculated from the weights? I would like a mathematical understanding of your algorithm as it currently spits out some rather unexpected results. Setting everything to 0 except the number of units killed, which is set to 1: (the top killer) It would seem that azeem1121 is the top killer, however, he is nowhere near as effective as vinme or Palin in game. In the matche I played with him, he had a very high kill death ratio because he surprise rushed a few inexperienced players in nomad mode. The total number of kills was less than 100 although he lost very few. The game ended in a crash instead of a proper finish so I think there is something to be fixed here. If you can explain to me how your algorithm works, perhaps I can propose a better mathematical model. The top resource gatherers: This is much closer to reality based on what I have seen from these players. 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted June 4, 2022 Author Report Share Posted June 4, 2022 Sure @Sevda. First of all, I can't see from your pictures the number of matches you've played with the players in your list, but I can imagine that the number of matches is small for those players that you believe being far from your expectations. When the number of matches is small, statistics are unreliable; you need many games to get significant data values. Notice that the mod (v0.25.6, the last version at the moment I am writing) allows you to filter out players whose number of games is small (from the Options > Player Filters menu). That being said, I'll do my best to explain the algorithm hereby; you can find more info at the repository page and, if you want to look at the part of the code that handles the rating computation, you can look at this file. Spoiler As an example, I assume we are computing the rating of a player whose username is Mentula. First, we scan all replays having Mentula as an active player and we ignore the others (this would be extremely inefficient from an algorithmic perspective; this is not how the mod works, but let's assume the mod works this way for simplicity). For each replay, we look at the statistics of Mentula at all instants of the replay (and not only at game's end). In fact, 0 A.D. stores data at given moments of the game, so you can imagine the timeline as a discrete timeline consisting of n instants t_1, ..., t_n. For each instant, we compute Mentula's score according to the weights you set: for example, with "Enemy units killed (number)" set to 1 and all other weights set to 0 ("top killer", as you say) we multiply the weight (1 in our case) by the number of units killed by Mentula at that instant of the game, so to get the score of Mentula at that instant. Thus, we obtained n scores s_1, ..., s_n, each relative to an instant t_1, ..., t_n. We then compute Mentula's average score over all instants. In formulas: (s_1 + ... + s_n) / n. This is Mentula's average score relative to the particular replay we are considering. We also compute the average score of the game, namely the sum of all the average scores of all players (as in step 4), divided by the number of players. This represents the average score of all players as if they were one single player. We compute the ratio (Mentula's score - average score) / average score. This is Mentula's rating relative to the particular replay we are considering. For example, if Mentula's rating turns out to be 0.1, then Mentula has an average score during the game 10% higher than the average score of all players combined. We produce the final rating of a player by computing the average rating over all replays (namely, we sum the ratings obtained at step 6 over all replays and we divide by the number of replays). Sorry for the long answer, I hope it's clear enough. 1 1 Quote Link to comment Share on other sites More sharing options...
rossenburg Posted June 4, 2022 Report Share Posted June 4, 2022 Great work, i suggest you make it autorebuild. Right after a game ends, it should be updated automatically instead of always having to rebuild replay cache before the ratings can be updated too 1 Quote Link to comment Share on other sites More sharing options...
Yekaterina Posted June 4, 2022 Report Share Posted June 4, 2022 1 hour ago, Mentula said: We produce the final rating of a player by computing the average rating over all replays (namely, we sum the ratings obtained at step 6 over all replays and we divide by the number of replays). Thank you very much for the explanation, it makes total sense to me. I think the problem might be alleviated if we take into consideration the average rating of the players present in each match. I am suggesting running the algorithm twice instead of just once; the first run generates a rough rating for everyone, which will contain anomalies like azeem. Then, we run a second pass, this time taking into account the average rating of the players participating in the game, and weight that game accordingly. The total score at the end of the second pass will be a weighted average instead of just an average. If a player participated a lot in OP TGs, even though they perform just 5% above average, their total will still be much higher than a player who dominates the newcomers a few times. High average player level -> more weight. Furthermore, instead of just comparing to the average in one game, we can change the rating +/- threshold depending on the players present: in a game surrounded by experts, even if you have done 10% below average, you still did a good job, being able to hurt those experts somehow. So we should give the player positive credit if they perform anything better than 10% below average. On the other hand, in a noob game, you must perform 150% better than them to show that you are not a noob like the others. Finally, I propose we build a replay bank using a service like Onedrive or Google drive, where everyone dumps their replays into the repository. Then we can query the repository with Mentula's algorithm for players' ratings. Players like I delete replays often to free up disk space, which results in the loss of many records and good games. I believe a repository will also benefit @mysticjim's videos. 2 Quote Link to comment Share on other sites More sharing options...
woodpecker Posted June 10, 2022 Report Share Posted June 10, 2022 Would it be smart to have more weight on, lets say, the last 20 games, less weight on the 20 previous to that and games before that again very little weight? To me that makes sence as people get better, but in different tempo. What matters is how you perform lately, right? 1 1 Quote Link to comment Share on other sites More sharing options...
sarcoma Posted June 10, 2022 Report Share Posted June 10, 2022 better yet, a filter from date x, to avoid scanning many replays 2 Quote Link to comment Share on other sites More sharing options...
Mentula Posted June 10, 2022 Author Report Share Posted June 10, 2022 Thanks @woodpecker and @sarcoma. The idea of filtering/weighting replays according to the date has been brought to attention multiple times (for example by @seeh and @BreakfastBurrito_007) and seems to be a must-have. I guarantee this will be implemented in a future version of the LocalRatings mod. Besides a new match filter that takes into consideration the replay date, I am imagining a customizable "weighting function" that assigns a weight of 1 to the most recent replay and slowly decreases back in time, according to user-given parameters. Thinking out loud, such function could be linear, logarithmic, a step function or even we can let the user choose among multiple possibilities. If any of you has thoughts / references / expertise feel free to give a comment. PS: thanks to all of you who are proposing new ideas, @real_tabasco_sauce, @Sevda and @rossenburg among others. I am happy to see that the LocalRatings mod has raised interest in many players. Although I am not responding to all comments, I am taking all suggestions and their feasibility into consideration. 2 Quote Link to comment Share on other sites More sharing options...
Yekaterina Posted June 10, 2022 Report Share Posted June 10, 2022 @woodpecker @sarcoma you can temporarily cut and paste the old replays into another folder. The names of the replays are based on the date they are played, so something like 20220607_0002 means the second game on the 7th of June 2022. 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted July 5, 2022 Author Report Share Posted July 5, 2022 Hi everyone, I have updated the LocalRatings mod, including a new match filter: the date of validity. This means that the mod now allows to filter out games played before a certain date to the aim of computing the rating. See picture below. This is probably the last update including new features before A26 is officially released. There have been many ideas on how to improve the mod and I wish to collect more feedback from users before committing to new changes. Spoiler Download: as usual, you can download the new release (v0.25.7) of the mod from the zip file attached to this post or from the zip file attached to the first post of this thread or from the official page. LocalRatings-v0.25.7.zip 1 Quote Link to comment Share on other sites More sharing options...
woodpecker Posted July 15, 2022 Report Share Posted July 15, 2022 Great. It would be even more nice if you could set a numer of matches used to calculate ratings individually for all players. Lets say i played 10 games with a player long time ago that i did not play with lately. Id still like to see his or her rating and that we actually played. Just setting a date excludes so much valuable info about players. Ofc it helps on your own rating lately but with mentioned limitations. 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted July 23, 2022 Author Report Share Posted July 23, 2022 (edited) CHARTS! Hello 0 A.D. friends, I am happy to announce a new release (v0.25.8) of the LocalRatings mod with new amazing features! Are we getting better over time? Is our archenemy becoming everyday stronger? Charts will tell! Explanatory picture below. And if you don't like the default chart colors you can always change them (see picture below). Spoiler Further, two new Score Weights have been added: the amount of resources used and the amount of resources sold at the market. Finally, Other minor issues have been addressed. Download: you can download the new release (v0.25.8) of the mod from the zip file attached to this post or from the zip file attached to the first post of this thread or from the official page. LocalRatings-v0.25.8.2.zip Edited July 23, 2022 by Mentula added file with .zip extension 7 Quote Link to comment Share on other sites More sharing options...
rossenburg Posted July 23, 2022 Report Share Posted July 23, 2022 36 minutes ago, Mentula said: CHARTS! Hello 0 A.D. friends, I am happy to announce a new release (v0.25.8) of the LocalRatings mod with new amazing features! Are we getting better over time? Is our archenemy becoming everyday stronger? Charts will tell! Explanatory picture below. And if you don't like the default chart colors you can always change them (see picture below). Hide contents Further, two new Score Weights have been added: the amount of resources used and the amount of resources sold at the market. Finally, Other minor issues have been addressed. Download: you can download the new release (v0.25.8) of the mod from the zip file attached to this post or from the zip file attached to the first post of this thread or from the official page. LocalRatings-v0.25.8 24 kB · 0 downloads this is fire! love it. about the file attached maybe change the extension to zip? had to do it myself before was able to unpack 1 Quote Link to comment Share on other sites More sharing options...
faction02 Posted July 23, 2022 Report Share Posted July 23, 2022 This is a very nice feature. What is exactly "performance over time"? It takes values that varies substantially from negative to very high values and currently, it seem very hard to see how "Rating evolution" is actually moving. Maybe making some tiny adjustment (using different scale or units of measurement, when generating the graph creating an option such that not everything is added ...) to get the different information on the graph more easy to read could be good? Quote Link to comment Share on other sites More sharing options...
Mentula Posted July 23, 2022 Author Report Share Posted July 23, 2022 Just now, faction02 said: What is exactly "performance over time"? It takes values that varies substantially from negative to very high values and currently, it seem very hard to see how "Rating evolution" is actually moving. Every (x,y)-point of the Performance over time graph has y-coordinate equal to the player's rating calculated on the x-th game played. For example, if a player has performed 10% better than the average on their first game (where "better" depends on the weights set, of course), then (1, 10.00) will be a point on the Performance over time graph. 8 minutes ago, faction02 said: Maybe making some tiny adjustment (using different scale or units of measurement, when generating the graph creating an option such that not everything is added ...) to get the different information on the graph more easy to read could be good? Thanks! Actually, the Performance over time graph can have very high and low peaks, making the Rating evolution graph poorly readable. I will put some option to hide/show certain elements of the chart and maybe readjust units/scale if needed. There are plenty of further improvements that can be made, charts have great potential. 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted July 23, 2022 Author Report Share Posted July 23, 2022 @faction02 raised a good point. I have updated the last release (v0.25.8) with a small update. From the Options>Charts menu you can now hide/show charts. This will improve readability, I hope. You will now be able to see charts like this: New features will appear in a future version (probably in A26). Attached you can find the updated version (v0.25.8.2). The same zip file can be found attached to the first comment of the current thread. LocalRatings-v0.25.8.2.zip 4 Quote Link to comment Share on other sites More sharing options...
Mentula Posted August 9, 2022 Author Report Share Posted August 9, 2022 Hi everybody, no version update here, just a little piece of information from the author of the LocalRatings mod (me). The next LocalRatings version (v0.26.1) will be released as soon as the new 0 A.D. Alpha 26 is out. I am impatient to play the new Alpha26 and to see how players' statistics change. As you may know, there will be a new civilization, the Han Chinese and it will be interesting to see how players perform with it. And speaking of civilizations... here's a preview of what you will find in the next LocalRatings version! Yes, per-civ statistics. Cool, isn't it? This is one of many new changes and features I have been working on. It will help giving a more concrete idea on civilization balance and I'm very curious to see what the Han Chinese civilization will reveal! I will provide a full list of new features when the release day comes. For the moment, if any of you wants to try it in advance, you can download the zip file of the development version (or download it manually from the GitLab page). It runs on both A25 and A26. I appreciate any feedback, in particular regarding the new explanatory page, that can be opened from the "About Local Ratings" button on the top of the page. Surely more content can be added and the English might not be perfect (my mother tongue is not English), so I appreciate any suggestion in that sense. Other type of feedback is welcome too. Should you have any thought, feel free to send me a private message. Thanks and... see you on A26! 5 5 Quote Link to comment Share on other sites More sharing options...
Player of 0AD Posted August 9, 2022 Report Share Posted August 9, 2022 @Mentula It's very nice to see this mod growing well! 1 Quote Link to comment Share on other sites More sharing options...
Mentula Posted August 12, 2022 Author Report Share Posted August 12, 2022 @seehat a first glance the issue does not seem to be related with the LocalRatings mod, but arising from somewhere else. I am not using any of the mods you listed but I'll give them a look to see where the issue is originated. One thing you can try to do is to run mods in a different order, 0 A.D. allows to do that (from the Mod Selection page). If this works for you, please leave a comment, other people might be affected too. Quote Link to comment Share on other sites More sharing options...
Mentula Posted August 12, 2022 Author Report Share Posted August 12, 2022 It likely seems to be an issue of StarGui mod introduced between StarGui4 and StarGui7. Maybe @Sevda can take a quick look? It's not the first time I find incompatibility issues across different 0 A.D. mods. Most of the times this is due to modders not proxying existing functions, but overwriting them. I don't know if this is the case... if time allows I can help looking into StarGui. Quote Link to comment Share on other sites More sharing options...
Yekaterina Posted August 12, 2022 Report Share Posted August 12, 2022 6 hours ago, Mentula said: StarGui4 and StarGui7 Across the 3 versions I have implemented actual changes to javascript files that control the shape, text and function of GUI elements; I think one of the minor changes could be interfering with localratings. Before StarGUI v4.0 it was all changing skins and actors, which was harmless. Quote Link to comment Share on other sites More sharing options...
Yekaterina Posted August 12, 2022 Report Share Posted August 12, 2022 The match numbers filter and match duration filters are very important; it allows us to remove flukes and only look at good games which lasted for a long time. This is my list after applying the filters of time > 23 minutes and minimum number of games is 3. I also set the weights to take only military performance (killing machine) into consideration. A new star player named violentviolet does not forfeit their name and scored the top place, leading me by more than 100 points! Quote Link to comment Share on other sites More sharing options...
Mentula Posted September 25, 2022 Author Report Share Posted September 25, 2022 Cheers and... WELCOME TO ALPHA 26! The newest version of the LocalRatings mod (v0.26.1) is attached to this post. Major new features: Automatic (and much faster!) list update. New per-civ rating chart. Ratings of players visible in the lobby page. Documentation, including the rating system and F.A.Q. is now available (clicking on the "Info" button). New match filter: starting resources. .. and other little perks: Player names colored as in lobby. Hovering on evolution chart shows detailed information. More statistical data (e.g. standard deviation). Release notes: the attached new version of LocalRatings v0.26.1 is compatible with Alpha 25. Installation: see the first post of this thread for instructions. Have fun! LocalRatings.pyromod LocalRatings.zip 2 Quote Link to comment Share on other sites More sharing options...
Mentula Posted September 25, 2022 Author Report Share Posted September 25, 2022 I happily announce that LocalRatings (current version 0.26.1, see previous post) is now available for download from the game mod downloader. For reference, here is the page on mod.io. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.