Entrance exam to the lobby: solution to fix inaccurate ratings, smurfs and hostilities in the lobby

[Yekaterina]

Playing a game called Megaglest gave me an idea to fix the rating arguments, smurfing, hostility against unknown / new players: 

Before a new player joins the multiplayer lobby, they are recommended to complete an entrance exam. This is a game where they have to fight against some AIs, and the player's initial rating will depend on how well they did in the exam. We can use their military score in the test game as a main reference, then set some 'grades' like an exam. Furthermore, we can add a time limit of 40 minutes or 1 hour to keep the exam fair for all candidates. 

Grading/scoring system:

Fail: player got killed by the AIs before time was up  (rating <900, city builder)

Weak pass: player was able to survive against the AIs until the end of the test, but did not kill any of the opponents. (900-1100)

Strong pass: The player defeated some AI opponents but did not eradicate everything before the time limit. (1100-1300)?

Excellent: the player had very strong eco, destroyed all of the AIs very swiftly (long before the time limit) and defended against their harassments. Very high KD. (1400+)

 

The time it took for the player to complete the test, the KD ratio, economy growth rate, military score and idle time should all be taken into account when calculating the exact ratings, but the grades should give you a rough idea of what level someone is at. Then each player shall have their exam score in their profile. It is very difficult to have a fluke or underperform in such an exam. We can even add a re-sit feature if they exam went badly for whatever reason. 

 

How it helps:

• Genuine noobs can be identified from their score; those who are really not ready will be blocked from the lobby. 
• Naturally good players or AI experts can have a good head start and gain a respectable grade without having to torture other noobs in rated 1v1s or mess up a few TGs. 
• This idea will prevent smurfing, in the following ways:

1. Smurfs have to beat this AI game for every new account they make, which is a hassle and a lot of time and effort, so most people wouldn't bother. 

2. If they are determined to make another account, then their real skill will be reflected in the result of the entrance exam. 

3. If someone tries to pretend to be a noob... well, it is quite difficult because we have build orders in our muscle memory already... Also, if they want low points then they have to sit there and play as noob as they can for 1 hour without failing the exam. I don't think the smurfs in 0ad have enough patience to do this. At least I don't. Now it is perfectly reasonable for hosts to kick noobs from TGs as well, so no chance of smurfing. 

 

As for the details of the exam, I would recommend a custom made map with both navy and land features that allows both their naval and land battle skills to be assessed. Then, give the player a population cap of 300? In terms of the AIs I think 1 defensive, 1 balanced and 1 aggressive would be a good combination. Of course, this is to be discussed further. 

 

@seeh you played megaglest before, can you give me more insight into their test, or any other ideas?

@Stan` @user1do you think this is acceptable? @Gurken Khan @KKaslana @wowgetoffyourcellphone I need you to suggest how difficult the exam should be and how to set it up since you are AI experts

 

Any thoughts on this, balancing advisors / OP TG players? 

@LetswaveaBook @BreakfastBurrito_007 @Player of 0AD @ValihrAnt @smiley @Dizaka @Jofursloft @vinme @JC (naval supremacist) @alre @Emperior @Wendy22

 

Edited January 31, 2022 by Yekaterina

[wowgetoffyourcellphone]

39 minutes ago, Yekaterina said:

3. If someone tries to pretend to be a noob... well, it is quite difficult because we have build orders in our muscle memory already... Also, if they want low points then they have to sit there and play as noob as they can for 1 hour without failing the exam. I don't think the smurfs in 0ad have enough patience to do this. At least I don't. Now it is perfectly reasonable for hosts to kick noobs from TGs as well, so no chance of smurfing. 

 

I think you underestimate how easy this would be to game.  

[Yekaterina]

34 minutes ago, wowgetoffyourcellphone said:

I think you underestimate how easy this would be to game

Please explain. 

Smurfs get low rating on purpose and get low rating as they wish. Now, when they want to join a TG, the host will assume they are noob kick them, because you can't tell them apart from genuine noobs since they all have low score on their exam. So I don't think smurfs can cause damage to TGs if the host does not let them in. 

They can still create 40 smurf accounts, but none of them would have any significant effect on anyone. 

Edited January 30, 2022 by Yekaterina

[Gurken Khan]

2 hours ago, Yekaterina said:

@Stan` @user1do you think this is acceptable? @Gurken Khan @KKaslana @wowgetoffyourcellphone I need you to suggest how difficult the exam should be and how to set it up since you are AI experts

Huh. I guess you are serious about this, contrary to what I expected reading the title. First of all, I don't think this would be a good idea. I understand we don't suffer from a too large player base, so I wouldn't want to put up too many hurdles for potential newbies. Personally, if I wanted to check out the online multiplayer part of a game and was presented with an ~hour long exam to enter I might say 'What is this $#!t ?' and leave right away. I'm not necessarily convinced saying a n00b is 'not ready' is a good approach; if they play poorly, they will get handed their butt swiftly, but anyone serious about playing strategic games against others will learn from that and improve.

As for smurfs, I think people will just have to live with it. A welcome message in the lobby could tell ppl that ratings might not be accurate and remind everyone to be kind; seems to me that one or two additional active mods would do more for a good experience than any technical approach.

However, should the swarm intelligence decide to go with this exam, I guess something like a lake map, populated with three medium AIs showing the behavior you suggested, would do fine; as you want to include all kind of things into the score ceiling or floor effects shouldn't be too much of a problem.

[Yekaterina]

@Gurken Khan I like your suggestion; not letting them into the lobby because of nubness doesn't help with player base. So perhaps we can make the exam optional; you can either join bare-handed and take a tour, or, you can get yourself an initial rating. Then the TG hosts can decide to kick those who didn't get a high enough score in the test

[Boudica]

I could imagine giving new players the option to become a "certified noob" and get a nice badge next to their name. To make this realistically work, we'd again probably need an automated hosting service. I kind of doubt a person would care to keep reviewing noob games, but I might be wrong.

It might be a nice idea for a small hobby project to set this up. Remind me to look into this when I retire. It could kind of look like a special campaign game you could play in the multiplayer lobby when you want. A bot would keep track of your progress and make it visible to others in the form of badges or other. Sounds like fun to me.

(Edit) A few ideas on how the campaigns might look like:

• Survive for X minutes on the Survival map. Gives you a bronze silver or gold Defender badge, depending on how long you can hold.
• The rush challenge. A small map with scattered groups of enemy soldiers. Your goal is to make a decent attacking army early and clear the map within a time limit. Get various Rusher badges, depending on how fast you can clear the map.
• The boom challenge. Get a Boomer badge depending on how fast you can reach a certain population number.
• The hunter challenge. Lure as many animals and gather as much food as possible within a time limit.

I mean, just keeping track of these results would make nice additional leaderboards. I've often heard people boast about how fast they can grow or something, so imagine there is an official, automatically managed leaderboard for that. What do you think?

Edited January 31, 2022 by Boudica

[Yekaterina]

3 hours ago, Boudica said:

What do you think?

Great idea! 

 

3 hours ago, Boudica said:

kind of doubt a person would care to keep reviewing noob games

We don't need anyone to review noob games; their score says everything. We can prevent them from changing game settings so that there is 0 chance of cheating. Now, if you are hosting an OP TG, you only let people with OP badge in (reasonable). This OP badge is only obtained by good performance in the test, which is optional. So, if a new player who is good comes along, you can just tell them to do the test, instead of saying 'I don't know you' and kicking them. It really hurts people's feelings and they have no obligation to be 'known' to you. 

Similarly, if a smurf nubs on purpose - they get certified noob badge and OP TGs will kick them out. 

City builders will get the cosmic badge and they can be left in peace to build their cities. 

Noob TGs don't really need balance because new players improve very very quickly (especially after reading my guide )compared to OP players, and frankly, most noobs don't really care about balance. 

when it comes to smurfing in a 1v1... if you see their noob badge you probably should refuse to play with them. If you feel bad then maybe unrated. 

 

Edited January 31, 2022 by Yekaterina

[alre]

remember the lobby is used also by casual players who want to do private games together with their friends, oftentimes against AIs or with modded versions of the game. they wouldn't have any interest in taking that test.

[Yekaterina]

40 minutes ago, alre said:

remember the lobby is used also by casual players who want to do private games together with their friends, oftentimes against AIs or with modded versions of the game. they wouldn't have any interest in taking that test.

Exactly, so it is optional. Consider the grade as an entry condition to TGs and not casual  part of the lobby. 

What I want is for both new and old players to see a friendly lobby instead of one with hosts who only says 'I don't know you' then kick

A guy with 200 IQ has pointed out to me why some people are extremely against changing nicknames: if you change the nickname, they can't predict your strategy (all they know is your skill level but they don't know whether you are boomer or rusher). In which case they can't just exploit your habits, instead, they have to scout you. Now, in my opinion, scouting your enemy is the correct play, instead of saying: "Oh, @Dizaka is next to me so he will rush me with merc sword cav. Yekaterina is his pocket and she will definitely boom." 

Prediction by player name is never the correct mindset because anyone can change their strategy at any time. If they do, it is even more unacceptable to say "oh, berhudar must have hacked her account" just because I rushed with mass cavalry in a game even though I normally boom. 

 

[Dizaka]

16 hours ago, Yekaterina said:

Grading/scoring system:

Fail: player got killed by the AIs before time was up  (rating <900, may not be allowed into the lobby)

Weak pass: player was able to survive against the AIs until the end of the test, but did not kill any of the opponents. (900-1100)

Strong pass: The player defeated some AI opponents but did not eradicate everything before the time limit. (1100-1300)?

Excellent: the player had very strong eco, destroyed all of the AIs very swiftly (long before the time limit) and defended against their harassments. Very high KD. (1400+)

 

The time it took for the player to complete the test, the KD ratio, economy growth rate, military score and idle time should all be taken into account when calculating the exact ratings, but the grades should give you a rough idea of what level someone is at. Then each player shall have their exam score in their profile. It is very difficult to have a fluke or underperform in such an exam. We can even add a re-sit feature if they exam went badly for whatever reason. 

Honestly, I think the grading/scoring system isn't okay.  People can manipulate this.

I think it's normally more about communicating information to players.  It's likely that a normal distribution can be created using IP addresses based on ranks of players that have played from that IP address.  Knowing the bell curve you probably can predict player rank, I would think.

[smiley]

17 hours ago, Yekaterina said:

If someone tries to pretend to be a noob... well, it is quite difficult because we have build orders in our muscle memory already

Is it though?

[Yekaterina]

1 hour ago, Dizaka said:

It's likely that a normal distribution can be created using IP addresses based on ranks of players that have played from that IP address.  Knowing the bell curve you probably can predict player rank, I would think.

Well yes but no...

Please don't track my IP

Also, what if I change it with a VPN?  

1 hour ago, smiley said:

Is it though?

Again, TG hosts can reject noobs from their games

[LetswaveaBook]

@Yekaterina I  don't know if this is a valid question:

Who will spent his/her time to create a entrance test?

 

Doing an entrance test before you can play ranked games does not seem so bad.

3 hours ago, alre said:

remember the lobby is used also by casual players who want to do private games together with their friends, oftentimes against AIs or with modded versions of the game. they wouldn't have any interest in taking that test.

I think this is a very valid concern.

 

Finally, what is the goal and what is the method? If stopping smurfs is the goal then I don't know if the entrace test is the ideal method. But I have a better idea for an entrance requirement for OP TGs and it does not require a programmer. Get your 1v1 rating above 1500 or be kicked.

 

Jokingly, I can't help to remark this: I don't know if this idea is fine with meth.

[Dizaka]

1 hour ago, Yekaterina said:

Well yes but no...

Please don't track my IP

Also, what if I change it with a VPN?  

Again, TG hosts can reject noobs from their games

I mean 1-way hashes exist for a reason.  Once you hash the data you can throw away the original data (IP addresses) for privacy reasons.

The normal distribution can be based on the different portions of the IP address.  There's four octects and they have a specific meaning.  So a 1st octet normal distribution can be the least predictive.  A 2nd octet distribution can be more predictive.  A 3rd octet distribution can tell you a likely scenario.  A 4th octet distribution is very predictable likely.

And there likely is a database that stores IPs based on frequency of use and/or purchase.  Meaning if it is a VPN ip you know its possibly a smurf.  If it's a college-bought IP range you know it's likely not a smurf but could be one.  If it's a one IP of a number of IPs from a common ISP it could be a smurf but probably not.

Most people aren't capable enough of changing their IP in this game anyway.

Basically, all of this allows you to make an educated guess on balancing.

Edited January 31, 2022 by Dizaka

[Yekaterina]

20 minutes ago, LetswaveaBook said:

Who will spent his/her time to create a entrance test?

I and whoever agrees with me, during the Hilary-trinity term break

21 minutes ago, LetswaveaBook said:

Get your 1v1 rating above 1500 or be kicked.

Of course you can reach 1500 by playing 1v1s, not hard, but:

1. Some TG hosts: hey, you only played 10 games, even though you are 1500+, 10 is too small a sample size, so I don't know you, hence I will ban you because I think you are smurf. 

2. If an AI expert comes, then it means they will have to decimate 10 or 20 noobs to get that rating. It wasn't fun for the 20 noobs. 

3. Takes a long time to find 20 1v1 games...

24 minutes ago, LetswaveaBook said:

I think this is a very valid concern.

Yes of course. That is why I have changed the exam to be OPTIONAL! It is a quick way to prove yourself to TG hosts. 

26 minutes ago, LetswaveaBook said:

Jokingly, I can't help to remark this: I don't know if this idea is fine with meth.

Is this what you mean? 

[LetswaveaBook]

1 hour ago, Yekaterina said:

Is this what you mean? 

No, I meant reza-meth.

[Yekaterina]

2 minutes ago, LetswaveaBook said:

No, I meant reza-meth.

His name is spelled reza-math and we should call him

reza-shah 

(king reza)

[Emperior]

@Yekaterina The idea which you have written here, I think I have posted before in lobby / chat room of my host in a23 / forum . Which was based on Counter strike cs go.  We used too have some debates regarding ratings in lobby / match lobby. Which i believe it is good except excluding anyone from lobby for being bad at it.

 

ELO point's doesn't need too be removed, but I believe that giving a player few games of unrated which would be "judged" by the system how many elo point's he will get it's not a bad idea. (Counter Strike global offensive has good matching due firstly if I remember correctly you must play 10 games for a new account to be assigned a specific rank. After that you must climb to the top, whatever the system gives you. (Often i got silver, one of the worst. Pain in the a...) To get to better rank a silver often was mixed with "nova" rank to be tested next to another players and beat the opposite teams quite few times. Pretty darn hard. Elo point's doesn't need to be removed, Simply apply elo points to simillar system which in cs go.

Edited January 31, 2022 by Emperior

[seeh]

21 hours ago, Yekaterina said:

@seeh you played megaglest before

yes takes more time for a slump. i did not played megaglest rated and not in lobby often (maybe some little times). sry dont know how it woks there.

[BreakfastBurrito_007]

Honestly, I think the best, and simplest feature for counter smurfing would be more stats about 0ad accounts. It would be way easier to identify the smurf or validate claims of smurfiness. The following information would be very useful

1. account creation date
2. hours the person has logged on lobby.
3. civilisation usage breakdown
4. top 15 players who join games with this player

I think at the bare minimum this would help prevent the most toxic smurf behavior (misleading others on skill level).