Matchmaking services

[Guest afeder]

Hi,

First off: I'm impressed by this game. I hope many skilled programmers (and artists etc.) will find their way here and help make it outshine the big commercial publishers.

I've read some references here on the forum to basic multiplayer 'lobby' functionality being planned. I'm a great believer in "doing one thing and doing it well" and agree that the main project shouldn't have to waste development resources on more advanced multiplayer functionality like matchmaking.

However, matchmaking is really what make games like Starcraft II a great experience in multiplayer and I think it would be a shame to deprive 0 A.D. of this functionality. Thus, I want to suggest a different approach: make an API for third party matchmaking services. This way 1) 0 A.D. developers can focus on what they do best (game development) and 2) the player may be offered not just one, but many different ways of finding an opponent matching his exact criteria.

The workflow could look something like this:

1. The player discovers the website of a matchmaking service he likes called AwesomeMatch. Maybe he had it recommended by a friend, maybe he read a review of it somewhere, maybe he found it some other way matching his exact habits. The point is: he found his way to their website.
2. The website explains how he may add their matchmaking service to 0 A.D. Specifically, he just needs to click a "Add matchmaking service" button in the multiplayer UI and enter the domain name of their service: awesomematch.org.
3. Once the player has done this, the game retrieves a machine-readable description of the service in a canonical way (e.g. using host-meta) from the server at the specified domain name. If the retrieval is succesful and the description is valid, the game adds it to its internal list of matchmaking services registered by the player.
4. The player may now initiate a new game matched by the AwesomeMatch service by clicking "New challenge", setting the desired game options (map, max players, etc.), selecting the service from the list and clicking "Find opponents."
5. The game then sets up a new game on the lobby server (which may be hosted by WFG or whoever) and opens a screen showing a list of players who have been matched so far (initially, only the player himself.)
6. The game finds, in the previously retrieved service description, the URI of the actual web service for finding opponents and asks the lobby server to query it. While waiting for a response it displays the text "Searching for opponents..." on the player's screen.
7. The lobby server queries the URI with the game options selected by the player as well as a callback URI for notifying the lobby server of players that has been matched for the game. Optionally, the service may return the URI of a web page where the player may specify additional, service-specific options (e.g. one service may offer to only match players speaking a particular language, another may offer to match players by 'clans' or 'guilds', etc.) If the service returns such a URI, the game can open it in a new browser window while the game continues to await response from the lobby server in the background.
8. Whenever the matchmaking service finds a matching player, it invokes the callback URI specified by the lobby server with an URI representing the matched player. The lobby server may invoke the matched player URI to retrieve options like name, race, team etc. and report this information back to the game client so it can be displayed on the player's screen.
9. When the matchmaking service is done searching, i.e. when the max players or similar has been reached, it determines which of the matched players should host the game by looking at their latency etc. It then asks the given player's game client to host, via his lobby server. If, for whatever reason, the game client fails or rejects to set up a game host, the matchmaking service asks the next client in line, and so on, until a host is created.
10. The new game host reports back to the matchmaking service with its IP address and the matchmaking service broadcasts the IP address to each of the matched players (via their lobby servers) whose game clients then automatically connect to the host.
11. An interim period now ensues where everyone waits for all matched players to connect to the host. If anyone leaves the game during the interim period or if it takes too long, or if any of the players (including the host) for any other reason no longer want to be part of the game, they may leave at any time at their own discretion, and either restart the matchmaking process (for their own part) or cancel it altogether.

Note that all players are peers in the matchmaking process - no one is designated host until the service has matched all players needed for the game to begin. Thus the player is not required to decide whether he wants to host prior to even knowing whether anyone is going to join or if he is the best possible host of those who do want to join.

I imagine the protocol between the lobby server and the matchmaking service as being HTTP-based because a) there is a lot talent for HTTP-based services out there and b ) HTTP-based hosting is cheap and widely available.

[DGMurdockIII]

http://grapple.linuxgamepublishing.com/

[Guest afeder]

http://grapple.linux...publishing.com/

Doesn't seem matchmaking related.

[Ykkrosh]

If I'm understanding correctly, I think the biggest problem is that you're proposing the game should support multiple independent matchmaking services, which will cause a fragmentation of the player community across the services. The value of a matchmaking service increases non-linearly with the number of players it can match between - 8 services with 1 player each would be useless (since nobody could find anyone to play with), and 1 service with 8 players would be much more useful. If some services are better than others, then it adds significant effort for players to learn about which is the best to use, and adds risk of them choosing a sub-optimal one. So I think the ideal situation is that there's only one matchmaking service and it's selected by default - anything else would make it worse for players.

In terms of implementation complexity, building a robust API between a central lobby server and third-party matchmaking service would probably be harder than just building a single central lobby+matchmaking service, given the difficulty introduced when multiple parties have to coordinate. So my current inclination would be to focus on designing a good 'official' matchmaking system (I'm not sure what would count as "good" so I imagine we need lots of input and research), rather than trying to spread the work to unspecified other people, since it's probably easier and should give a better result.

[quantumstate]

I think it would be nice to have the central lobby server running over http, then it lets people easily set up their own servers on cheap hosting. Ideally you could add multiple lobby's and use them all at the same time which would solve the fragmentation issue. The difficulty of implementation would increase significantly though so it is probably worth implementing the easiest thing first as you say.

[bstempi]

If I'm understanding correctly, I think the biggest problem is that you're proposing the game should support multiple independent matchmaking services, which will cause a fragmentation of the player community across the services. The value of a matchmaking service increases non-linearly with the number of players it can match between - 8 services with 1 player each would be useless (since nobody could find anyone to play with), and 1 service with 8 players would be much more useful. If some services are better than others, then it adds significant effort for players to learn about which is the best to use, and adds risk of them choosing a sub-optimal one. So I think the ideal situation is that there's only one matchmaking service and it's selected by default - anything else would make it worse for players.

In terms of implementation complexity, building a robust API between a central lobby server and third-party matchmaking service would probably be harder than just building a single central lobby+matchmaking service, given the difficulty introduced when multiple parties have to coordinate. So my current inclination would be to focus on designing a good 'official' matchmaking system (I'm not sure what would count as "good" so I imagine we need lots of input and research), rather than trying to spread the work to unspecified other people, since it's probably easier and should give a better result.

I think it would be nice to have the central lobby server running over http, then it lets people easily set up their own servers on cheap hosting. Ideally you could add multiple lobby's and use them all at the same time which would solve the fragmentation issue. The difficulty of implementation would increase significantly though so it is probably worth implementing the easiest thing first as you say.

I'm currently evaluating using the game (specifically programming AIs) as an educational tool. This topic interests me.

I think having a central service is a good thing (it's a good starting point for finding other players, etc). I don't think players should be locked into this. I think it would be useful to have the ability to have private servers for clans or for large LANs (or for my students).

I also think that even if we control the lobby and the game, we are still going to need to build a robust API . We're not going to want an upgrade of the lobby or the game to break people's ability to utilize either. It's also worth noting that just because the API is robust does't mean that the implementation has to be. If a robust API were designed, it could be incrementally implemented so that over time users see new features without having to worry about backwards compatibility.

If anything, out implementation of such a think should serve as an example to others or should be independently runnable itself...sort of like the way that some games can operate in pure server mode.

[Ykkrosh]

I think it would be useful to have the ability to have private servers for clans or for large LANs (or for my students).

For private servers, do we need anything more than the ability to connect to an arbitrary IP address (i.e. how the game works now)? For LANs, I think the game should switch between finding players via the standard online lobby server and via LAN broadcasts (without any server at all; just do a UDP broadcast and see if any clients on the LAN respond), which doesn't sound too hard in theory.

(In any case I think we should let players choose a different lobby server as a config option if they want to do something special, and the server software ought to be designed to be relatively easy for them to set up their own local copy, because I don't like trying to enforce lock-in, though I still think we should encourage convergence on a single server to provide the best experience for players by default.)

I also think that even if we control the lobby and the game, we are still going to need to build a robust API .

I think afeder's suggestion was something a bit like

 C--..   ..--M
 |    \ /
 C-----L-----M
 |    / \
 C--''   ''--M

while the simple model with a single central lobby+matchmaking service is like

 C--..
 |    \
 C-----L
 |    /
 C--''

where C is a game client, L is the lobby server (controlled by WFG), M is a third-party matchmaking service.

C<->C communication is what we do now (i.e. clients connecting to each other by IP address, using ENet, to run a match) and will remain basically the same. C<->L communication is largely the same in either model, but L<->M communication is new if we're supporting third-party matchmaking services (coordinated by a central lobby server). The C<->L API is relatively easy to implement robustly, since we know exactly what technology will be used on either end of the connection and can optimise the protocol for that, and it will probably be the same people implementing both ends so there's little danger of miscommunication. L<->M would be much more complex since (as I understand it) the whole purpose is that there are multiple independent implementations of M, so the protocol needs to be more technology-independent and much better documented and tested. So we need the C<->L API in any case, and my concern was just that implementing the additional L<->M API robustly would be hard.

[Guest afeder]

If I'm understanding correctly, I think the biggest problem is that you're proposing the game should support multiple independent matchmaking services, which will cause a fragmentation of the player community across the services. The value of a matchmaking service increases non-linearly with the number of players it can match between - 8 services with 1 player each would be useless (since nobody could find anyone to play with), and 1 service with 8 players would be much more useful. If some services are better than others, then it adds significant effort for players to learn about which is the best to use, and adds risk of them choosing a sub-optimal one. So I think the ideal situation is that there's only one matchmaking service and it's selected by default - anything else would make it worse for players.

There is nothing to say that the individual matchmaking services couldn't interoperate behind the scenes.

I understand that a single 'official' server is conceptually simpler, but, being open source, the player community inevitably will fragment anyway, because folks will be setting up their own servers. An API for "federating" the matchmaking will bring the players back together again, whether there is an official server or not.

[Ykkrosh]

being open source, the player community inevitably will fragment anyway, because folks will be setting up their own servers.

I don't follow why that would be the case. If there is a single lobby/matchmaking/etc server that works adequately, why would people bother setting up their own that compete with it?

(I'm assuming that single server would be maintained by WFG, or if WFG dies then by some other single group that wants to continue supporting the game. The server should be designed to support forks of the game engine (maybe by treating them like mods, or like completely independent universes of players, or whatever) so non-WFG forks could choose to still use the WFG server and wouldn't need to set up their own.)

If it's true that there would be multiple matchmaking servers that could potentially interoperate, then I agree it'd be good to design a way to let them interoperate, but I don't see why we can't (or shouldn't) prevent that fragmentation in the first place.

[Guest afeder]

I don't follow why that would be the case. If there is a single lobby/matchmaking/etc server that works adequately, why would people bother setting up their own that compete with it?

By way of the second law of thermodynamics as applied to software: if it can be forked, it will be forked. Specifically, if people are not content with the matchmaking rules of the official server, or if there are features that the official server doesn't want to support (say, posting notices of new games to my favorite social network.) A centralized server can only ever be the "best common denominator" for all players, not the best possible for every single individual or sub-community, and that's what lead to forking.

[bstempi]

For private servers, do we need anything more than the ability to connect to an arbitrary IP address (i.e. how the game works now)?

I would think so. Direct IP communication doesn't work well through NAT'ing firewalls. The hosting party has to either be directly connected to the internet (no NATing) or has to open up firewall ports. No such thing needs to be done with several other games. By allowing someone to host a central server, the server can have the firewall rules or direct connection, while the clients are all behind their NAT'ing firewalls without having to make modifications.

My suggestion was more like:

C--.. ..--S

| \ /

C-----L-----S

| / \

C--'' ''--SWhere C was the client, L was the lobby server, and S was some dedicated server.

In TF2, Urban Terror, and other first person shooters, people can start dedicated servers. Those servers can choose to announce itself to the central lobby (thus making itself public) or to not list itself on the lobby (making it private). Dedicated servers could offer maps, run the AIs, etc.

If a player is looking for a game, they could go to the central lobby to see what servers are available. They can find one playing a map that they like with the number of people they'd like, and then they can join that server and play.

[Ykkrosh]

Direct IP communication doesn't work well through NAT'ing firewalls. The hosting party has to either be directly connected to the internet (no NATing) or has to open up firewall ports. No such thing needs to be done with several other games.

There are ways to get around most NATs automatically (UPnP and NAT punchthrough, which we should implement), though I agree it's a pain in general and it's helpful to have servers on direct IP connections.

But I think the network topology (servers and clients and NATs etc) generally shouldn't be exposed to players, because it's the wrong level of abstraction. Players should only have to care about other players, and about concepts like who's allowed to change the game settings, not about the implementation details of dedicated servers and port forwarding and hosting vs joining etc. (Most traditional PC games do seem to expect the players to learn about networking details, but I think that's bad since it distracts from the goal of playing the game and makes casual players likely to get frustrated and give up. Console games and more recent PC games seem to prefer to hide the details and do everything automatically, since that provides a better experience for most players.)

Our game's network architecture means that the server doesn't do any real computation - it simply copies messages between all the clients (and provides some global ordering and synchronisation etc), so its resource usage is very low. FPS-style resource-heavy dedicated servers aren't the right model for what 0 A.D. servers do - instead they're more like relays, and a single machine could provide a relay service for many matches (maybe hundreds?) simultaneously. The downside is that they add network latency, but if there were several relay servers spread around the world then it shouldn't be too bad. (I expect it wouldn't be hard to find people to donate or fund machines (or virtual machines if they don't have unreasonably jittery CPU scheduling) to run the relay service on.)

So instead of telling players to set up dedicated servers, I think it would be better to have a lobby/matchmaking/etc service that lets players set up matches in a more abstract sense (so all that matters is the set of players, and the game settings, and who has admin control to kick other players or terminate the match etc), and then it can automatically figure out how best to run that match (either by picking one player as host, perhaps using UPnP/punchthrough to open their ports, or by detecting the lowest-latency relay server and connecting everyone through that). (Plus an "advanced" mode that lets players debug or override the automatic behaviour - things should work by default but the player should be in control.)

(That's mostly unrelated to the question of how to implement the lobby/matchmaking/etc service, though.)

[Sonarpulse]

Hello everybody. Guess it's my first post here so I'd just like to say 0 a.d. has to be the greatest even remotely FOSS RTS I have seen.

Back on topic, for those of you familiar with the Command and Conquer series of video games, there as been an effort to recreate a multi-player service for the older, now unsupported games at http://cncnet.org/. Code is at https://github.com/hifi/. I'm not quite sure on the details, but I'm pretty sure CnCNet is fairly tolerant of whatever it's supported game's protocol may be, and mainly focuses on just setting up the networks to tunnel it's games' protocol through. It has both dedicated server and p2p modes, which or going to be made completely seamless in a future version, so no matter what setup you guys are thinking of, this could still be relevant.