As we know thanks to the debugging work of @Shad_ and others that the problem seems to be related to client certificates, I dug into it a bit deeper.
While I have no Windows available, I couldn't figure out the root cause, but I'm confident I found a workaround to get rid of the problem server-side without players having to do anything.
As I mentioned, it isn't possible to disable requesting client certificates in ejabberd right now. After a discussion with the ejabberd devs (https://github.com/processone/ejabberd/discussions/4593), this will be added to the next version of ejabberd though (https://github.com/processone/ejabberd/commit/5b03ca51e003bae785d186f183c0cd276fc89e97), so once we update to that version at some point in the future, we should be fine.
Until then, the workaround I have is to instruct the clients to only send us certificates signed by a specific CA. That CA is a dummy CA we have created and where clients haven't signed certificates from, which should result in clients not sending certificates at all. I have prepared all changes necessary for that (https://github.com/0ad/lobby-infrastructure/pull/43) and will likely deploy them in the upcoming days.
It would be awesome if we'd have somebody who is currently affected by this problem or can reproduce it, so we can test if the changes indeed fix it.