Regardless of the discussion channel, IRC, XMPP, Matrix, Revolt, etc, when starting a match, there should be no problem in using an external tool for voice provided it's very easy to use and requires nothing to install. And example is https://entraide.chatons.org/en/ => Video-conference => create. (use the circular arrows to switch to another server if it doesn't work) It will pop you in a Jitsi Meet room and the link can be shared on the discussion channels for the played to join. Leaving only the microphone enabled and here you go, voice chat channel joinable and leavable.
It's libre, and on a network of ethical hosters (CHATONS) so no centralization and no hassle to pick up a server. (If using regularly, go to the website of the hoster you are using and give a few buck from time to time. That's how that can scale.)