Charmar

My brother and I are trying to play some 0AD with friends but only one of us seems to be able to connect to lobbies at a time, including observing games with open spots. Have opened inbound ports but still only able to have 1 person in at a time. Any tips or is there something wrong on my end?