UPnP Fix for multiple clients/consoles playing the same game

[Saber]

@encrypt1d said in UPnP Fix for multiple clients/consoles playing the same game:

@saber
I think you mentioned in an earlier post that you have a linux firewall that works.

Is that a similar config? I.e. miniupnpd with pf?

If it is, you could run it with the same debug level, and perhaps diff the responses from miniupnpd (via packet compare) that say the port is already taken.

It wouldn't fix anything, but could point to a deeper reason why the playstation isn't trying new ports each time.

Prior to the patch, my windows/Call of Duty client would randomly pick new ports forever, filling up the UPnP mapping table without bounds. After the patch, it gets the 718 error the first time, picks a new port and the second one sticks.

I've removed the Linux firewall, mostly because I prefer PFSense but I'm really curious now what Linux does differently with how it handles multiple devices asking for the same port. I'll see if I can standup a Linux firewall in a VM and put the Playstation's in the same VLAN and log the miniupnp / packet capture to try and narrow down what the differences are if any.

To answer your question, Linux isn't the same as Packet Filter. Linux uses IPTables / Netfilter. While BSD / Unix based OS's use Packet Filter.

I thought for sure I'd see a difference in the miniupnp responses considering both Playstations now show a NAT Type 2 with Static Port mapping enabled, but nothing. Miniupnp responded the exact same way, yet Playstation is reporting a different NAT Type...

What is kind of funny, if not a little ironic is that Playstation 4 based Operating System is FreeBSD.... lol

[encrypt1d]

@saber

I'll hopefully save you some time on setting up that test firewall. If it will running miniupnpd & pf, it will not work at all if the firewall's wan ip is a private RFC1918 address. I'd say there's low value in choosing that config for your test anyway, since the outcome would likely be the same. I'd go with a different implementation.

I have a separate thread tracking that bug, and I have proposed a code change to the miniupnp developers. You can catch up on these threads if you are interested.

https://forum.netgate.com/topic/169773/miniupnp-full-cone-double-natincorrectly-adding-rules
https://github.com/miniupnp/miniupnp/issues/598

[Saber]

@encrypt1d said in UPnP Fix for multiple clients/consoles playing the same game:

@saber

I'll hopefully save you some time on setting up that test firewall. If it will running miniupnpd & pf, it will not work at all if the firewall's wan ip is a private RFC1918 address. I'd say there's low value in choosing that config for your test anyway, since the outcome would likely be the same. I'd go with a different implementation.

I have a separate thread tracking that bug, and I have proposed a code change to the miniupnp developers. You can catch up on these threads if you are interested.

https://forum.netgate.com/topic/169773/miniupnp-full-cone-double-natincorrectly-adding-rules
https://github.com/miniupnp/miniupnp/issues/598

That saved me a TON of time. CGNAT what a pain. I'm spending extra with my ISP to get a Public IP so I'm not behind a CGNAT.

I'll have to see if I can find some old hardware to throw a Linux Firewall distro on it and see what is different. The problem is always time.

[rivageeza]

Had some time to perform further testing just now.

I disabled NAT Reflection mode for port forwards and unchecked both Enable NAT Reflection for 1:1 NAT and Enable automatic outbound NAT for Reflection and the patch still works.

Tested, PC + PS5, Warzone, both in the same lobby/match at the same time. Both devices reported open NAT.
Pre-patch this wasn't possible.

[m0nji]

@rivageeza and no static port mappings in NAT Outbound Rules?

[rivageeza]

@m0nji nothing to do with the PC or consoles. I used to but I removed them and retested last week.

I did create one rule over the weekend but that's for Wireguard, my LAN is 10.10.10.1/24, PS5 is 10.10.10.30 and PC 10.10.10.38.

Wireguard rule below

[yorke]

@jimp
I applied the patch fix for upnp reboot Pfsense deleted all rules
under portforward and hybrid outbound NAT Reboot Pfsense,
Network address translation all unchecked and disabled,
enabled UPnP & NAT-PMP and added ACL rules,
Tested upnp with warframe, Call of Duty Vanguard, Overwatch, Borderlands
StatusUPnP & NAT-PMP UPnP & NAT-PMP Rules the Playstation connections
were listed, but i could not connect to the servers or play.

I had to recreate all portforward and hybrid outbound NAT in order
for the playstation to connect,NAT-2
but now the StatusUPnP & NAT-PMP UPnP & NAT-PMP Rules and
Shell Output - pfSsh.php playback pfanchordrill are empty
when PlayStation is connected and playing games.

BEFORE CREATING portforward and hybrid outbound NAT RULES

Shell Output - grep miniupnpd /tmp/rules.debug
binat-anchor "miniupnpd"
nat-anchor "miniupnpd"
rdr-anchor "miniupnpd"
anchor "miniupnpd"

Shell Output - pfSsh.php playback pfanchordrill
listed all connections

StatusSystem LogsSystemRouting
Mar 2 14:16:17 miniupnpd 18758 HTTP listening on port xxxx
Mar 2 14:16:17 miniupnpd 18758 no HTTP IPv6 address, disabling IPv6
Mar 2 14:16:17 miniupnpd 18758 Listening for NAT-PMP/PCP traffic on port xxxx

I remove and apply the patch following the advice in this post but with no luck
I see the connection under Shell Output - pfSsh.php playback pfanchordrill
and StatusUPnP & NAT-PMP UPnP & NAT-PMP but cannot connect to server can't play
put rules back in place every thing works.

[jimp]

@yorke Please start a new thread for your issue, it doesn't sound related to this. You have some other type of problem with UPnP.

[fusioncha0s]

@rivageeza I can confirm that this setup after applying the patch works with 2 PCs trying to play Call Of Duty Vanguard, It took rebooting the pfsense box and both my pc's, I can run Call of Duty Vanguard without any problems with both PCs having an open NAT. My setup runs a VLAN with both PCs on that VLAN.

Before Applying the Patch - Had same setup as you mention. I could launch Vanguard on PC1 and get to the multiplayer screen and it would show Open but sometimes moderate. While PC1 was sitting in the multiplayer screen waiting to start a match, i tried to launch Vanguard on PC2. I was able to get to the "Connecting to services" screen during bootup until I was met with the "Disconnected from Server" which forces me to exit to desktop.

After Applying the Patch - Keeping the same setup you mention. Only difference is I rebooted the pfsense box and also shutdown both computers while the pfsense box was rebooting. Gave it a good 5 minutes and turned both computers on. Started PC1 with vanguard, boots up as usual to multiplayer screen and an Open NAT this time. Booted up PC2 vanguard and it passes the initial "connecting to services" screen and I can get into multiplayer. PC2 also showed and Open NAT.

[m0nji]

@fusioncha0s he is not using this specific config anymore. you have to scroll down a bit ;)

[fusioncha0s]

@m0nji thank you for letting me know. I have also disabled NAT Reflection mode for port forwards and unchecked both Enable NAT Reflection for 1:1 NAT and Enable automatic outbound NAT for Reflection with the patch and both my pc's run Vanguard without and issue and contains Open NAT for both. I still have my outbound setup though along with my ACL's.

[chief]

Can anyone that has this patch applied tell me what result running this test is returning?

https://github.com/automation-stack/nat-discovery

Used to use PF a while back before I noticed it didn’t support 2 PCs playing Halo (1 person would get kicked out)

Really want to make the switch to pfsense but the only reason I stay with WRT is because I get full cone NAT, which is the best for gaming.

Thanks!

[encrypt1d]

@chief said in UPnP Fix for multiple clients/consoles playing the same game:

I get full cone NAT

@chief You'll want to read up on this thread. Those of us behind a full cone double NAT aren't fully operational with this patch (yet). Once they issue a miniupnpd release, and pfSense includes it, it will work, assuming nothing else gets broken of course. I am running a version I built myself, and both STUN and the "Override WAN address" options work flawlessly on 2.6.0 CE. Without the additional miniupnpd fix, only those with public IPs on their WAN interface can successfully use UPNP.

https://forum.netgate.com/topic/169773/miniupnp-full-cone-double-natincorrectly-adding-rules

[encrypt1d]

@chief said in UPnP Fix for multiple clients/consoles playing the same game:

https://github.com/automation-stack/nat-discovery

Also a note on using a tool like the automation-stack/nat-discovery link posted above:

Without manual changes, this will never result in anything other than a restricted NAT result if run on a host behind the firewall. The tool does not actually make any UPNP port mapping requests, so its tests will not be NATed as desired or allowed back through the firewall. You can probably add the NAT/FW rules yourself to make that work, however ...

The STUN implementation included in miniupnpd includes a very similar check, which runs directly from the firewall itself when STUN is enabled. Assuming you have the firewall rule to allow the inbound connection attempts directly to the firewall WAN IP, it does in fact report a fully open NAT.

Mar 18 10:05:43 firewall miniupnpd[782]: STUN: Performing with host=stun.sipgate.net and port=3478 ...
Mar 18 10:05:43 firewall miniupnpd[782]: resolve_stun_host: stun.sipgate.net:3478 => 217.10.68.152:3478
Mar 18 10:05:43 firewall miniupnpd[782]: perform_stun: local ports 38769 13999 4669 15715
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: waiting 3 secs and 0 usecs
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: received responses: 1
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: waiting 3 secs and 0 usecs
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: received responses: 3
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: waiting 3 secs and 0 usecs
Mar 18 10:05:43 firewall miniupnpd[782]: wait_for_stun_responses: received responses: 4
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: Type 0x0101, Length 68, Magic Cookie 2112a442
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: MAPPED-ADDRESS X.X.X.X:38769
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOURCE-ADDRESS 217.10.68.152:3478
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: CHANGED-ADDRESS 217.116.122.136:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: XOR-MAPPED-ADDRESS X.X.X.X:38769
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOFTWARE Vovida.org 0.96
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: Type 0x0101, Length 68, Magic Cookie 2112a442
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: MAPPED-ADDRESS X.X.X.X:13999
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOURCE-ADDRESS 217.10.68.152:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: CHANGED-ADDRESS 217.116.122.136:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: XOR-MAPPED-ADDRESS X.X.X.X:13999
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOFTWARE Vovida.org 0.96
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: Type 0x0101, Length 68, Magic Cookie 2112a442
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: MAPPED-ADDRESS X.X.X.X:4669
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOURCE-ADDRESS 217.116.122.136:3478
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: CHANGED-ADDRESS 217.116.122.136:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: XOR-MAPPED-ADDRESS X.X.X.X:4669
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOFTWARE Vovida.org 0.96
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: Type 0x0101, Length 68, Magic Cookie 2112a442
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: MAPPED-ADDRESS X.X.X.X:15715
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOURCE-ADDRESS 217.116.122.136:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: CHANGED-ADDRESS 217.116.122.136:3479
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: XOR-MAPPED-ADDRESS X.X.X.X:15715
Mar 18 10:05:43 firewall miniupnpd[782]: parse_stun_response: SOFTWARE Vovida.org 0.96
Mar 18 10:05:43 firewall miniupnpd[782]: STUN: ext interface igb0 with IP address Y.Y.Y.Y is now behind unrestricted NAT 1:1 with public IP address X.X.X.X: Port forwarding is now enabled

[chief]

@encrypt1d Thanks for your reply. The "Override WAN address" is that for just users that are behind a double NAT? Are you saying that if I run the latest pfsense with only the patch would I be on a full cone NAT? My public IP is on my WAN. I wouldn't need the additional miniupnpd fix then right?

I used to use this site to test my full cone nat http://nattest.net.in.tum.de/
It looks like they brought it down and are building a new test that uses javascript instead of the old one that used Java-Applets https://www.net.in.tum.de/research/software/

I found the github test which I now know is not quite the same thing, since it doesn't do upnp port mapping requests. Do you know of any test that does do something like that website that I posted above?

[encrypt1d]

@chief said in UPnP Fix for multiple clients/consoles playing the same game:

The "Override WAN address" is that for just users that are behind a double NAT?

Yes, that is correct. Same goes for the STUN option. You wouldn't need either option or the miniupnpd fix if your setup isn't double NAT. When you asked about full cone NAT, I assumed you meant your ISP was NATing.

Full cone NAT is a term which typically applies to your ISP's upstream router. If they are not NATing anything, then it is fully under your control.
This page has a good write-up on the definition:
https://dh2i.com/kbs/kbs-2961448-understanding-different-nat-types-and-hole-punching/
And they have a tool as well, which should show as "permissive" when you run it (assuming you are defaulting your outbound traffic with static port NATing.)
https://clients.dh2i.com/NatTest/

The truth is any NAT test tool you use only tests your existing NAT rules, not what a game will see, because the game will program new NAT/Firewall rules via UPNP that override anything you have configured.

[Jon8RFC]

I don't know where to find UPnP-related logs, but I had an odd issue with it not working just now (2022-03-20 ~02:15-02:30, for my own reference). COD:Warzone was showing as "strict NAT"

Filter reload and reset states did not resolve it. The firewall had been rebooted earlier in the day when there were internet issues outside my local network (both with my provider as well as with google overall and their DNS servers), on the WAN side, and the intermittent internet connectivity continued for many hours after the reboot. The internet has been fine for many hours now, but for whatever reason, UPnP wasn't functioning as expected with the patch. It was as if the patch wasn't working.

I don't know if a reboot resolved it alone, because I also reverted the patch and re-applied it before rebooting. But, the issue was resolved and UPnP would port forward normally and COD:Warzone showed as "open NAT" again.

It makes me wonder if there was some weird issue where the firewall got confused and stuck in a bad state in one or more of these scenarios:

• whenever there was a valid WAN IP, but no functional connectivity
• when one of the DNS servers was dysfunctional
• when it flip-flopped between 192.168.100.10 WAN IP (local, assigned by the modem when no IP can be retrieved due to internet connectivity issues) and and the real IP once it was retrieved and assigned, as internet connectivity went up and down over a period of many hours

Just wanted to share that if it's something else worth looking at.

[sclawrenc]

As soon as I saw this UPnP fix for pfsense, I knew that I wanted to try it out so I just installed a fresh copy of pfsense 2.6 on my HP 730t and applied the patch. After installing the patch and turning UPnP on, I rebooted the firewall and then started up my PS5 and Xbox X. My Xbox immediately returned an Open NAT which was awesome, but my PS5 is getting Moderate NAT (NAT Type 2). I have included some screenshots showing my configuration, and I'm curious if anything looks off? I have no manual Port Forward, 1:1 or Outbound rules set. I have also rebooted the firewall and PS5 several times with no luck. Thanks in advance!

[encrypt1d]

@sclawrenc

You may have to perform your test in the reverse order.

Another tester provided evidence in an earlier post that Sony is not respecting the UPnP protocol RFC. That evidence suggests that the PS tries for the same port indefinitely instead of trying new ones. It was around post 65 above https://forum.netgate.com/topic/169837/upnp-fix-for-multiple-clients-consoles-playing-the-same-game/65

Power down your consoles, then delete your Miniupnpd mappings (you can clear them from the status page). Power up the Playstation first. If it doesn't get Open NAT in this scenario, then there is some other issue. If it does have Open NAT, then power up the Xbox.

You may also want to ensure the following NAT settings are set:

[Jon8RFC]

@encrypt1d I'm not familiar, so I'm asking...
From your testing to solve the upnp problem, does pfsense/miniupnp just drop those packets or reject them?

Again, I may really be misunderstanding how a system requests or initiates sending data over a port. Even if Sony doesn't follow the RFC standards, could a workaround be rejecting packets for an occupied port rather than just dropping/discarding them? Would something like that signal devices like sony's to try another port...if the packets aren't already rejected by pfsense/upnp?