Test Request: UPnP Fix for Multiple Consoles playing the same game / static port outbound NAT
-
That in itself isn't a
pfissue really. It's what happens when there is a port conflict.There can only be one NAT state for
<wan addr>:<source port>-><destination addr>:<destination port>. If there is already a state for that and something else comes along and tries to start another connection that would end up using the same external source port and destination. It can't create a state for the second one because it already exists, but the firewall rules have passed the traffic, so it leaves without NAT.What that does tell me is that in your case they are both matching some other outbound NAT rule that is set to use static port, not the rules you printed above, but it does give me an idea.
Try applying this change, then either reboot or do a filter reload then reset states:
diff --git a/src/etc/inc/filter.inc b/src/etc/inc/filter.inc index d36d6df2e2..5a7c21bc2a 100644 --- a/src/etc/inc/filter.inc +++ b/src/etc/inc/filter.inc @@ -2091,6 +2091,8 @@ function filter_nat_rules_generate() { $natrules = "no nat proto carp\n"; $natrules .= "no rdr proto carp\n"; + $natrules .= "binat-anchor \"miniupnpd\"\n"; + $natrules .= "nat-anchor \"miniupnpd\"\n"; $natrules .= "nat-anchor \"natearly/*\"\n"; $natrules .= "nat-anchor \"natrules/*\"\n\n"; -
@jimp
Very promising! I just tested, and I think that change will work for everyone, except me. I can see the packets getting translated now. So that might be the fix for everyone else, but sadly I cannot completely verify.I hadn't brought this up yet to avoid mixing too many issues. I am unfortunately behind a ISP that gives me a 10.x.x.x address on my firewall. I have read all about the changes made a while back in miniupnpd that broke double NAT. My ISP gives me a dedicated Public IP, and forwards everything fully with no translations other than the public IP to my 10.x WAN IP. (paid extra for that).
What I see now is it using the public IP instead of my WAN interface IP. I have the ext_ip=x.x.x.x line in my miniupnpnd config file, without which it will never even add nat/rdr pairs. I tried using STUN, but then miniupnp says it is impossible to forward under that config via a debug message.
So I think I have had 2 issues all along. The first being those missing lines from the filter.inc file, and now I've moved to the 10.x address on my WAN interface.
Thought I was almost there. Is there a way to get miniupnp to use 10.x addresses for the NAT rules? From what I've read, it seems like no, so I might just be SOL. If there is an easy answer to that, I'd appreciate it. If I need to open another discussion over at miniupnp I'll follow that.
Their default config has an option to revert the IGD behaviour:
https://github.com/miniupnp/miniupnp/blob/master/miniupnpd/miniupnpd.confforce_igd_desc_v1=yesAlas my version says this option is unrecognized. Not sure if it would even help.
I encourage everyone else behind a 'normal' ISP to give @jimp's change a try.
vi /etc/inc/filter.inc *Make the changes /etc/rc.filter_configureThen from the GUI, reset the states under the diagnostics menu.
-
You can install the System Patches package and then create an entry for the diff I posted to apply the fix without manually editing files.
That said, being behind double NAT will be a problem for anyone in the same situation. Do they maintain static port on your outbound NAT even? Seems like most CGNAT type places won't do that. Some time ago, miniupnpd decided that it wouldn't run with private addresses. I'm still not sure why they won't give us a way to override that, as it also makes testing it internally more difficult.
-
-
I am trying my luck on a pull request that a user was working on that was aimed at fixing this double NAT issue, I think.
https://github.com/miniupnp/miniupnp/pull/565
-
@encrypt1d said in Test Request: UPnP Fix for Multiple Consoles playing the same game / static port outbound NAT:
I am trying my luck on a pull request that a user was working on that was aimed at fixing this double NAT issue, I think.
https://github.com/miniupnp/miniupnp/pull/565
Hopefully they do pull in something like that but I can't believe they won't just add a knob to disable all those checks. There are plenty of people with private addresses on WAN that know full well what they're doing with upstream NAT and it would work if they'd stop forcing the current behavior.
-
@jimp
I couldn't agree more. Options options options.
Let the network admin configure it the way they want.
We'll see what comes of it. -
I've also added my findings to a note on Redmine and setup a merge request to get the change into development snapshots. It's too late for 22.01/2.6.0 but once we're done with the release work, we'll get this into dev snapshots for easier testing.
-
@jimp Thank you!! Fixed for me on my plain old ipv4 cable internet with PFSense.
I was able to disable every single nat/rule I had setup for one particular game, and both PC and PS5 displayed "open NAT", automatically. Reverted the patch, reloaded filters, reset firewall state, and went back to "strict NAT", applied patch again, reloaded, reset, back to "open NAT". I've been so accustomed to doing this, that I wonder how many additional rules I could potentially delete for which the software supports UPnP!
This will be so nice to delete the many things I have in there, making it easier to see what I have setup, not to mention no more frustrations from my nephews "how long is this going to take??" every time I have to reconfigure something so we can play games together online.
Thank you @encrypt1d and @jimp for getting this resolved!
-
@jon8rfc For me the old days of simple NAT rules were easier.
I have found this era of supposedly automatic port forwarding problematic.
Not to mention that when I looked up the documentation the thing apparently needs more than 10 ports open to make it all work on modern consoles, I dont know how it all got so messy.
I also had to disable upnp a few weeks back when I was getting broadcast storms from the internet taking down my lan entering via upnp.
-
@jon8rfc I definitely would not thank @jimp .
If anything; he just admitted it's not a UPNP issue and it's been an issue under their control the whole time for years on end without them doing anything.
(Even people have been complaining for years and he even asked me go to the UPNP dev years ago claiming it was a upnp issue; for apparently; no reason.)Happy you guys finally solved something a standard router out of the box could do since it's inception.
Most of us have moved on by now. -
@firetop said in Test Request: UPnP Fix for Multiple Consoles playing the same game / static port outbound NAT:
(Even people have been complaining for years and he even asked me go to the UPNP dev years ago claiming it was a upnp issue; for apparently; no reason.)
miniupnpdcould not add the NAT rules needed in pf until recently. That only changed in a release in the last year or so. We couldn't do anything until that was done. After that was in place, it wasn't until a day or two ago that someone with an appropriate setup and diagnostic skills was able to get us the information we needed to know what else might be wrong. -
@jimp said in Test Request: UPnP Fix for Multiple Consoles playing the same game / static port outbound NAT:
it wasn't until a day or two ago that someone with an appropriate setup and diagnostic skills was able to get us the information we needed to know what else might be wrong.
This must include yourselves then? I'll agree I'll thank @encrypt1d; but lets face it as I already said most of us have moved on to working solutions already without this being fixed.
The fact is this reeks of "idk what to do to fix it; so I won't do anything" from the PFSense team; and that's just the worst kind of thinking to have for a project like this. It's really not that hard to setup a few consoles or PCs behind a PFsense firewall to test this "appropriate setup". So the only thing left is from what you said is that the PFSense team didn't have the diagnostic skills and left it for years on end. (Sad)
Even if UPNP did need a change; the fact that you asked a community member to go do a report for the UPNP dev instead of you or your team when you had an open issue for 2 years prior with no action; is still a sad fact to this day; I hope this helps the PFsense team reflect on how they deal with issues like this in the future.
-
@firetop said in [Test Request: UPnP Fix for Multiple Consoles playing the same
This must include yourselves then? I'll agree I'll thank @encrypt1d; but lets face it as I already said most of us have moved on to working solutions already without this being fixed.
Reproducing and confirming things required two identical consoles (that support UPnP) running the exact same game (that also needs UPnP). None of the developers or TAC crew here have a setup like that.
As for how many people may have given up on pfSense in that time, it's unlikely to have been as many as you suggest.
The fact is this reeks of "idk what to do to fix it; so I won't do anything" from the PFSense team; and that's just the worst kind of thinking to have for a project like this. It's really not that hard to setup a few consoles or PCs behind a PFsense firewall to test this "appropriate setup". So the only thing left is from what you said is that the PFSense team didn't have the diagnostic skills and left it for years on end. (Sad)
It's more difficult than you think (and expensive) to have multiple identical consoles and games with active online support.
It wasn't "we don't know what to do" but "we need someone with access to an appropriate setup to get us more information". I mentioned the potential need for this change even before the miniupnpd NAT code was ready but again, without an appropriate setup we couldn't diagnose it internally.
Even if UPNP did need a change; the fact that you asked a community member to go do a report for the UPNP dev instead of you or your team when you had an open issue for 2 years prior with no action; is still a sad fact to this day; I hope this helps the PFsense team reflect on how they deal with issues like this in the future.
If Netgate asks them to do it, it has the appearance of one company wanting them to do work for free. If users request changes, they can see that it was a request from the wider community and would have more benefit than only helping Netgate/pfSense.
But no matter how we arrived here, what appears to be a solid fix is pending and what we need now is more testing. I'm going to make a fresh thread once the fix is merged in snapshots so people don't need to go through years of history and discussion to find it.
-
@jimp You can justify this how you'd like; I still think someone should be help accountable for how long this took and so far it seems to be the pfsense team. (The UPNP dev made his change within what? 2 days of his report?) But this has been opened on the PFsense side since 2017... Took 3 years before it even made it's way to UPNP through the community and not through yourselves, and then another year before you guys thought to bring it back up. Sure I'll admit you mentioned the potential need for this change; but why did it take a year for you to re-address it? Just because you didn't have 2 PCs or 2 Consoles with the same game? It's things like this I think the team may need to let sink in. There is more that could be been done to speed this up for sure.
-
If you want to blame someone, you're welcome to blame me, but that accomplishes exactly nothing and serves no purpose.
I've already explained multiple times why it wasn't possible or feasible for us to reproduce it here and we needed feedback from others, which was the original purpose of this thread, before it was derailed repeatedly by people complaining rather than helping or staying out of the way so people could offer productive feedback.
Rarely does this kind of situation happen because in most cases we can either reproduce a problem in lab conditions or at least see where in the code it might occur, but not here. We can't just make changes arbitrarily hoping it might help, we need some evidence that the change will correct the problem and not have a negative impact. Which we now have.
If you have moved on, then move on. There is no need to keep clogging the thread. I'm going to lock the thread and make another comment below with the patch and links to the fix. There will be a new thread up once there are images to test.
-
J jimp locked this topic on
-
Thread closed. New thread coming soon.
Summary:
Thanks to analysis by @encrypt1d we were able to determine the last piece of the puzzle to solve this.
Redmine Issue: https://redmine.pfsense.org/issues/7727
There were multiple components necessary here:
miniupnpdneeded the ability to add the correct outbound NAT rules corresponding to the ports it used for inbound port forwards- The firewall ruleset needed NAT anchors to ensure that the rules from UPnP would be matched before automatic outbound NAT or manual outbound NAT rules
The version of
miniupnpdin current releases of pfSense Plus and CE software adds the NAT rules, but a patch is required to setup the appropriate NAT anchors:diff --git a/src/etc/inc/filter.inc b/src/etc/inc/filter.inc index d36d6df2e2..5a7c21bc2a 100644 --- a/src/etc/inc/filter.inc +++ b/src/etc/inc/filter.inc @@ -2091,6 +2091,8 @@ function filter_nat_rules_generate() { $natrules = "no nat proto carp\n"; $natrules .= "no rdr proto carp\n"; + $natrules .= "binat-anchor \"miniupnpd\"\n"; + $natrules .= "nat-anchor \"miniupnpd\"\n"; $natrules .= "nat-anchor \"natearly/*\"\n"; $natrules .= "nat-anchor \"natrules/*\"\n\n";That patch can be applied using the System Patches package. Create a new entry and paste in the diff, save, then apply changes.
After applying the fix, either reboot the firewall OR trigger a filter reload (Status > Filter Reload) and then reset the state table (Diagnostics > States).
It's too late for this change to be included in pfSense Plus 22.01 or CE 2.6.0, but it will be in the next release.
The patch is also available on the Redmine issue.
I will create a new thread seeking feedback once the fix is merged and available in development snapshots. Those who have applied the fix on other versions and are testing it are also welcome to provide feedback once that thread is available.
Thanks!
-
E encrypt1d referenced this topic on
-
E encrypt1d referenced this topic on
-
J jimp unpinned this topic on
-