Limiter source mask now after NAT when using gateway groups - 2.8 change?
-
Basic overview of setup:
Multiple WAN connections in a load balance gateway group
LAN with private address range
Limiter with source mask (e.g. /32 for per device) configured for upload
LAN rule to apply limiter to uploadOld behavior - I would see limiter per /32 private address range range
New behavior - I'm getting a limiter for each WAN IPIf I remove the gateway group from the LAN rule and use default routing, it works as I'd expect.
So I think the source mask for the limiter is now determined after outbound NAT has been applied. Which means I can't have a limiter per device/subnet at the moment.
Download works fine (direction 'out' of the interface the rule is on)
-
Think I found someone reporting the same in a big, so I've appended my description there.
-
@Konan-0 , Hi friend , I have the same problem , When I use WAN 2, the input limiter does not use the private IP, the public IP as the source. This confirms that the limiter is applied post-NAT. The limiter used on the default gateway does work correctly.
Do you have any new information? I'm going to test with versions prior to 2.8, which I'm currently using.
-
I was testing versions 2.6.0, 2.7.0, 2.7.1, 2.7.2, and 2.8.0 on new virtual machine with the same network configuration of the image.
In version 2.6.0, the limiters worked fine when applied to any WAN. In versions 2.7.0, 2.7.1, 2.7.2, and 2.8.0, the limiters use the post-NAT IP address. This divides the bandwidth and doesn't limit each connection from the LAN to upload.
The rules were never modified between versions.
limiter on 2.6.0 not default gateway
limiter on 2.7.x to 2.8.0 not default gateway
It is not clear to me if the problem is the version of pfsense or freeBSD and the PF package
-
G gemg83 referenced this topic on
-
@gemg83 Hi,
I'm afraid I'm no further with it. Honestly, I'm starting to wonder about the state of PFSense. That or I've massively misunderstood how they deal with feedback and bugs (or the right place to go).
I added a description to this bug as it sounds to be the same thing:
https://redmine.pfsense.org/issues/15770
I've had a bug that's similar open (although much more niche, it's how the limiters apply when going through an OpenVPN based interface) for a year with no response.
If you look in the limiters section of their issue tracker, there's confirmed bugs 11 years old there and 'new' issues that are now 7 years old.
-
@gemg83 I see what you're saying - it could be the jump from 12.3 to 14 on the BSD side.
It really hampers the use of limiters in multi-WAN setups so it feels like an important bug (I call it a bug as it doesn't behave at all how the UI or documentation suggests, it's more like using them on a floating rule).
-
K Konan 0 referenced this topic
-
S stephenw10 referenced this topic
-
On the bug report the reporter claims it works correctly in 2.7.1 but doesn't show which version they thought was affected.
However the 2.7.2 release was specifically to address a regression in the route-to behaviour that would be used by this. So I could believe it was that.
But you are now also seeing the issue in 2.7.0 and 2.7.1?
-
@stephenw10 , In versions 2.7.x and 2.8, the problem with limiters on a WAN that isn't the default route occurs. The last version that worked correctly was 2.6.0.
The evidence and tests performed in each version are documented. Thank you very much and I hope you can validate from version 2.7.x onwards that the limiters no longer work in a WAN that is not the default .
thanks.
In 2.6.0 the limiter uses the private IP as source and destination, to control the BW for each IP
In 2.8 and 2.7.x the limiter uses the public IP as the source and the private IP as the destination, that is, for the upload it uses the public IP after applying NAT, this does not limit each connection from the LAN, it limits the entire bandwidth
-
@stephenw10 I'm happy to configure this is a virtual and step through versions to find out where the behavior change happens, if it helps. Sounds like @gemg83 has got it dialed down though.
-
Yup I just to replicate it here.
-
@stephenw10 Please tell us what other information you need. I still have the test firewall with snapshots in versions 2.6.0, 2.7.x, and 2.8.0. If I can help with more information, please count on me.
I think the test cases are well documented with all the evidence in this chain, but I'll be on the lookout if I can help with anything else. Thanks.
-
OK we replicated this and are digging...
-
I'm not sure if it's helpful or not to link these two - but I've also noticed that if sticky connections are enabled, the source tracking table remains empty. Which I think is described here:
https://forum.netgate.com/topic/197911/pfsense-2-8-0-sticky-connections-in-dual-wan-setup-not-maintaining-source-tracking
I felt it worth a mention as it relates to gateway groups and source IPs, but I don't know if they use the same mechanism underneath (the above post also appears to be linked to 2.8, where it appears this one was also present in 2.7.x).
-
I don't believe it is the same root cause. That is fixed in 2.8.1-beta. https://redmine.pfsense.org/issues/16282