Sticky connections not working with dual WAN
-
Yeah, it's also https, but I've honestly had enough now. It's a simple enough workaround so I've done that and am moving on. Hopefully, they will fix it sometime.
Thank you for all your help, It's been greatly appreciated.
-
Hello,
I read all the posts with interest in hope that I'll solve the same problem that I have.
Unfortunately the end is clear: -Problem solved! It dosen't work. :) -
@curcanus Sorry to hear that.
Keep in mind that load balancing is a hack, and corner cases are very difficult to debug. Having said that, no one can assure you that there is no bug somewhere.
What I can say is that it works great for many, and when it doesn't post pile up.
If you still need some insight to the situation at hand, please fill in the details. -
@netblues either the pfsense documentation is wrong and stickiness is not per source and gateway or it's buggy. I've got the weights > 1, the default gateway is a Gateway Group, stickiness is for 1200 seconds and some machines still keep showing different IPv4 addresses within a few minutes when I visit https://www,whatismyip.com/ and https://whatismyipaddress.com/
Maybe most people encountering the problem don't report the bug and just resort to workarounds?
-
@bijore8887 No , this is not the case
Documentation IS correct
I did had the chance to check it recently (again)I would suggest to make a new lbgroup, with equal weights (since this makes stickiness more needed), and use policy routing on the lan interface to direct outbound traffic to the lbgroup.
This is just for testing. Sometimes defaultgw group is messed with floating rules, limiters etc and might be difficult to debug in real world situations.Now for testing, using whatismyip.com and whatismyipaddress.com is plainly wrong, since both of them are doing dns load balancing leading to wrong results.
And they also have cloudflare at front....
Who knows what else is happening at the backend.nslookup whatismyip.com Server: 185.12.64.1 Address: 185.12.64.1#53 Non-authoritative answer: Name: whatismyip.com Address: 104.21.89.158 Name: whatismyip.com Address: 172.67.189.152 Name: whatismyip.com Address: 2606:4700:3035::6815:599e Name: whatismyip.com Address: 2606:4700:3036::ac43:bd98 nslookup whatismyipaddress.com Server: 185.12.64.1 Address: 185.12.64.1#53 Non-authoritative answer: Name: whatismyipaddress.com Address: 104.16.155.36 Name: whatismyipaddress.com Address: 104.16.154.36 Name: whatismyipaddress.com Address: 2606:4700::6810:9b24 Name: whatismyipaddress.com Address: 2606:4700::6810:9a24 -
@netblues using whatismyip.com etc is not wrong if the documentation is correct that only the source IP and gateway matters (and thus the destination IP doesn't matter). In my case I have disabled IPv6 so it won't be trying IPv6 destinations.
Using whatismyip.com is wrong if the documentation is incorrect and the destination IP matters (you can test this by adding a hosts file entry so that the client uses a specific destination IP for whatismyip.com etc).
Anyway my workaround is to have multiple source IP ranges that default to different gateway groups that prefer different WAN links.
FWIW I've experienced this with 2.5.1 and 2.6.0
-
@bijore8887 Well, stickiness is about local ip to a specific destination ip. If the destination ip changes, then it will first load balance and THEN stick.
Of course if you are testing with hosts file, then dns round robin doesn't matter.I have experienced issues with load balancing and stickiness which I didn't dig into (and is most probably a config issue from my side) with default gateway group.
Since you have the setup of multiple source ip's to different gateway groups, enable stickiness, and make tiers equal on those groups and it will work.
It has been like that for quite some time with no issues.Latest test on 22.05 and 2.5.2 and it works as expected.
Stickiness is at 2500.Regards
-
-
if the documentation is correct and stickiness works, DNS round robin should NOT matter[1]! Since the internal source IP is not changing whatismyip etc should be seeing the same public source IP whichever destination IP you get. You should not be seeing different public source IPs every few minutes (but that's what I and presumably a few others are seeing).
-
With my workaround there should be no need to enable stickiness, since the firewall rules are mapping the different source ranges to the relevant WAN links (actually gateway groups in failover configuration, but keeping it simple for you). e.g. 10.0.0.0-31 -> WAN1, 10.0.0.32-63 -> WAN2 etc. It's ugly but this works and stickiness doesn't.
[1] Quote documentation: "When using sticky connections, the firewall remembers an association between the client IP address and a given gateway, it is not based off of the destination. When the sticky connections option is enabled, a single client would not load balance its connections between multiple WANs, but it would be associated with whichever gateway it happened to use for its first connection.
-
-
@bijore8887 Yes, you are right about the documentation, my bad for that.
However working with smaller values of stickiness (or the default 0) the gateway is switced everytime all connections complete.Do you have a high value in stickiness?
As for the stickiness test, another way to test is using ookla speedtest in multiple connection mode (the default)
If stickiness is off, you will get max speed on all links and speedtest will report the aggregate bandwidth.
If stickiness is on then traffic will saturate only one link.Try it out.
-
@netblues said in Sticky connections not working with dual WAN:
@bijore8887 Yes, you are right about the documentation, my bad for that.
However working with smaller values of stickiness (or the default 0) the gateway is switced everytime all connections complete.It would also be a bug if the gateway was switched every time all connections complete if stickiness was set to more than 0. Because it should be sticky for the number of seconds specified.
Do you have a high value in stickiness?
In my first comment I mentioned 1200.
Try it out.
No thanks. You have not bothered to read the pfSense documentation or the comments properly.
But at least others can now know that there could be a bug and use workarounds instead and not waste further time.
