Need pf-feature: sticky-address

tmueko

4-Port (Lex-Neo) Router:
rl0 = LAN
rl1 = WAN (2MBit SDSL)
rl2 = DMZ
rl3 = LoadBalancer on 6xADSL: xx.1 xx.2 xx.3 xx.4 xx.5 xx.6

I use a Rule to redirekt traffic out to the loadbalancer something like

pass in an <lan> route-to { <lb-ipaddresses> } round-robin inet … from <lan> to  port = $lb-ports</lan></lb-ipaddresses></lan>

Some Websites (mostly php-sites) where i have to login, login don't work. I guess it's because of my IP-Address changing.
When i change the lines /tmp/rules.debug to
… round-robin sticky-address inet ...
It works.

Simple Patch would be


–- ./etc/inc/filter.inc.orig   Tue Mar 13 10:48:19 2007
+++ ./etc/inc/filter.inc        Tue Mar 13 10:50:53 2007
@@ -1485,7 +1485,7 @@
                                                                $routeto .=  "( {$int} {$gateway} ) ";
                                                                $foundlb = 1;
                                                        }
-                                                       $routeto .= "} round-robin ";
+                                                       $routeto .= "} round-robin sticky-address ";
                                                }
                                        }
                                        /* Add the load balanced gateways */

tmueko

sullrich

Please add a GUI tunable checkbox where you can turn on and off this behavior and we will commit.

We don't want to change the defaults for everyone, this needs to be an option.

hoba

We have this option as a general option for all rules in head at system>advanced iirc. An implementation per rule would be nicer though imo.

sai

Sticky ensures that a connection always uses the same interface? Isn't this usually better than having packets go out randomly?

hoba

It depends on what kind of traffic you need or how your application is working. It prevents a single client from using all available WANs as it maps this traffic then to a single connection only. This might be not a bad idea for a setup with lots of clients behind but if you just have a single machine and want to download from several wans at the same time sticky could prevent you from doing so.

tmueko

@hoba: what is the name of the configuration with this sticky thing (I guess it's for the incomming LB slbd?)?

I think, when this feature is on, we can turn it on on the outgoing lb too?!

should be something like#

if (!isset ($config['system']['sticky_lb'])) {
    $routeto .= "sticky-address ";
}

hoba

Don't know about the backend implementation but I have seen a checkbox for this in the head code tree. Probably have a look at the sourcecode: http://pfsense.com/cgi-bin/cvsweb.cgi/pfSense/usr/local/www/system_advanced.php?rev=1.200;content-type=text%2Fplain (search for sticky).

Not sure if this will easily work with the current releng1 code or what else needs to be modified.

ivan

I would certainly like this feature, since this is the main reason I do not use Load Balancing on our network.
Instead, I use a very primitive kind of load balancing (through policy based routing): 10.1.1.1/24 uses WAN1, 10.1.1.2/24 uses WAN2…

tmueko

OK, patch should be:


–- ./etc/inc/filter.inc.orig   Mon Mar 19 09:24:56 2007
+++ ./etc/inc/filter.inc        Mon Mar 19 09:31:09 2007
@@ -1389,7 +1389,10 @@
                                                                 }
                                                                 $foundlb = 1;
                                                         }
-                                                        $routeto .= "} round-robin ";
+                                                       $routeto .= "} round-robin ";
+                                                       if ($config['system']['lb_use_sticky']) {
+                                                               $routeto .= "sticky-address ";
+                                                       }
                                                }
                                        }
                                        /* Add the load balanced gateways */

right?

tmueko

techatdd

Dont know if this Information is usable for somebody:

using latest snapshots (1.2beta1 05-04 and 05-06) using gateway loadbalancing with a pppoe-wan and opt interface with "old style" config (gatewayip|monitorip) all works fine,
but after hacking filter.inc with

- $routeto .= "} round-robin ";
+ $routeto .= "} round-robin sticky-address ";

I get erros like

May 7 04:21:11 kernel: arpresolve: can't allocate route for 82.119.162.241
May 7 04:21:11 kernel: arplookup 82.119.162.241 failed: could not allocate llinfo

and loadbalancing stop working proberly. furthermore it only working sometimes.
revert the filter.inc change and everything works right.
strange.

hoba

Seth just fixed some PPPoE related multiwan bugs last night. Please try with a newer version. The "old hack" to create pools when a PPPoE WAN is in use should not be necessary anymore.

techatdd

I can confirm that it work with the current snapshot, no more "old style" config.xml changes needed.

But the Problem with sticky-address persists. after cahnging filter.inc errors like
kernel: arpresolve: can't allocate route for 82.119.162.241
kernel: arplookup 82.119.162.241 failed: could not allocate llinfo
are shown in the log and loadbalancing problems starts. most connection attemps fails from now.

Revert filter.inc and everything works as aspectef, but I have issues with some webstites (mosty php sites with password logins)

Any hints?

techatdd

@techatdd:

I can confirm that it work with the current snapshot, no more "old style" config.xml changes needed.

But the Problem with sticky-address persists. after cahnging filter.inc errors like
kernel: arpresolve: can't allocate route for 82.119.162.241
kernel: arplookup 82.119.162.241 failed: could not allocate llinfo
are shown in the log and loadbalancing problems starts. most connection attemps fails from now.

Revert filter.inc and everything works as aspected but I have issues with some webstites (mosty php sites with password logins)

Any hints?

Same problem with the new sticky connection option in the Advanced tab.

Pootle

How's sticky doing?
Also can I check my understanding: the words on the System - Advanced under load balancing I find slightly confusing ???.

I think the terms to use should be:
source: a PC or server (this matches the use on screen)
web server: an internet web server such as www.bbc.co.uk (on screen this term seems to refer to the WAN connection used)
WAN connection: the port on pfSense that will be used for the packet in question.

My understanding is that sticky means:

New connections are directed to web servers using the WAN connections in a round robin manner. The firewall remembers the source IP, the destination IP and WAN connection used as a 'sticky connection'. Subsequent requests from the same source to the same destination will be routed down the originally selected WAN connection. Once the 'sticky connection' expires a further request will once again use the round robin mechanism to select the WAN connection to be used. This is useful for web sites that get confused where different requests from the same user arrive from different source IPs (as seen be the web site) - frequently true for secure web sites (those using https).

Is this right? This will make some of the rules required for load balancing redundant (that's nice!), but probably only makes sense where the WAN connections are at least DSL speed, or there are always going to be several systems (users) accessing the internet at the same time.

I'll put info on this into the new version of MultiWAN doc that I'm working on.. ;D

techatdd

I think your are right with this. But "sticky connection" will break LB with PPPoE WAN atm.