Howto Multi WAN-Load Balancing and failover with pfsense 2.0-BETA4

daedalous

This guide was tested with pfSense-2.0-BETA4-20100905-1704.

1- Go to 'System->Routing' and create a new Group as shown in the
figures.
– For load balancing, select more than one gateway on the same tier (e.g. tier 1)
-- For failover, select gateways on different tiers (e.g. wan on tier 1, when down will fail to wan2 on tier 2)
Here we created 3 groups:

Wan1BalanceWan2, for load balancing WAN1 and WAN2
Wan1FailoverWan2, failover WAN1 | WAN2
Wan2FailoverWan1, failover WAN2 | WAN1

2- Go to 'System->Routing' and edit WAN and WAN2 gateways.
Fill Monit IP field with a reliable IP address beyond your WAN/WAN2 network that responds to ICMP pings.
The monitor IP address in the figure was taken from OpenDNS as example. Please, use your own IP addresses in your setup.
Here you can also customize advanced settings such as Weight (useful when balancing connections at different speed), Latency and Packet Loss thresholds, and Down time for the alarm to fire.

3- Go to 'Firewall->Rules' and set up rules to direct outbound traffic to your gateway groups, activating Load Balancing and Failover (see attached figures).

DNS and Load Balancing

Go to 'System->General Setup' and be sure that you have at least one DNS server for each ISP. This ensures that you will have DNS service in case one WAN connection goes down. You may need to setup static routes for each DNS server unless your DNS servers are WAN/WAN2 IP addresses.

Finally, you can check your gateway groups at 'Status->Gateways' page.

Credits and references:

Thanks to jimp for your comments and suggestions!

http://forum.pfsense.org/index.php/topic,27761.0.html
http://forum.pfsense.org/index.php/topic,27127.0.html
http://forum.pfsense.org/index.php/topic,27390.0.html
http://forum.pfsense.org/index.php/topic,27803.0.html
http://forum.pfsense.org/index.php/topic,28044.0.html

02-monitip.png_thumb

03-fw-rules.png_thumb

01a-loadbalance.png_thumb

01b-failover.png_thumb

01b-gw-groups.png_thumb

Schnulch

Hi,

and thanx a lot for your tutorial, it was a lot of help to me.

But I've got some suggestions to make:

With PFSENSE 2.0 there is no need for 3 gateway groups like in 1.2.3 anymore.

I set up a Outbound loadbalancer with 4 Wan interfaces, one cable 50/2,5 MBit, one ADSL 16/1 MBit, one ADSL 3MBit/384KBit and one ADSL 1MBit/128 KBit.
I only made one gatway group, where 50MBit and 16 MBit are Tier1, the other two (3 and 1Mbit) are Tier2.
Load Balancing works very well between first two lines, if one fails, the other gets all the traffic (Failover in Tier1). If both Tier1 are down, I get Loadbalancing on the Tier2 Interfaces. If one of them fails
too, i get failover in Tier 2.

And you have the benefit of only needing one firewall roule for load balance AND failover.

And, no more relevant with only one firewall rule and one gateway group, but anyway I want to mention that, your screenshot of firewall shows the failover rule before the loadbalancing rule. Since the first rule to match triggers the event of leaving firewall rules, you will only get failover with one line active with your ruleset. As any traffic from lan subnet will be redirected to failover gateway pool, which has two different Tiers, so as long as the Tier1 gateway is online, no traffic will be passed to Gateway with Tier2.

Sincerely
Schnulch

Schnulch

Hi again..

btw, I set up a second gateway group, since one of my 4 internet lines (the 16Mbit one) is no longer accepting smtp (Mail out), because a user had some viruses spamming the whole day, so i set up a group with the other 3 gatways, 50Mbit, 3Mbit and 1 Mbit as Tier1, Tier2 and Tier3, and added a firewall rule (before all others) to send pop and smtp (I know, pop would not be needed, but to avoid problems with mail clients useing pop before smtp and then coming to the mailserver with different source IP's) to this gateway group. So Email (on pop/smtp standard ports) always use the 50Mbit line, if it is down, the 3Mbit, and if all else fails, the 1 Mbit line.

i'm not sure, if your attempt with 3 gateway groups with same ruleset in firewall will work as you desired. If you have some spare time to check, i would appreciate your efforts.

Thanks,

Schnulch

daedalous

@Schnulch:

And, no more relevant with only one firewall rule and one gateway group, but anyway I want to mention that, your screenshot of firewall shows the failover rule before the loadbalancing rule. Since the first rule to match triggers the event of leaving firewall rules, you will only get failover with one line active with your ruleset. As any traffic from lan subnet will be redirected to failover gateway pool, which has two different Tiers, so as long as the Tier1 gateway is online, no traffic will be passed to Gateway with Tier2.

You're right, thanks for pointing out. The failover rule before loadbalancing forces the traffic to go always via the online gateway with more priority Tier. That kind of failover rule is generally used when we want some specific traffic to go via certain gateway, for example:


Proto  Source  	Port 	Dest  Port        Gateway            Description  	
TCP    LAN net    *      *     22 (SSH)   Wan1FailoverWan2   LAN -> WAN1|WAN2
*   	LAN net    *      *     *          Wan1BalanceWan2    LAN -> WAN1+WAN2

The above Firewall rules make all outbound SSH traffic to go via WAN1, and if it is down via WAN2. And load balance between WAN1 and WAN2 all else traffic.

daedalous

@Schnulch:

i'm not sure, if your attempt with 3 gateway groups with same ruleset in firewall will work as you desired. If you have some spare time to check, i would appreciate your efforts.

Yesterday I've configured a virtual network with pfsense pfSense-2.0-BETA4-20100905-1704(i386) and 2 routers. I created a Load Balance group (same tier). Then I applied the Firewall Load Balance rule and made sure that I had a DNS for each provider.

Load Balancing worked fine, but for whatever reason failover didn't work. The gateway was correctly marked as offline, but it didn't affect my pfsense routing tables…

Today, I repeated the tests with pfSense-2.0-BETA4-20100911-1502 (i386), but I got even worse results. Not only it didn't work, but the Gateway Status were always 'unknown'. Fortunately, I'm not using pfsense 2.0 in production environments. :)

As I wasn't able to make L.B. and failover work with just 2 gateways, obviously I didn't go for 3. I'll probably repeat the tests in a couple of weeks or so. Sorry.

demarchip

Hi guys… I tried to configure the gateways in failover mode, like daedalous, and don´t work.

-I configure 2 gateways (1 cable modem and 1 ADSL). I put the monitor IP and Advanced options (weight and Packet Loss thresholds).
-I make the group, Cable Tier1 and ADSL Tier2, tigger by packet loss.

-Put the Firewall-> Rule with advanced option Gateway and select the correct gateway group.

When I go to Status->Gateways menu I see the gateways status it´s Online.

But the status Gateways Group it´s Unknown

In the Status->System Log I can see:


Sep 12 21:58:52     php: : The gateway: Fiber_failover_Arnet is invalid/unkown not using it.
Sep 12 21:58:52     php: : The gateway: Fiber_failover_Arnet is invalid/unkown not using it.
Sep 12 21:58:52     php: : Gateways status could not be determined, considering all as up/active.

Iam using the latest version available online:
8.1-RELEASE FreeBSD 8.1-RELEASE #1: Sat Sep 11 15:29:22

This must be related to:
http://forum.pfsense.org/index.php/topic,28212.0.html

Patricio

daedalous

@daedalous:

Today, I repeated the tests with pfSense-2.0-BETA4-20100911-1502 (i386), but I got even worse results. Not only it didn't work, but the Gateway Status were always 'unknown'.

Same for me, demarchip, the devel team has made some change that affect gw groups, making impossible to get working load balance or failover.

ticket ~~#889~~, #876 explains the issue:
http://redmine.pfsense.org/issues/876

stramato

i'm using BETA 4 Sep 11 release now and Load Balancing seems not to work

jimp

Yes, it is broken at the moment, hopefully fixed soon.

jimp

What part of "it's broken at the moment" implied it should be fixed yet? :-)

jimp

I haven't had a chance to try it yet but there were some fixes checked in. If you update and then gitsync, and maybe reboot after that, it would be worth trying.

h20ray

hufft..
i still cant use load balance.. can you help me? sorry for my bad english

roi

@h20ray:

hufft..
i still cant use load balance.. can you help me? sorry for my bad english

The pictures don't load.
Try writing what have you triad to do…

h20ray

http://tujuhcahaya.com/oray/pfsense/1.jpg
http://tujuhcahaya.com/oray/pfsense/2.jpg
http://tujuhcahaya.com/oray/pfsense/3.jpg

that is the adress.. im sorry..

h20ray

http://tujuhcahaya.com/oray/pfsense/1.jpg
http://tujuhcahaya.com/oray/pfsense/2.jpg
http://tujuhcahaya.com/oray/pfsense/3.jpg

that is the adress.. im sorry..