Balance/failover with 3 ISPs (DSL+T1+CableModem)
We have a client with 3 ISP connections to various providers (DSL+T1+CableModem), and they manually swap the ethernet cable to another ISP when the one currently in use goes down. They need high availability and as much bandwidth as possible. They want us to set up a system that will automatically balance their traffic over up/available ISP connections. Their Cablemodem service goes down 10+ times a day, but it has the highest bandwidth; and, when the cablemodem goes out, they want to notice it as little as possible. For hardware, I'm guessing a Soekris bundle with extra LAN ports will do nicely ( http://www.soekris.com/bundles.htm ). After setting up multiple WAN ports, how do I set up pfSense to handle this load-balance + auto-failover situation?
A soekris might reach the limit or is hard at the limit for this kind of setup (how much bandwidth do all these wans have together?).
However, everything you want to do is possible. Just keep in mind you need static gateway IPs for all the interfaces. This will work for DHCP for example if the gateway IP is always the same. This might not work for PPPoE unless you get a static IP assigned. There is a way to "fake" static IPs if needed. Just use a modemrouter in NAT mode with the pfSense WAN IP behind as DMZ IP. I'm using this kind of setup at our office to get an additional 6 mbit/s down using a dynamic ADSL line.
If you have managed to get 3 static gateways this way just set it up like described at http://www.netlife.co.za/content/view/34/34/ . You don't need the advanced outbound NAT entries btw unless you have to use advanced outbound nat for something else.
As monitoring IPs use something near you. In case you "fake" the static gateway with a modemrouter in front of you use the next hop. pfSense will ping the gateway every 5 seconds for availability and exclude the gateway it belongs to from the pool until it becomes available again.
If you use the DNS forwarder you should enter DNS Servers from different WANs at system>general and add static routes to the DNS servers via the appropriate interface. This way DNS will work if the original WAN is down too.
Some known limitations:
- Traffic shaping will only work at one WAN
- the ftp-helper can't use balancing so ftp-connections will only run at the original WAN
- you can't have different weights for your WANs (different speeds), it always will be simple roundrobin for new connections
Quite helpful hoba, thank you! I also noticed that HTTPS should be on one WAN only, and SMTP on the email server's WAN.
Is it possible to enter the fastest source ISP (cablemodem) twice in pfSense to simulate 2-1 load balancing?
From the frontend yes and it also is presented this way in the created ruleset, however the roundrobin still is used without weighting. We did some tests with that and billm did some debugging and knows why it is not working. This behavior can't be changed atm.
If you want HTTPS and SMTP on one WAN only make sure it is one of the reliable ones. We don't have interface failover (if WAN dies use WAN2 for example) in pfSense 1.0. This is something that already is worked on in HEAD. The poolfailover however will work.
There are also some (poorly written) webservices that won't work with plain HTTP or other protocols and multiwan. You should add an hosts alias for these destinations to add IPs to when you encounter problems and send these out one of the WANs only by creating a rule for this alias destination.