How to disable loadbalancing and use failover on dual wan config?

V-man · Mar 25, 2008, 1:11 PM

Hi all!
I have two Internet connections coming in. One is Cable Modem and another one is T1. I set up basic dual wan configuration based on tutorial http://doc.pfsense.org/index.php/MultiWanVersion1.2. My WAN is Cable Modem connection and OPT1 is T1.

I notices that with load balancing in most of the cases all traffic is going through OPT1 -T1 interface, though i would like it to go through WAN- Cable Modem interface. Is there a way to set up pfSense just for fail over instead of load balancing? Or to have WAN interface act as my primary interface in load balancing mode.

Thanks.

Perry · Mar 25, 2008, 2:43 PM

Strange. First of all if your go to a site like www.myip.dk and refresh it changes ip every time.

Use WanFailsToWan2 as gateway to only use failover (as in the doc).

Post your rules if more help is needed.

V-man · Mar 25, 2008, 5:08 PM

Thanks for reply!!!

I changed LoadBlancer to:

WAN1FailsToWAN2 gateway(failover) opt1 66.111.111.11
wan 66.111.111.112 WAN1FailsToWAN2

I also updated my rules based on your suggestion (please correct me if I am wrong):

Proto Source Port Destination Port Gateway Schedule Description

* LAN net * 172.16.10.0/24 * 172.16.10.1

LAN net * 172.16.16.0/24 * 172.16.16.1
- * * * WAN1FailsToWAN2
- * * * 172.16.16.1

Now it looks like it is working. www.myip.dk shows ip of my Cable modem unless, cable modem connection is dropped and routed to T1.

It also looks like there is no fail back unless the connection with WAN- Cable Modem is restored and T1 connection is physically dropped.

PfSense forum http://forum.pfsense.org/index.php/topic,8290.0.html suggests that there is no fail back yet, unless you use work around from http://forum.pfsense.org/index.php/topic,7808.0.html. Is that true for my case?

Any way it works!!!
Thanks a lot Perry!!!
Thanks a lot pfSense team!!!

hoba · Mar 25, 2008, 5:18 PM

New connections will fail back but already established states through the second WAN will be kept there until the state expires or is closed which will happen sooner or later unless you are talking about services like SIP that try to keep the registration alive all the time.

Perry · Mar 25, 2008, 5:29 PM

If you want to connect to a ftp site like ftp://ftp4.freebsd.org/pub/FreeBSD it's most likely that a rule at the top is needed.

Outgoing FTP (LAN -> Internet) UPDATED PORTS, please check!

1. Ensure that the FTP helper is not disabled on Interfaces, LAN
2. If you have a restrictive ruleset or are utilizing policy based routing for multiple-wans then ensure that you have permitted traffic to 127.0.0.1 / ports 8000-8030. IE: allow LAN subnet to 127.0.0.1 8000-8030. This rule should be on top of all other LAN rules that utilize policy based routing.
3. If you are running windows try turning off the windows firewall

V-man · Mar 25, 2008, 6:58 PM

I just noticed that pfSence is switching back from WAN to OPT1. I disconnected OPT1 interface and it switched back to WAN in the matter of minutes it switched back to OPT1 again! It looks like OPT1 is default no matter what:)!

I can see it on http://www.myip.dk/ as well!

Am i doing something wrong?

hoba · Mar 25, 2008, 7:06 PM

What does your pool status report (status>loadbalancer)? In case WAN is down though it should not be down check your monitor IP for reliability.

V-man · Mar 25, 2008, 7:39 PM

I was all green I mean WAN is green and OPT1 is yellow and then it all switched to green. I trace routed google and found third hope router from the source. I ping -t that router and have not seen any dropped packets I changed my monitor on WAN to that one and it is still the same.

I also noticed that as soon as OPT1 switches to green ip on http://www.myip.dk/ changes to opt1.

hoba · Mar 25, 2008, 7:50 PM

Then better check your firewallrules. Maybe you are using some other pools for some special rules? There definately is no issue with this. I'm using it with a mix of failover and loadbalance rules at work with 3 wans. Recheck your configuration.

V-man · Mar 25, 2008, 7:58 PM

Can using Automatic outbound NAT rule generation (IPsec passthrough), or Sticky Connections be a problem?

hoba · Mar 25, 2008, 8:05 PM

Advanced outbound nat won't be a problem. I'm using it as well as my setup is using CARP VIPs. Not sure about sticky connections but that might be a problem with failoverpools. Disable and retest and report back please.

V-man · Mar 25, 2008, 8:13 PM

Disabled Sticky Connections, pulled out opt1 interface and rebooted pfSence with only one WAN connected to it. As soon as I plugged opt1 cable into nic it changed ip to OPT1.

I also don't have any other firewall rules but one i mentioned above!

Can it be faulty hardware? ( I am guessing not?!) I was thinking overnight that my pfSystem runs of Dell Poweredge 2600 with integrated 1GB nic card which I am using as WAN in and the rest of nic's are 100M Netgear. Can it possibly be that pfSence assigns priorities base on speeds?

hoba · Mar 25, 2008, 8:28 PM

I'm out of ideas for now :-\

V-man · Mar 26, 2008, 3:17 PM

Hi there!

I went further and switched wan interfaces, changed Failover WAN2FailsToWAN. It did not help. I got into the same thing!!!

So I went further and moved rules around. So, from

* LAN net * 172.16.10.0/24 * 172.16.10.1
* LAN net * 172.16.16.0/24 * 172.16.16.1
* * * * * WAN1FailsToWAN2
* * * * * 172.16.16.1

I changed it to:

* LAN net * 172.16.10.0/24 * 172.16.10.1
* LAN net * 172.16.16.0/24 * 172.16.16.1
* * * * * 72.16.16.1
* * * * * WAN2FailsToWAN1

WAN- T1 with 172.16.10.1 gateway
WAN2- Cable Modem with 172.16.16.1 gateway

So now my WAN2- Cable Modem Interface become primary, but as soon as it fails it will not switch to WAN- which is T1.

I am guessing that the problem has something to do with rules.

Can anyone explain what am I doing wrong??? ???

Thanks.

hoba · Mar 26, 2008, 3:33 PM

I still think this is a state issues. Does it work if you manually reset states after failover (diagnostics>states, reset states)? If so it's the effect that I described already above which is normal.

V-man · Mar 26, 2008, 4:33 PM

Well I rebooted pfSence. Then I double checked interfaces(ip's and gateways). Load balance showed everything in green. I verified that I been connected through cable modem IP. Then I pulled the plug off WAN(integrated GB NIC)- T1 and that was it. I can ping from WAN interface to T1 router, but I can not get internet to work.

Is not that strange? Before I could not get the Internet to work using GB nic as my Cable modem WAN and now I can not get to the Internet through the same GB nic. But in the second case GB nic serves as a T1 WAN(fail over lan)?

V-man · Mar 27, 2008, 4:09 PM

Thanks Hobo for helping out!!!

This is not the hardware! I just re-did the system on another pc. I setup pfsense on different Internet networks and set up worked.

Now I striped everything down to WAN/ Lan setup. I am having a problem even with trying to get out to the internet. I checked monitor, static ip on the cable modem and on pfsence interface. I enabled pass any from wan rule and could not ping Comcast wan interface.

Have you ever come a cross that Comcast had issues with their SMC router modems and pfSence?

Thanks again for helping out!!!

hoba · Mar 27, 2008, 6:00 PM

I have heard from a lot of people using comcast and pfSense together. One common issue seems to be that the cablemodems sometimes need a reboot if you connect a new device (like replacing an old router with pfSense) as thy seem to cache tho old macadress for ages in their ARP-cache. I also have heard from people where the nexthopgateway seemed to not clear the ARP-cache and they had tto take down the line for 10-20 minutes before a new pfSense install was working there.

V-man · Mar 27, 2008, 6:05 PM

Thanks Hobo for info!

I also was wondering if static ip and enabled firewall on Cable modem router may also cause problems?

Shell I use DHCP instead?

hoba · Mar 27, 2008, 6:19 PM

For sure I would shut down the firewal of the cable modem. Maybe that's exactly what's happening, your DHCP IP gets dropped and that's why traffic stops then.