Having Issue with Dual/Triple WAN Failover
-
Hello All,
Before proceeding, let me first say that when we first installed our pfsense routers, our dual-wan failover worked as it should. That was running 2.1. I honestly didn't test it after that and our primary ISP never dropped, so I never made an attempt to test it again. Before proceeding I will say we're running a dual-router environment with CARP, but the CARP portion is currently disabled as the traffic shaping rules I have in place keep forcing the secondary router to reboot endlessly. This is a known bug that I guess is going to be addressed in 2.4.
Now running 2.3.2, I have swapped out original back-up wan and added a 3G cellular option. Both the back-up WAN and 3G Cell are DHCP.
When pulling the physical cable on our primary WAN out, pfsense detects that the gateway is down, but never switches ISPs. I first attempted to try this with my 3G back-up a month or so ago, but it wouldn't work (I also tested my existing back-up WAN, and it also didn't work). Wanting to get a better back-up ISP, I waited until that was installed to test that but it's giving the same results as the 3G. I'm assuming there's something wrong with my configuration, so I ditched the old gateways and gateway groups I had and re-made them, but no dice.
I've attempted pinging out of my secondary WAN as well as the 3G connection and I get responses from both, so I know they're up.
Alternate Configurations I've attempted that didn't work:
- Manually assign 3G or WAN2 as primary gateways.*
- Made 3G a Tier 2 option.
- Removed WAN2 as an option in Gateway Group (leaving WAN and 3G)
- Removed 3G as an option in Gateway Group (leaving WAN and WAN2)
- Enabled "Enable default gateway switching" under Advanced > Misc.
- Disabled "Enable default gateway switching" under Advanced > Misc.
- Set DNS Server Gateways to individual interfaces.
- Set DNS server gateways to "none" for all.
- Disabled "Allow DNS server list to be overridden by DHCP/PPP on WAN
- Enabled "Allow DNS server list to be overridden by DHCP/PPP on WAN
-
- This one confused me, as feel like I should have been able to use either 3G or WAN2 as my primary internet connection, but this didn't work. It's almost like there's something that's allowing me to only use the primary WAN as internet?
I'm attaching some screenshots of my current configurations. I've tried following a few YouTube tutorials (https://www.youtube.com/watch?v=O0e13_q-ImY this in particular), which makes it look like I've done everything I need to, but no dice.
I'm hoping this is just something stupid I'm missing, but I'm honestly at a loss for what it's going to take to get this pfsense router to failover at this point. I can only test this on weekends, so I'm hoping by next weekend I'll be able to try out some other configurations.
Thank you in advance for any help any of you can offer!
-
Hello,
I saw the youtube video and assuming you were following the guide. I also noticed that you are using default gateway switching options. I wouldn't recommend to do it since it is not controllable from my experience.
Here is what you have to do and hopefully it will fix the issue.
On the routing > Gateways, you need to edit the WAN/3G/or whatever interface to have monitor IP. In my case, I have 4 ISPs (no joking, that's how NOT reliable my internet here) connecting to my pfsense box. Each of them I set monitor IP to different public IP. For instance:
ISP1 monitor to 8.8.8.8
ISP2 monitor to 8.8.4.4
ISP3 monitor to 4.2.2.1
ISP4 monitor to 4.2.2.2You also need to specify the latency thresholds, packet loss thresholds, probe interval and loss interval at a very basic to determine when to trigger the fail-over. Remember, these settings will have to be configured on each ISP interface.
You also missing one big point on getting the fail over to work.
Under firewall > rules > LAN, you need edit the existing rule called "Default allow LAN to any rule". Click on show advanced options and scroll down until you find gateway. Select the gateway group you created before then hit save and applied. You can test the fail over. It should be working.
That's all I got for you.
Thank you
-
ccmks, thank you so much for your reply! I thought for sure the trick was going to be the firewall LAN rule, but I modified the gateway to my gateway group and it still isn't switching. I know the router sees the 3G connection as active as the update checker on the main dashboard can always check for updates, but none of the devices connected to the router wants to switch over to the back-up ISP.
I know this worked way back on 2.1, so I'm really just baffled as to what I'm missing.
Again, thank you for your reply on this! At least I feel like I'm potentially getting closer to figuring out what I'm missing.
-
ccmks, thank you so much for your reply! I thought for sure the trick was going to be the firewall LAN rule, but I modified the gateway to my gateway group and it still isn't switching. I know the router sees the 3G connection as active as the update checker on the main dashboard can always check for updates, but none of the devices connected to the router wants to switch over to the back-up ISP.
I know this worked way back on 2.1, so I'm really just baffled as to what I'm missing.
Again, thank you for your reply on this! At least I feel like I'm potentially getting closer to figuring out what I'm missing.
Did you setup the gateway monitoring like I mentioned on previous post? You need to have ways for pfSense to know when the gateway will be considered down. Otherwise, it won't do the switch if the pfSense still see the gateway online.
-
Currently having the same issue on 2.3.2
Our 4G gateway shows as active and online and I can ping the ISP DNS server through that gateway (DHCP). After the gateway goes down and comes back up, it gets a new IP from the ISP and shows as down under Status > Gateways
The ISP DNS is still pingable but the failover is not working.The failover is determined by Probe Interval.
We had our failover working in 2.1.3 but the same setting no longer work.
-
ccmks, thank you so much for your reply! I thought for sure the trick was going to be the firewall LAN rule, but I modified the gateway to my gateway group and it still isn't switching. I know the router sees the 3G connection as active as the update checker on the main dashboard can always check for updates, but none of the devices connected to the router wants to switch over to the back-up ISP.
I know this worked way back on 2.1, so I'm really just baffled as to what I'm missing.
Again, thank you for your reply on this! At least I feel like I'm potentially getting closer to figuring out what I'm missing.
Did you setup the gateway monitoring like I mentioned on previous post? You need to have ways for pfSense to know when the gateway will be considered down. Otherwise, it won't do the switch if the pfSense still see the gateway online.
I hadn't before and I just now got a chance to try it again this weekend. I had left them blank previously (there was a note that it defaulted to a certain value, so I assumed that was good enough), but I put in actual values this time. Still no change on my end - when pulling the plug on WAN1, neither my 3G nor my satellite back-up fail into its place. Again, I know the router itself is using the internet from one of those two other ISPs, as it's able to still check if it's on the latest version of software.
Something else I noticed: as soon as I plug my WAN back in (even while it still shows the status at Offline or Packetloss within pfSense), I can ping google.com again from my desktop. To me that tells me pfSense isn't even switching gateways on its end, otherwise there should be a delay before I start receiving responses again.
If there are any other screenshots or bits of information I can share (and you're still willing to help), please let me know.
And thank you again for taking time out of your day to help me with this! I greatly appreciate the help you've given me thus far.
Currently having the same issue on 2.3.2
Our 4G gateway shows as active and online and I can ping the ISP DNS server through that gateway (DHCP). After the gateway goes down and comes back up, it gets a new IP from the ISP and shows as down under Status > Gateways
The ISP DNS is still pingable but the failover is not working.The failover is determined by Probe Interval.
We had our failover working in 2.1.3 but the same setting no longer work.
Sounds similar to what I'm seeing. It once worked, but I can't get it to go now. I'm assuming you did auto-upgrades from 2.1.3 to current? I know that's how I've upgraded. I'm wondering if I need to just purge config and start clean. I have a spare router, may try doing that one weekend to see.