Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN
-
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
I disabled the DHCP server on the CL modem.
Ahhhhh....there it is. This makes sense now. Perfect! I'll give this a try tonight and see if I get the same result.
-
My guess is that pfsense is re-authenticating with C-Link every 15 minutes and something occurs to cause the issue at that time.
Though I am unsure why this hasn't come up before with other users trying to utilize similar setups.. I use Astound and Verizon here and have no issues. Neither of my ISPs use any kind of PPP.
-
Generally speaking, CenturyLink (now called Brightspeed) has been the worst ISP I have ever had. Until Starlink, they were the only option in my area.
That being said, it worked fine in Transparent Bridging for a long time. Not sure what changed, but it sure broke things. So far, so good. Things seem to be back to normal. Hope it works for you @jimeez.
Thank-you again!
-
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
Generally speaking, CenturyLink (now called Brightspeed) has been the worst ISP I have ever had. Until Starlink, they were the only option in my area.
That being said, it worked fine in Transparent Bridging for a long time. Not sure what changed, but it sure broke things.
This was my exact experience as well. Only option available until StarLink (I mean I choose to live in the middle of nowhere). Worked just fine in transparent bridge mode forever....and still does when it's the only active interface. But something changed on or about August 22, 2024.
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
Hope it works for you @jimeez.
I'm not quite there yet. Although it does seem promising. I spent more hours on this last night than I will admit and am still struggling with the CL modem settings. I have an old Protectli 4-port device with which I decided to start fresh. Got StarLink up and running no problem on the main WAN interface. Adding the CL interface is another story for some reason. I must be doing something wrong.
(i disconnected the StarLink interface while setting up the CL interface)
- I initially connected a laptop to the factory-reset CL modem via the WAN port (laptop to WAN port).
- After initial config of the CL modem (turn off WiFi etc.) I connected the pfSense device OPT1 interface to Port 1 on the CL modem and reserved an IP for it. In this case I was not able to use 192.168.0.2 because the laptop already took it, so I gave it 192.168.0.5.
- When the DHCP service is active both the laptop and pfSense see the modem and have internet.
- As soon as I disable the DHCP server on the CL modem I can no longer resolve DNS addresses. The laptop and pfSense devices both now show that they no longer have internet.
- I can ping actual IP addresses on both devices (like 8.8.8.8), but can't resolve addresses (say google.com).
Basically I'm stuck here. Grateful for any input on the likely obvious thing I'm doing wrong. ;-)
-
Short response right now is that this works for me too. Thank you so much!!
Will post back later with more detail.
-
@stephenw10 any comment from the balcony seats?
This seems to be reproducible but the particulars need to be understood a bit more I think.
-
A bit of bad news here. After about 24 hours, I lost the Centurylink connection. pfSense shows the Centurylink WAN as "pending" and will not reconnect. Restarted dpinger, rebooted pfSense, and it is still offline. I also have lost the ability to connect to the CenturyLink modem interface.
Perhaps disabling the DHCP server on the CL modem caused the lease to time out even though I assigned an IP address to the pfSense connection. I had to factory reset to get back to the CL interface.
I'm going to try it with the CenturyLink DHCP server enabled to see what happens. Back online now.
I'm going to play with lease times and see what happens.
EDIT: The lease expire time seemed to be the culprit. The default lease expire was 24 hours. I left the CenturyLink DHCP server enabled and changed the lease expire time to 5 minutes. It made it past the 5 minute mark.
More testing to come.
-
@preston Yes.. you have to set your pfsense CL WAN to static and use something like 192.168.0.5 as its address and 192.168.0.1 as its gateway.
-
@chpalmer said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
@preston Yes.. you have to set your pfsense CL WAN to static and use something like 192.168.0.5 as its address and 192.168.0.1 as its gateway.
I did try those settings in pfSense but couldn't get the CenturyLink WAN connection to show online. So, for now at least, I have CenturyLink as WAN2 with the IPv4 config type as DHCP.
-
@chpalmer said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
@preston Yes.. you have to set your pfsense CL WAN to static and use something like 192.168.0.5 as its address and 192.168.0.1 as its gateway.
I think in @preston 's and my setup it's OK to leave it as DHCP if we are reserving the address in the CL modem for the MAC address of the pfSense interface. It's worked for me so far. I have the DHCP service active on the CL modem, reserved an IP address of 192.168.0.2 for the pfSense MAC on WAN 2, and kept the pfSense CL WAN set to DHCP. So far so good. The connection has been solid other than the daily 4:00 AM EST brief down time.
The only problem that remains for me is that now I have CGNAT on both of my connections. Used to be able to use the CenturyLink connection for Dynamic DNS ....which gave me a remote gateway into my network via WireGaurd. So this is now toast as well as some other port forwarding like XBOX open NAT and a handful of others.
Would really LOVE to better understand what happened that caused the transparent bridge mode to just stop working after it worked for nearly two years.
-
@jimeez said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
Would really LOVE to better understand what happened that caused the transparent bridge mode to just stop working after it worked for nearly two years.
I agree. Things worked just fine for a long time on my end too.
-
@jimeez said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
The only problem that remains for me is that now I have CGNAT on both of my connections. Used to be able to use the CenturyLink connection for Dynamic DNS ....which gave me a remote gateway into my network via WireGaurd. So this is now toast as well as some other port forwarding like XBOX open NAT and a handful of others.
-
I have had great success with Tailscale to access my network while away. Its free and gets around CGNAT.
-
Not sure what DYN DNS service you are using, but I noticed that there are Dynamic DNS settings in my CL modem interface.
-
-
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
I have had great success with Tailscale to access my network while away. Its free and gets around CGNAT.
Are you running that on your pfSense device?
-
Yes, there is a Tailscale pfSense package. I am able to access the home (pfsense) network with my phone and laptop when I'm away.
-
P preston referenced this topic on
-
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
Yes, there is a Tailscale pfSense package. I am able to access the home (pfsense) network with my phone and laptop when I'm away.
Yep. Got that up and running no problem. I really like having it instralled on the pfSense device rather than a client machine like how I was using WireGuard (on an unRAID box). But I don't see how this is going to help get around the CGNAT specific to port forwarding for things like the XBOX and say a bittorrent client. I still cannot get an open NAT on the XBOX.
But anyway, I'm (mostly) very satisfied with this current solution. It's got me back to a solid stable dual WAN failover setup and has helped me iron out a couple other kinks in my network as I started fresh from scratch on a new device. Can't thank @chpalmer enough for his suggestion.
-
@jimeez said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
The only problem that remains for me is that now I have CGNAT on both of my connections. Used to be able to use the CenturyLink connection for Dynamic DNS ....which gave me a remote gateway into my network via WireGaurd. So this is now toast as well as some other port forwarding like XBOX open NAT and a handful of others.
I was thinking more about this issue today. Is your CenturyLink really a CGNAT connection?
Can you use a policy routing rule to send the device you want out through the CenturyLink WAN? For example, I set up a rule where my Synology NAS uses only the CenturyLink connection. I use Synology's DYNDNS service, opened a port on the CenturyLink WAN, and use that for an OpenVPN connection. Would something like that work for your setup?
-
@preston said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
Is your CenturyLink really a CGNAT connection?
You know, I'm not really sure anymore. I just spent some time reading up on it and running some tests. Apparently it's not. And it seams like the StarLink connection no longer is either.
I'm pulling a 98.97.xx.x IP address for the SL connection and a 75.165.xx.xxx IP address for the CL connection. pfSense sees them as 100.64.x.x and 192.168.0.1 respectively, but when I check my IP address on "what'smyIP" that's what I get. The XBox now shows open NAT to boot. So I'm not quite what I did (if anything) to fix this. But it's working now. Maybe the TailScale settings I applied did something? I also enabled UPnP & NAT-PMP.
Whatever happened, everything is back to normal. Better than normal actually.
-
@jimeez said in Dual WAN Fail-over Issue - Tier 1 WAN frequently failing upon activation of the second Tier 2 WAN:
I also enabled UPnP & NAT-PMP.Whatever happened, everything is back to normal. Better than normal actually.
Good deal. Just a guess but I would think that UPnP and/or NAT-PMP would help.
Thanks to you and @chpalmer for solving this issue!
