@thefixersson btw: I forgot to mention that you should xxx-out your public IP. ifconfig: That looks indeed good, can't see anything wrong. Is the ISP gateway still in the ARP table? And the routing table is also ok? I was just on the phone with the ISP while it was down and they could ping my device They could ping your WAN IP on the pfSense? Unless you have created a firewall rule for it on WAN that's not possible.

@ncat I don't understand the use of gateway groups. Why not use a routing protocol like OSPF to select an active OpenVPN tunnel to send traffic over?

@SteveITS No gateway monitoring is active on the pfSense This is the routing table (in the file attachment) pfs-pmx-v4-routes.txt

@Bob.Dig Works as expected. Thank you, sir! (And thank you, @cmcdonald!!!)

FreeBSD 14.0-CURRENT #1 RELENG_2_7_0-n255866-686c8d3c1f0 I can see similar issues. Have two WAN interfaces in a gateway group with one on a lower tier. Trigger level is set to member down. Despite this, if I tcpdump both my main WAN and lower tier failover WAN, I see traffic on both while both interfaces are up and reporting no packet loss. Model is https://www.mini-box.com/APU-2E4-System?sc=8&category=2019 AFAIK.

@patient0 Hi and thanks for taking the time to respond. Yes, I have an x550 add in card. This morning I decided to factory reset and still could not get the WAN interface on LAGG0.4090 to get a public IP through DHCP. I had a suspicion that the x550 card, with the same hardware id's as the XG-7100 switch (ix0, ix1 SFP+ ports) was inducing a conflict. I put an older 4 port gigabit nic in it's place and now everything is working as I would expect it to. Fiber worked on this x550 card until I tried to use the built in LAN/WAN ports ETH1, ETH2. There may be another work around but for now I really have no benefit from 2 gigabit synchronous fiber, and plan to just drop it back to the gigabit tier. Again, thanks for pointing out the ix5 DHCPDISCOVER in the log. Looks like the LAGG0.4090 interface never bothered to reach out for an address and I suspect the issue was the x550 nic. Have a great day.

In my instance, monitoring a different IP instead of ISP didn't make a difference. I have disabled monitoring for now, we'll see if it helps.

@MyKroFt said in wan and gateways on different networks,: ip and gatewas had first 3 the same with 4 being different Well if your old IP started with 69 from your ddns info you posted. Seems like they changed the IP range you are connected too is all. As @patient0 pointed out your IP and its gateway are in the same network. 47.5.16.1 - 47.5.31.254 Doesn't really matter what the 2 IPs are, as long as they are in that range they are the same network. Your old gateway most likely answered ping, your new one in this 47 network does not it seems.

@ivo.a.v.tavares yes because the 192.168.4.0/24 is not in DMZ SUBNETS. Also any software firewall in the target needs to allow connections from the other subnet.

For anyone that has found this topic and dont want to create container or VM with tailscale running, Just swap your WAN1 and WAN2 in interfaces and restart tailscale on pfsense. You might need to adjust your default gateway in routing and any other rules that rely on that naming BUT it works. I was able to move my tailscale connection to my secondary WAN with this method.

@beloc I think you mean https://docs.netgate.com/pfsense/en/latest/routing/gateway-configure.html#:~:text=The%20amount%20of%20time%2C%20in%20milliseconds which is the averaging time period not necessarily a maximum. (?)

@SenseiNYC That's probably the core of your issue : @SenseiNYC said in pfsense UI hangs up and internet stops working until device is restarted.: it magically came back online We all presume that a "2.5 Gbit NIC" works like a 10,100 1000 NIC, but faster. When you run the numbers on a "2.5 Gbit NIC" (how it works, what it does, whats needed to make it works) you'll find situation where the electric pulses must be going faster as what we know is "the speed of light". So, special care is needed. Above 1 Gbit, the quality of NIC plugs (both sides), the cables used, and even the position of the moon right now become important factors. Ok, maybe not the moon. Btw : Yes, I saw 10 Gbit electric NICs working in front of me. I could use it in the kitchen and boil eggs on it. For me, it can't work, but it some how does ^^ error 64 : the interface used was 'not available', 'not ready' or 'out of sync' or more commonly said : it is 'down'. Check the NIC leds, get the doc of the NIC, check what the colors mean. Are the (very fast) flashing ? If so : sync errors => NIC is going up down very fast => it's having a hard time syncing. Keep in mind : I'm just thinking out loud here, except for the '64 error'.

Ok got it, on the allow ipv4 rule it was set to allow from port2 networks. My nested router isn't a port2 network so it would never be passed on and thus hit the default deny. Switched that to any source network as a test and it worked.

@wzkds See: Routing Public IP Addresses

@Jaritura I wonder if that really works. On WAN direction 'in' means connections from the public to the WAN. Your first rule keeps the state for all these connections. Have you implemented this and it works?

Really appreciate you circling back with the full explanation — this is extremely useful for anyone running multi-WAN with Starlink in the mix.