@jecker Can you show your gateway groups? And example rules?

https://docs.netgate.com/pfsense/en/latest/multiwan/load-balance-and-failover.html

@viragomann
I think "skip rules, when gateway is down" in system/Advanced/Miscellaneous that you mentioned is the point that i didn't know.
Thank you so much.

@rbthomas
Don't know about step by step instructions, but Netgate documentation is pretty good.
https://docs.netgate.com/pfsense/multiwan/load-balance-and-failover.html

@Gertjan You are right, the ipv6 adoption here is pitiful. I just wanted to try my hand on it too see what I can do with it. But, it breaks my running system so I will stick with v4 for now.

@dcreations you can use FRR package, OSPF and BFD.
BFD with default settings, will try a hello packet every 50ms, and if it looses 3 packets, will switch the traffic to the backup path.
You can also set one side to administratively down, and by doing that, you don't need to change the cost at the other side to shift the traffic.

I know the SG2100 only supports the older SIM cards

@marcosm said in Failback state killing with "Automatic" failover?:

When you use an IP for gateway monitoring, a route is created for it via the gateway.

Got it. I switched the secondary monitoring address to 9.9.9.9 since I don't use Quad9 for DNS resolution. The extra states on the secondary, while the primary is up, disappeared. Thanks!

[24.03-RELEASE][admin@pfSense.home.arpa]/root: pfctl -i igc1.95 -s state igc1.95 icmp 192.168.95.2:24256 -> 9.9.9.9:24256 0:0

@not-a-bot2024
Why do you bother with the ISP DNS if it doesn't work reliably?
The DNS Resolver on pfSense requests the DNS root servers directly, unless you're using the forwarding mode.

@johnpoz Thank you!

@Stee7ic So you have double NAT situation at all your sites?
As in Public IP -> Modem -> 192.168.100.1 -> pfsense -> LAN IP
So I'm assuming when you say pfsense is 10.120.10.254, that is the LAN IP?

It shouldn't matter what the pfsense WAN IP happens to be, which would be unique for each site as well (at least the public IP).

I'm assuming with double NAT that the modems are set up to do port forward of ports 500, 4500 or whatever you use for IPSec?

@samweli

https://docs.netgate.com/pfsense/en/latest/multiwan/load-balance-and-failover.html
https://docs.netgate.com/pfsense/en/latest/routing/gateway-groups.html

@byusinger84 But the fact that pfsense may not like have multiple WANs going to the same gateway shouldn't have anything to do with the ISP not seeing the individual MACs.

How do you connect the ports towards the ISP?

I just placed a managed switch in between but I suppose any dumb switch would do. Which in fact is what the other thread had... And in both cases all the IP's are DHCP, although mine never change and my ISP needed to register the MACs...

@chpalmer said in Multiple WAN with Static IPs DHCP assigned from ISP:

This would only be true if they were trying to utilize multiWAN failover or load balancing..
1:1 NAT should work fine, which I believe is the intent here.

Ok, so then there are no other steps necessary than getting things upgraded to 10Gig...

@viragomann thank you ... greatly appreciated!

The initial configuration will be basic. Simple failover and no policy routing.

@mdonner Perhaps some pictures of your setup might help, from the Routing / Gateways and Gateway Groups pages.

Pfsense isn't really involved in traffic between devices on your LAN, so it's really strange that you would see this happening?!
Apparently you are able to access pfsense UI from a client, so do you have internet access when on the failover connection? And do the servers have internet access?

Are you using VLAN's to separate your servers from your clients??