SG-5100 Multi-Wan Setup
-
Hi All,
I have an SG-5100 running 21.02-RELEASE and I am having an issue with setting up multiple WANs.I recently purchased a Starlink kit to act as a backup of my primary residential cable modem (just a modem).
I've setup my gateway groups and set the default gateway to the group I created:
However, if both WAN links are connected at the same time, one of them will go down:
The "public" IP of my secondary connection dies, and in the ARP table all I am seeing is a 192 address
If I shut off the primary and bounce the port on the secondary it gets an address and works fine:
Outbound NAT is set to Automatic outbound NAT rule generation.
I have no virtual IPs setup.
Looking at the logs when I have both enabled I see this:
What am I doing wrong?
-
@nickf1227 I'm having the same problem when I plug Starlink into the SG-1100. I followed a guide posted on Reddit but it is a no go. Did you ever solve this?
-
@chief764 I ended up virtualizing a second pfsense box to solve the problem. I created a L3 link between each of the two pfsense boxes. On the SG-5100 the second pfsense box is a secondary gateway. I think the problem was that both ISPs were DHCP and were advertising themselves as a route to 0.0.0.0/0.
You can then use the existing failover mechanisms in pfsense. If the internet connection dies on your default gateway, the failover algorithm will send the traffic through the first router and out to to the other router to get to the internet. If you are using static routing only, you will have WAN redundancy, but not firewall redundancy, and you will actually need two working firewalls to keep both connections usable. You will also need to make sure you properly write the firewall rules between each to allow for this.
I also implemented OSPF in conjunction with the failover settings in pfsense in my environment. I was super annoyed by this problem, so I went all in snd I'm also using dynamic routing with the frr package.
My internal network is area 0, the l3 link to my sg-5100 is area 1, the l3 link to my virtualized pfsense box is are 2 and the link between the two pfsense boxes is are 3.
This yielded the best results for me, and my internal network can route to either the virtualized box or the sg-5100 and out to either connection.
The extra connection between the two firewalls helps eliminate issues, so I wouldn't skip that step. When the SG-5100s connection to my cable ISP was down it was still advertising itself as the default route in OSPF sometimes. By introducing that extra l3 link in conjunction with the wan failover settings in pfsense, it can still be the default gateway if it wants to be. I also weighted the area links so that my network prefers the sg-5100 and will only route out of the other pfsense box if there is a failure. You don't have to do this, and you can dynamically route your traffic through both at the same time of you want. Just be careful with the asymmetric routing problems that may arise.
I hope that's helpful
-
@nickf1227 Thanks for your reply. You have gone above and beyond what I am capable of doing right now. My course of action will be to work with Netgate support and see if they can help solve it. If it works out I will be sure to report back. Thanks again for taking the time to respond to my post.
-
@nickf1227 Wanted to give you an update. I was able to resolve the issue by hard coding the gateway IP monitor address to 8.8.8.8. Starlink is working great on my SG-1100 now.