Using pfsense with multiple WANs

lewis · Mar 30, 2022, 8:47 PM

I thought I should come back and update this now that I have part of my solution. Doubt anyone will ever read to the end but if they do, they will find this.

It's not completed but it's allowed me to migrate everything as I needed and the last step will be a final configuration.

First, the DCLAN was disconnected and not used after all.
Both pfsense LAN interface were connected to the same DC LAN / VLAN.

Before doing this, I made sure all devices had unique IPs otherwise this would break things.
I then migrated all of the servers/services I needed to for now over to the new network (where you see the servers).

Almost nothing had to change other than putting the gateway into the DHCP config for servers that get DHCP IPs and hard coding the gw on those that don't. Almost all machines start off by getting a DHCP IP so I can get their MAC then usually just assign a permanent DHCP so I can keep track of assets.

What's nice about this is that I can just change the gw on a device and it then uses the pfx that I need it to.
Still have some problems but I think most of it is ARP caching that will eventually clear up.

Right now, this works for what I need. The final config will be along the same lines but working out any problems that would be found during this migration period.

I do not need all of the servers to be in the same network so even if some cannot reach others on the LAN side, it's fine.

stephenw10 · Mar 30, 2022, 9:14 PM

Nice.

Did you see any issues with servers needing to accept connections forwarded from both pfSense installs?

lewis · Apr 28, 2022, 8:41 PM

@stephenw10

Hi, no issues what so ever so far. It just ends up being maybe a bit non standard but it works perfectly well and is quite flexible in that I can add more networks in the future as well.

lewis · May 3, 2022, 7:43 PM

Ah it dawns on me there is one thing I should mention so someone else doesn't get caught.

ARP cache played a huge part in this and an accidentally left over rule in one of the firewalls as well.

The firewalls were basically competing to be the gateway so things would get weird like a vm would boot up with a gw then a while later change to another.

Once ARP cleared up, everything was fine.