Layer 3 Switches, but VLAN routing through pfsense

rad · Mar 14, 2019, 3:38 AM

I have two layer 3 switches with VLAN enabled on both the switche and pfsense. pfsense seems to be controlling the traffic routed over the VLAN. Vmware environment powering all of this. Public webservers, private ip for database servers. If I traceroute from public webserver to public webserver, it's one hop to the next public ip. If I do the same for the database server, it first routes through the pfsense and then to the database private ip. It's introducing some added latency that I'm trying to avoid.

I feel like the VLAN should all be controlled by the layer 3 switches and not pfsense, but I'm not sure if that's wise or possible? Am I overthinking this, or is there a better way than how I have it setup now?

rad · Mar 14, 2019, 7:33 AM

If I add a local ip in the private range on the public server, it can try to access the other ip directly without hitting the router, but because the servers are on two separate VLAN they can't directly communicate.

The pfsense router is adding anywhere from .200-.400 ms according to mtr. I'd like to reduce that.

JKnott · Mar 14, 2019, 11:09 AM

A layer 3 switch is just a router built with custom hardware. It works the same way. What route is configured? If the devices have only a default route, then every packet that's not intended for the local LAN will be sent to the default router and forwarded appropriately.

rad · Mar 14, 2019, 5:01 PM

I have six different routes configured in pfsense, but I don't think these are relevant for the specific issue so maybe I need to look elsewhere? The firewalls have a different block of IP than the other blocks that are routed to devices behind the firewall. I think this could be solved if I tinker with the virtual host ip addresses and vlans.

marvosa · Mar 15, 2019, 1:38 AM

@rad said in Layer 3 Switches, but VLAN routing through pfsense:

I feel like the VLAN should all be controlled by the layer 3 switches and not pfsense, but I'm not sure if that's wise or possible? Am I overthinking this, or is there a better way than how I have it setup now?

Is it possible? Absolutely. That's exactly how my network is configured. Is it wise? That depends on where your concerns are leaning towards... security or performance.

From a performance perspective, you will get better performance out of your network by having your VLANs terminated on your L3 switches.

If you have a need to firewall your VLANs, then your VLANS need to be terminated on PFsense. However, now all inter-VLAN traffic has to traverse the firewall.

Personally, I always lean towards performance, but everyone's needs are different.

rad · Mar 15, 2019, 2:12 AM

I would agree with you on the performance side. I don't do this type of work on our infrastructure every day, and it's been over 5 years since I configured this. I'll need to do some digging to figure out the best way without disrupting traffic for an extended period of time. At least I have two switches and two firewalls...

marvosa · Mar 15, 2019, 5:27 AM

You will definitely want to do a fair amount of planning and schedule an after-hours change window because there will be some downtime. You'll need to configure a transit network, configure the SVI's on your switches, if dhcp was previously coming from PFsense... you'll need to figure out where your new DHCP server is going to live, add helper addresses to each SVI, change the default gateway for all of your dhcp scopes, change the default gateway for all static devices, etc.

There's quite a bit of work to do, but it will be worth it.