Setting up pfSense and L3 switch

Wholelottapfsense · May 18, 2019, 4:25 PM

Hi,

I'm a newbie in networking and the learning curve has been steep but I find it very interesting and I am eager to learn. This is my first post and I've already been crawling through many threads on this forum to learn about setting up pfSense but I've got still some questions

I’d like to set up a network as illustrated below:

The Netgear L3 switch would be the downstream router, routing between VLANs and acting as DHCP server for the VLANs:
VLAN 10: 192.168.10.0/24; SVI 192.168.10.254; (Win Server 2012, domain joined clients, trusted mobile devices)
VLAN 20: 192.168.20.0/24; SVI 192.168.20.254; (VoIP)
VLAN 30: 192.168.30.0/24; SVI 192.168.30.254; (IP cameras)
VLAN 40: 192.168.40.0/24; SVI 192.168.40.254; (IoT)
etc.

What I would like to confirm/ask is:

To create the transit network, I would:
In the pfSense box: set the LAN interface to 172.168.0.1/30
In the Netgear L3 switch: set port 1/0/27 to 172.168.0.2/30 via 'Routing > IP > Advanced > IP Interface Configuration' (p301 of http://www.downloads.netgear.com/files/GDC/M4300/M4300_M4300-96X_UM_EN.pdf).

I guess I should enable ‘routing mode’ for port 1/0/27 (see p301 of manual)?

Is it necessary to create a VLAN for the transit network?

I should create a gateway in pfSense (System > Routing > Gateways) as follows:
Interface: select ‘LAN’
Check ‘Default gateway’
Gateway: 172.168.0.2 (L3 switch)
to allow internet access for the VLANs, I should set up a static route in pfSense (System > Routing > Static Routes) as follows:
Network: 192.168.0.0/16
Gateway: 172.168.0.2 (= L3 switch)
I should set a default route in the L3 switch as follows:
Network address: 0.0.0.0
Subnet mask: 0.0.0.0
Next hop address: 172.168.0.1 (= pfSense box)
Is this sufficient for the L3 switch to know that it should route internet traffic via its 1/0/27 port to pfSense ?
I would create a management VLAN 50: 192.168.50.248/29 and set management IP addresses e.g.:
pfSense: 192.168.50.249
L3 switch: 192.168.50.250
L2 switch: 192.168.50.251
Ubiquiti AC AP Pro 1: 192.168.50.252 (connected to L3 switch via trunk port)
Ubiquiti AC AP Pro 2: 192.168.50.253 (connected to L3 switch via trunk port)

Should I put the management VLAN as:

a tagged VLAN on the 10 Gbit uplink between the pfSense and the L3 switch
or
an untagged VLAN between a 1 Gbit port of the pfSense and a 1 Gbit port of the L3 switch?

I assume this would be the only VLAN to be created in the pfSense box (unless the transit network needs a VLAN (see question above))?

Do I need to configure something in pfSense for NAT ?
When configuring DHCP in the L3 switch for the different VLANs, I assume I can set pfSense (172.168.0.1) as the DNS server (I understood one sets public DNS servers during the initial configuration of pfSense) and the respective VLAN SVIs as the gateway?

Did I forget anything (apart from setting ACLs in the L3 switch) ?

Wholelottapfsense · May 22, 2019, 11:56 AM

Nobody?

Either I wrote total rubbish or it all looks quite allright ...

Just want to avoid nasty surprises because my family members are allergic to prolonged internet outages

bmeeks · May 22, 2019, 1:13 PM

You have a needlessly complex design for a home network. Drop the transit network and L3 switch entirely. What's the point of doing that? pfSense can easily route and take care of tagged VLANs. If you have extra NIC ports on the pfSense box you can put the different VLANs each on their own physical interface, or you can use trunk ports.

You are way over complicating this. If you want to experiment to see how hard you can make something, then go ahead and have fun. But if you are new to networking and want to implement this setup, then prepare for some network outages ...

Derelict · May 22, 2019, 6:17 PM

Wholelottapfsense · May 23, 2019, 2:41 AM

@bmeeks said in Setting up pfSense and L3 switch:

You have a needlessly complex design for a home network. Drop the transit network and L3 switch entirely. What's the point of doing that? pfSense can easily route and take care of tagged VLANs. If you have extra NIC ports on the pfSense box you can put the different VLANs each on their own physical interface, or you can use trunk ports.

I thought of that design because my network is currently setup with the Fritzbox as modem/firewall and my L3 switch routing between VLANs.
You are right, letting pfSense do the routing is going to make things easier - even though L3 switches are said to be more efficient in routing (but that's probably not noticeable in a home network). Creating rules in pfSense seems definitively more intuitive than the ACLs in the switch!

Thanks for your feedback, I'll read up on the router on a stick approach.

Wholelottapfsense · May 22, 2019, 10:37 PM

@Derelict Thanks for that graphic, which seems to confirm the approach in my first message.

bmeeks · May 23, 2019, 2:11 AM

@Derelict shows a way to implement your original design, but my question would still remain -- why? That's extremely complex for a home network.

Unless you use hardware so old it has an 8086 8 Mhz CPU in it, you will not tax pfSense running a gigabit home network. So the routing efficiency of a Layer 3 switch is not going to improve anything.

Just let the pfSense box do everything with your ISP-supplied box in passthrough or bridge mode so the pfSense WAN has your public IP address on it. Then create VLANs and/or physical interfaces for your LAN on the pfSense box. Let pfSense provide the DHCP and DNS services for your networks.

JKnott · May 23, 2019, 2:41 AM

@Wholelottapfsense said in Setting up pfSense and L3 switch:

even though L3 switches are said to be more efficient in routing (but that's probably not noticeable in a home network)

Of course, passing through 2 devices, instead of just 1 will also be a performance hit.

Wholelottapfsense · May 23, 2019, 12:50 PM

@bmeeks said in Setting up pfSense and L3 switch:

@Derelict shows a way to implement your original design, but my question would still remain -- why? That's extremely complex for a home network.

Unless you use hardware so old it has an 8086 8 Mhz CPU in it, you will not tax pfSense running a gigabit home network. So the routing efficiency of a Layer 3 switch is not going to improve anything.

No, pfSense will be on a Supermicro 5018D-FN8T machine with an Intel Xeon D-1518 processor (2.2GHz, 4 cores), in which I'll put 8 Gb RAM.

In terms of network performance, my only wish is to have a 10 Gbit connection between my desktop and my server in VLAN 10 because I do a lot of video editing and copying of large video files between those 2 computers.

If I let pfSense take care of the routing as you suggested, I could reserve a 10 Gbit link between the SFP+ ports of the Supermicro 5018D-FN8T and the Netgear L3 switch for VLAN 10 and put the other VLANs on the other ports.

As explained before, the reason why my first idea was the option with the transit network is that my L3 switch is currently set up to handle the routing between VLANs (my Fritzbox modem/firewall can't handle VLANs) and I was thinking this would require less (re-)configuration when adding the pfSense box to the network. But I definitively see the advantages in your approach.

Hugovsky · May 23, 2019, 2:41 PM

Sometimes I like to overcomplicate my home network too. I use it to learn new things. Most of the time, I get back to previous design but, in the process, I've learned something and sometimes even stump on stupid errors made before. I love KISS technology but it's nice to go over the head sometimes. ;)

richley88 · Jan 23, 2020, 12:44 PM

How did you create the transit network in the end Wholelottapfsense as I am trying to do the same thing and do not want to use the management VLAN?

Wholelottapfsense · Jan 23, 2020, 8:25 PM

I followed the advice of bmeeks and have the VLAN routing done by pfSense.

As my main goal was to ensure high throughput between my Server and domain joined clients (all on the same VLAN) and all of those devices are wired to the Netgear M4300-28G-PoE+ switch, the data is handled at L2 level by the switch and does therefore (to my understanding) not pass via the pfSense box.

In the end, I also ditched the ISP Fritzbox because I didn't manage to get PPPoE passthrough working; my ISP gave me a fiber to ethernet converter instead.

Everything has been working great ever since.