Multiple NIC ports with different VLANs or 1 NIC Port for all Vlans
-
I think I know the answer to this question and yet I wanted to get some opinions anyway.
I've run pfsense for a number of years, always on server hardware and am looking to build a new pfsense box.
What would be the better choice from a purely performance point of view for a physical hardware interface NIC setup.
Configuring multiple interfaces (different NIC ports) on pfsense each with different vlans or pushing all vlans through 1 interface out of pfsense
It has always been my understanding that you should try to avoid as much cross interface traffic as possible as routing between ports is not as good as a switch and I do have a nice vlan capable switch
I have often seen people physically segregate their wifi or dmz by using different NIC ports out of a PFsense box instead of using Vlans. Why? This is really my debate in build a new router.
Example
Config 1
WAN
LAN > VLAN 10 lan AND VLAN 20 dmz AND and VLAN 30 wifiConfig 2
WAN
LAN1 VLAN 10 Lan
LAN2 VLAN 20 ie DMZ
LAN3 VLAN 30 ie Wifi -
If each of your networks run from a dedicated port you don't need vlans at all.
-
"I have often seen people physically segregate their wifi or dmz by using different NIC ports out of a PFsense box instead of using Vlans."
When you run multiple vlans on an interface you hair pin any traffic that is intervlan.. Ie traffic between vlan A and B that are on the same physical interface means your traffic enters and leaves the same physical interface - hairpin. Your available bandwidth is therefore cut..
So when possible it is always better to use native interfaces to allow full bandwidth and not hairpin. If your devices are low bandwidth usage or you have little intervlan traffic between the vlans on the same physical interface than its not all that big of a deal. But all vlans on an interface means that the bandwidth of that physical interface is shared between all the vlans on that interface when they are sending traffic and from pfsense. If it goes out the wan its not a hairpin.. But traffic between vlans on the same physical interface is double bad because your traffic is going through the same physical interface twice.
If you have the physical interfaces available and you are concerned with bandwidth constraints then yes it makes good sense to remove any and all possible hairpins. And to spread your traffic between as many physical interfaces as possible.
So vs putting your 2 networks on the same physical interface put 1 network on each and run an uplink for each network to your switch.. Sure you would isolate these networks with vlans on your switch.. But now your not sharing bandwidth of the interface with multiple networks.
-
Thanks for the responses. This has been greatly helpful. I think its my lack of knowledge about the effects of vlans on performance which most informs me where as I have seen how poor cross nic port performance can be, especially when bridging interfaces, that I have always tried to limit such traffic.
I'm looking at the X10SDV-4C-7TP4F as a board for PFsense 2.4 so I would be using the 10GB SFP+ uplinks but just because its 10gb doesnt mean I should chose a non optimal setup for the network traffic so you're answer again is helpful. Sounds crazy but my ISP will give me up to 8 simultaneous IP address so although this board + an intel i350-t4 would give me 10RJ45 I would be running out of ports if i wanted to create 8 gateways, hence my question about the need for seperate wifi or dmz ports.
-
Well if you have a 10ge uplink from your switch and 1ge interfaces on your switches for your devices. A hairpin prob not going to be much of an issue ;) hehehe
-
The only time you really need to be concerned about the hairpinning is when you have, for example, a file server on one VLAN and its clients on another. If there is just ancillary traffic between the VLANs and everyone is primarily accessing the internet, I wouldn't sweat it.
Like John said, with a 10G trunk link and 1G links on the switch you're probably good.
If that isn't enough, LACP a couple 10G to the switch. :)
-
So I will run at least 2 switches, maybe 3 if there arnt enough ports, off this pfsense machine. The main switch has 16 10GB ports for fast access to my Nas as it seems frequent transfer of large video and image files. clients do connect to it from other vlans. The other switches have 10gb uplinks with 24 1GB ports which i'll use to subnet other traffic.
Sort of along the lines of my originally question what is the optimal setup for multiple switches off of pfsense
config 1
Switches off Switchesconfig 2
Switches connected to different PFsense Nic Ports.I've never liked switches off switches, but these are managed switches
Even though a 10GB uplink is unlikely to get saturated from a bunch of 1GB connections doesn't mean i shouldn't care how its done. Going for technically optimal setup here. Obviously I could set this up a number of different ways but would like to chose whats best.
-
If you want fast access to a NAS on different subnets I would use a layer 3 switch. I would not put that traffic through the firewall 10G or not.
You generally have to go switch-to-switch at least once.
That you only have two switches makes it less obvious that the one connected to the firewall would essentially be a core/backbone switch, and the second would be an edge/workgroup switch. If you were going to the core switch then out to a dozen different wiring closets it would be more obvious.
You would not want to "daisy-chain" more switches on the edge switch but connect any additional switches directly to the "core."
-
Thanks. That makes a ton of sense. I think i knew this but just needed to think about it theoretically.
-
So as I understand it seems like I would essentially have a choice. Manage Vlan traffic through PFsense for ease of manageability or do it through an L3 switch for performance. Is that correct.
Also what would be the implications of essentially splitting up where vlans are managed, managing most Vlans through pfsense while using an L3 switch to
manage other vlans. I dont see why that wouldnt work but then again i'm not a networking expert which is of course why im here. : ) -
If you need a firewall between the segments, put it through pfSense. If you don't (especially if you need performance such as between the NAS and its clients) use a L3 switch.
Not that pfSense or any firewall won't move a lot of data, but it will never equal what a L3 switch can do. Well, at least not right now.
-
Hi,
If you use traffic shaper, then only individual network ports, on VLAN it does not work (it is written in the documentation) -
"I will run at least 2 switches, maybe 3 if there arnt enough ports"
Are these ports needed in the same area or you going to run an uplink to another room/closet to have ports there, ie another part of the building? If you need to start thinking about adding a 3rd switch because of ports in the same area - its prob time to get a higher density switch..
Or this does sound like a business with 10G and 24 port switches, etc. Then get stackable switches vs having to daisy chain them.. Also if you do need multiple switches off your core then uplink them to the core… Avoid this...
CoreSwitch -- switch -- switch
You would do this
switch -- Coreswitch -- switch
I agree completely about the L3 switch if you need performance between segments if you do not need to firewall between these segments for sure! But in small setup its also just easier if you need performance between devices to just put them on the same L2 if your not worried about firewall..
So if you have NAS and you have clients that need max speed to this NAS... its much easier to just put them on the same network vs routing it at all be it at your firewall or some L3 switch.
