2 Newbie Questions on Network Architecture

[Brain Damaged Pilot]

Hello all,

I'm looking to use pfSense with my home network and had a couple of questions that my google-fu failed me on. My reasons for using pfSense are:

1. I mistakenly watched the Craft Computing video about the $140 Microtik 10Gb managed switch and thought "ooh shiny, I wanna try that" which resulted in diving further down the rabbit hole of home networking. I was going to have pfSense do everything to include switching (literally every ethernet cable in the house terminating at the back of the machine) but general consensus places that squarely in the "don't" category.
2. To improve general network security with VLANs (main computers, IOT, etc)
3. "cuz I wanna." Technical term, I know.

Basic hardware would be an i5-4690 on a Gigabyte z97 Black "Server Grade" board with 8-32 GB of ram and ssd for SNORT and other add-ins.
For my questions assume 2 VLANs, "Trusted Computers" with 10 Gb NICs and "Everything Else" with 1 Gb NICs
Basic network setup is Internet -> pfSense -> 10 Gb managed switch (to "Trusted Computers) -> 1 Gb manged switch (to "Everything Else"). The 1 Gb switch would be plugged into the 10 Gb switch and "downstream" (for lack of a better term).

Question 1: 10 Gb or 1 Gb single cable connection between pfSense and 10 Gb switch.
If I want to transfer a file from the "Trusted Computers" VLAN to the "Everything Else" VLAN presumably the managed switches would pass the traffic upstream to pfSense to route between virtual networks. If so, and I use a single 1 Gb connection between pfSense and the 10 Gb switch, would my theoretical bandwidth across networks only be 500Mb, 500 Mb upstream from the "Trusted Computers" and 500 Mb downstream to "Everything Else"?

Question 2: If the managed switches are aware of VLAN tags and knows to keep traffic separated, is the purpose of pfSense purely to act as the router between those 2 virtual networks?

Thanks in advance for any assistance.

[stephenw10]

Yes pfSense would be routing between those subnets in that scenario.

If you ever put more than one VLAN on the 10G segment then you would need a 10G link to pfSense if you ever wanted to see more that 1Gbps between them.

A single 1Gb Ethernet connection offers 1Gbps full duplex so 1Gbps in both directions at the same time. So bandwidth between two VLANs sharing that can be more than 500Mbps but will be less that 1Gbps due to the reply packets needing to use some bandwidth the other way.

You probably won't ever need more that 8GB of RAM.

Steve

[johnpoz]

@stephenw10 said in 2 Newbie Questions on Network Architecture:

So bandwidth between two VLANs sharing

Lets also be clear because users seem to not grasp the concept of physical interfaces and shared vlans on it when doing intervlan traffic.

Your physical interface has limit of 1ge full duplex.. When you put vlans on this interface they share that bandwidth.. So when you do vlan X to Y on the same physical interface you have actually cut your bandwidth in half when devices on these different vlans are talking to each other.

If your gig vlans are on a 10ge physical interface you shouldn't have much to worry about. But if you wanting to move files from box X to box Y and doing intervlan over the same physical interface on the router your not going to see what you would expect to see over gig..

This is why having a router with multiple physical interfaces is nice since you can put it physical interface as uplink from your switches for each vlan, so now your not putting intervlan traffic up/down over the same physical interface.

So for a home setup, my different wifi vlans all share the same physical interface uplink to the router.. But they almost never talk to each other, and if so they are not going to be using full gig anyway.

Understand your data flow paths and the bandwidth requirements between vlans will help you plan out your network, etc.

You should prob want to use 10ge as your uplinks from your switch to pfsense for both networks. If all your devices on untrusted vlan are only gig, this allows you to for example talk to multiple devices on untrusted from trusted at the same time at full gig speeds.. If your trusted and untrusted not going to really talk to each other does't matter. If your wan connection is gig or less then anything other than a 1 gig uplink from that vlan to pfsense prob a waste.

[stephenw10]

Yes, the best you can expect is 1Gbps half-duplex. If that's TCP traffic then any reply packets have to interrupt that or are delayed by it causing less than Gigabit line rate. Often a lot less.

10GbE cards are not that expensive these days. If you already have 10G infrastructure in place just use a 10G link and avoid any of those issues. IMO

Steve

[Brain Damaged Pilot]

Thank you for the replies. So just to make sure I'm understanding, if 2 vlans are using the same physical connection, and all traffic is moving 1 way (from 10 Gb -> 1 Gb network), I could theoretically see 1 Gb line speed, but functionally it would be lower based on overhead, receive packets, and other real-world factors.
If traffic is flowing both directions simultaneously (10 Gb <-> 1 Gb) I'm looking at 500 Mb theoretical speeds.

Makes me lean towards using one physical 1 Gb nic dedicated to the 1 Gb switch, and a another 10 Gb nic for the 10 Gb switch, just to get full ~1 Gb transfer speeds across the router.

[stephenw10]

Yes, you would be limited to 1Gbps total in both directions but will see less than that in reality. And if you are trying to actually push traffic both ways simultaneously probably a lot less as it will be competing for bandwidth.

If you have 10G and 1G link then obviously that doesn't apply for traffic between them.

If you have just a 10Gb link it probably doesn't apply anyway as you would be limited be hardware before hitting 10Gbps in most cases.

Steve

[JKnott]

@johnpoz said in 2 Newbie Questions on Network Architecture:

So when you do vlan X to Y on the same physical interface you have actually cut your bandwidth in half when devices on these different vlans are talking to each other.

It's not quite that simple. While both VLANs are on the same wire, the portion of bandwidth will depend on the traffic patterns. For example, with a large file transfer, most of the bandwidth will be used in one direction, with only a small amount in the other. Bear in mind, with full duplex, what happens in one direction does not affect the other, so when doing that file transfer, one VLAN will have most of it's traffic in one direction and the other VLAN, in the other direction.