Interface + VLANs setup advice

johnpoz

@gwaitsi said in Interface + VLANs setup advice:

LAN 192.168.1.1/24

If you just use IPs that are sub of this /24 for different devices, those are not "vlans"

Yes vlans would have their own interfaces and have their own network assigned that does not overlap with other networks.. If you have 192.168.1/24 on your lan - you could not create a vlan with 192.168.1.64/26 - this overlaps with your lan network.

Seems like you have really no actual vlans setup, and are just trying to use different IP space on the same L2..

If you downstream vlans on your switches, that they route - you sure couldn't use a network that overlaps them all as the transit to pfsense..

4o4rh

@johnpoz said in Interface + VLANs setup advice:

@gwaitsi said in Interface + VLANs setup advice:

LAN 192.168.1.1/24

If you just use IPs that are sub of this /24 for different devices, those are not "vlans"

Yes vlans would have their own interfaces and have their own network assigned that does not overlap with other networks.. If you have 192.168.1/24 on your lan - you could not create a vlan with 192.168.1.64/26 - this overlaps with your lan network.

Seems like you have really no actual vlans setup, and are just trying to use different IP space on the same L2..

If you downstream vlans on your switches, that they route - you sure couldn't use a network that overlaps them all as the transit to pfsense..

You misunderstood.

before (4port) used 2 for wan and 2 for Lagg to SW1
sw2, 3, 4 hang off ports on sw1
Lagg was configured with 192.168.1.1/24
vlan10, 20, etc had corresponding 24 addresses
SW1/2/3 assigned corresponding vlans as untagged ports for devices and the connection ports and Lagg to pfense was tagged

Now, with 8 ports
I have vlans divided by function, i.e. media/streaming, guest, etc. they cross all switches
I discovered, that i can't assign a vlan to a bridge, so i can't replicate the above that way.
So i am asking, if the correct approach is to;
use 26 subnets on the interfaces, and create a interface group to apply firewall rules to.

so
igb1 lan 192.168.1.1/26
igb2 lan 192.168.1.65/26
igb3 lan 192.168.1.129/26

int_group_lan igb1,2,3

igb1 vlan 192.168.10.1/26
igb2 vlan 192.168.10.65/26
igb3 vlan 192.168.10.129/26

int_group_vlan igb1,2,3

so then the firewall rules are based on int_group_lan / 24

or should i do something different

johnpoz

@gwaitsi said in Interface + VLANs setup advice:

or should i do something different

I told you what you could do already.. Use specific physical uplinks for your vlans.

If you want to use different lags for the different vlans - sure.. Laggs are over rated if you ask me - unless your talking a DC sort of setup where you have laggs connecting switches where you have lots of users and need more bandwidth than 1 physical link can provide.

But throwing a bunch of vlans on a lagg on a router you loose control over what physical path traffic can take, and you could run into hairpin

4o4rh

@johnpoz said in Interface + VLANs setup advice:

@gwaitsi said in Interface + VLANs setup advice:

or should i do something different

I told you what you could do already.. Use specific physical uplinks for your vlans.

If you want to use different lags for the different vlans - sure.. Laggs are over rated if you ask me - unless your talking a DC sort of setup where you have laggs connecting switches where you have lots of users and need more bandwidth than 1 physical link can provide.

But throwing a bunch of vlans on a lagg on a router you loose control over what physical path traffic can take, and you could run into hairpin

again, the wires are cross. the lagg config is what i had of the 4port J1900. But i am looking now at the 8 Port i3 i am migrating to.

So if i connect each switch directly to the 8 port box, each interface needs an ip because i can't use a bridge with vlans.

so, the only way i can see to get vlans in a backward compatible way, is to use each switch with a subnet, and each vlan with a subnet on each device where the vlan is required. then use interface groups for the firewall rules. so no lagg in this setup

or am i not getting it?

johnpoz

@gwaitsi said in Interface + VLANs setup advice:

or am i not getting it?

Your not getting it... Why would you have to or ever want to bridge over a lags? If you have free ports on your router and switch - either make a bigger lagg (more ports).. Or break them up into multiple lags that carry different vlans, or break out the interfaces to just carry only specific vlans.

So if i connect each switch directly to the 8 port box

So your question is this... You have the top, and your wondering how to do the bottom?

You wouldn't set that up - the whole point of distribution and access layers in a switch design..

You could do something like this

How to best setup would really depend on your vlans connection to your core or routing point would depend on traffic flows, etc. etc.

akuma1x

@gwaitsi I also think you're getting your IP address space and networks wrong, like you listed above.

igb1 lan 192.168.1.1/26
igb2 lan 192.168.1.65/26
igb3 lan 192.168.1.129/26

I don't believe that those are different networks. These are technically different networks, and will VLAN much easier/better:

igb1 lan 192.168.1.0 /24
igb2 lan 192.168.2.0 /24
igb3 lan 192.168.3.0 /24

I'm actually surprised pfsense let you make separate interfaces with those addresses. I would have assumed that it would have squawked at you about that. I don't think a network size of /26 or /24 really matters in this case. It's usually common practice to start with size /24 and adjust up or down from there, depending on your needs.

johnpoz

@akuma1x said in Interface + VLANs setup advice:

I don't believe that those are different networks.

Exactly - pfsense wouldn't allow him to do that on different interfaces.

4o4rh

@johnpoz exactly, the 2nd diagram. but these are access layer switches i am talking about.

my house has 3 floors, with 2 managed switch. 1 on the upper floor and 1 in the basement.
i was using the lagg to increase the bandwidth between the two.

on the upper floor switch is an openwrt tplink for wireless access and the switch ports are used as unmanaged vlan ports

on the basement switch is two openwrt tplink for wireless access on the ground and basement with switch ports configured in unmanaged vlan ports.

on each floor are media devices that connect to the internet i.e. gaming, netflix, etc. these are all connected to the same MEDIA vlan like in pic 1

on each floor is a wifi and wifi-guest on separate vlan and the switch ports have untagged vlans for either work PCs or home or kids. everything works well in this config on the J1900 but i don't get full performance of the 1GB ports

I figured, the performance would be better if i hang each of the managed switches and tplinks, directly off pfsense but am struggling with how a setup my MEDIA vlan for example to then service all switches devices.

johnpoz

@gwaitsi said in Interface + VLANs setup advice:

i don't get full performance of the 1GB ports

Well you could be running into your issue with same physical path with your intervlan traffic. Which if you broke out multiple uplinks you couldn't run into that problem..

Lets look at it like this.

Here you have you have setup lagg from your dist switch connected to pfsense.

And this traffic happens to go up down over the same physical link.. This could be hindered.. And you don't see full gig.

If you broke out your uplinks to different physical uplinks you control the path.. So vlan X is on 1 phsyical path, and vlan Z is on different physical path and you don't have a hairpin.

Your issue could be that for whatever reason your pfsense can not just route at full wire speed? But hairpin over a physical interface would never be a helper..

Maybe you do 2 port lag for vlan X and 3 port lagg for vlans Y and Z.. All depends on your traffic flow - what is talking to what and how much..

Trying to bridge interfaces in pfsense sure and the hell not going to make for better performance that is for sure..

Without understanding the data flow and amounts trying to optimize your physical paths is just guess work.. My point from before was that a lagg gives you simplicity and redundancy on a failure - but it does come at a cost of actual control of what physical path traffic might take.

4o4rh

@johnpoz if i take the use case to more detail.

PFSENSE -> WAN
        |-    SW1 (basement 8 port netgear) <- TPLINK basement (WIFI + media + home + IoT)
                                                                 <- TPLINK ground (WIFI + media + VOIP)
              SW2 (3rd floor 12 port netgear)    <- TPLINK 3rd (WIFI + work + office  + MEDIA + NAS + IoT)

So the majority of traffic is internet outgoing.
Internal traffic from SW1 -> SW2 is home media server, nas (smb/nfs) + IoT servers
VLANs dedicated for MEDIA, VOIP, IOT, SERVERS, HOME, WORK, OFFICE

Now I change from the J1900 to i3 8 port, I want to maximize the port usage for pfsense and performance overall.

So, what do you recommend as the way to best optimize the usage of the ports and performance overall

johnpoz

@gwaitsi what are those specific switch models - do they do vlans.. tplink has had some serious issues with vlans before..

What is your internet traffic, unless its higher than gig.. That comes out of the equation completely.. Doesn't matter if you had 100ge coming into pfsense if internet is only 1 gig, etc.

So is the majority of your intervlan traffic.. What mostly talks to your nas for example - home or media?

4o4rh

@johnpoz said in Interface + VLANs setup advice:

@gwaitsi what are those specific switch models - do they do vlans.. tplink has had some serious issues with vlans before..

What is your internet traffic, unless its higher than gig.. That comes out of the equation completely.. Doesn't matter if you had 100ge coming into pfsense if internet is only 1 gig, etc.

So is the majority of your intervlan traffic.. What mostly talks to your nas for example - home or media?

TPLinks are WR1024ND running latest OpenWRT so they work quite well with vlans.
i use the lan/wan bridge as unmanaged

there are two wan connections with 120mg cable and 100mg fibre
the nas box is also server for media and some IOT apps with total 4x vlans, so yeh traffic is mostly local PCs to NAS, media devices to nas on media vlan, etc.

4o4rh

@akuma1x just did some testing and configured

igb2 192.168.1.1/26 LAN_igb2
igb3 192.168.1.65/26 LAN_igb3
igb4 192.168.1.129/26 LAN_igb4
igb5 192.168.1.193/26 LAN_igb5

then created an interface group
LAN comprising LAN_igb2, LAN_igb3, LAN_igb4, LAN_igb5

put some test firewall rules in LAN group

gives the desired effect, which means in an alias I can use a network address of 192.168.1.1/24
i can apply the same principle for the VLANs i.e. if i have devices for VLAN10 on igb3 and igb5, just create 2 VLAN10 with 26 subnet, then a VLAN10 group to simpify rule management

4o4rh

@johnpoz
Hi John, i created a bridge for the separate LAN ports, so i can plug each switch directly into the quotom. they are only 8 port and 12 port netgears, but i need to free up some ports, which is why i got the 8 port qotom as an upgrade.

that approach also worked for the VLANs, but i am finding that increasing the total interfaces, causes the CPU to be running hotter even though the load is minimal (still on the bench)

some of the VLANs, i can isolate to upstairs or downstairs (thereby reducing a bridge + 1 vlan interface) that leaves me with the WIFI.

Currently i have 3 different access points, 2 connected to downstairs switch, 1 to upstairs.

I have internal WIFI network and a GUEST network.
I have them all setup with the same SSID and channel.
They are all on the same vlan (with the present setup) and work seemlessly.

with the new setup, i also bridged the vlans and it seems to work.

My question is;
if i put them on a separate vlan for upstairs and downstairs, will i still get seemless switching when moving between levels?

akuma1x

@gwaitsi said in Interface + VLANs setup advice:

that approach also worked for the VLANs, but i am finding that increasing the total interfaces, causes the CPU to be running hotter even though the load is minimal (still on the bench)

I think you're getting higher temperatures on your pfsense box because you are literally asking it do do everything on your network - switching, routing, firewalling, lagging, all of it. That's not normal for a router/firewall device on a small home computer network.

In my opinion, and I know you probably don't want to hear it, you don't have your network setup correctly.

johnpoz

@akuma1x said in Interface + VLANs setup advice:

you don't have your network setup correctly.

From these statement I would agree

@gwaitsi said in Interface + VLANs setup advice:

i created a bridge for the separate LAN ports

@gwaitsi said in Interface + VLANs setup advice:

i also bridged the vlans and it seems to work.

Generally speaking if your "bridging" your doing it wrong ;) If you need switch ports - get a switch.. While there are legit reasons to create a bridge, normally it would be when you need to change media types.. A typical legit reason for a "bridge" would be in say a AP, which is designed be a bridge from ethernet to wifi..

Seems like he bought a router with 8 discrete interfaces, just so he could use it as switch that he could of gotten for like $40...

I would like to see a drawing if this final setup - but yeah from comments of bridging I would say its not optimal at all..

4o4rh

@johnpoz here is my original/current network with a Qotom J1900 that runs with a temp of 47degC at 25% CPU utilization

The only additional packages to standard are pfblocker and suricata. everything works like a charm

J1900_Design.ndg.png

4o4rh

@johnpoz here is the new design, with the introduction of an additional managed switch for the IOT which has more than the 2 depicted.

i bridge the Lagg0/ProPOE/GS108 with the LAN 192.168.x.x/24 for the original LAN
i bridge the Lagg0/GS108 with the VLAN Home of the original 192.168.x.x/24
i bridge the Lagg0/GS108 with the VLAN WIFI of the original 192.168.x.x/24

according to the docs, the bridge will use spanning tree, so most of the traffic will go directly to the internet, but some will go to the server or home vlans.

the reason for 2 and 3 was to keep them backward compatible to not impact my windows /smb networking setup.

the IOT i made as 2 vlans and created an interface group to apply the rules to both
the VLAN GUEST i made as 2 vlans and created an interface group to apply the rules to both
WRK1/WRK2/MEDIA are single interface VLANS so backward compatible rule set
- media server will move into the SVR VLAN

as with the original, i have pfblocker and suricata plus ntopng. CPU utilization is at 5-10% with avg temp 50-55degC. increasing the number of interfaces makes the temp go up, by about 5deg as i added them in one by one and monitored.

On this config, there are 21 interfaces in total vs 15 of the J1900 config

i3_Design.ndg.png

4o4rh

@akuma1x see my current and upgrade designs. i am open to advice

4o4rh

@johnpoz Hey John, so i am wondering if this would be a better construct. would get rid of the lagg on the qotom, but don't see that it would help reduce the interfaces or firewall/routing greatly.

Media, home, wifi all pass to svrs through firewall rules only.
media, guest, DMZ/IOT/VOIP pass exclusively to internet
media/iot pass to svrs through firewall rules only.

i could reduce interfaces on the quotom for example by

vlan guest to only gs108 therefore 1st floor would switch through gs110 to gs108
vlan wifi to only gs110 therefore wifi ground/basement would switch through gs108 to gs110

what are your thoughts?

2022-03-22 18_43_11-C__Users_Stevo_OneDrive_Documents_i3_Designv2.ndg.png