VLAN question for noob moving from Cisco ASA
-
pfSense has its own interfaces that are then tied to the physical/vlan interface so you have to think from the ground up layer wise. When you click on "Interfaces -> Assignments" it becomes quite clear.
You got physical interfaces in your box. e.g Intel GBit nic would become igb0 (Layer 1)
To those Interfaces you assign VLANs and you get a VLAN interface: igb0.10 (Layer 2)
Then you assign the VLAN interface in pfSense's GUI to a virtual pfSense interface, or short: interface. If igb0.10 is the LAN-interface it will be called "LAN".
And then you assign an IP-Address/subnet to said "LAN"-interface. (Layer 3) (Internally all that happens is that the IP is assigned to igb0.10). Those interface names will become "aliases" in any other configuration that is going on, e.g. packet filter, nating, routing...
For all the other interfaces it works the same, you will get igb0.20 and igb0.30. They will become OPT1 and OPT2 in pfSense. When you click on the configuration of such an interface you can just change its name. I usually to something like: IF_V10, IF_V20 and so on.
The good thing about this is: When you add another NIC for VLAN 20, because the load gets to high, let's say it's igb1, then all you do is create VLAN 20 on igb1, get igb1.20 and then go to the configuration of IF_V20 and change the underlying interface there.
After a click on save and apply the whole thing including pf rules and everything for vlan 20 just magically moves over to the new interface
It's really awesome, but it has a downside:
When you save the overall configuration to your machine and try to restore it on other hardware with different nics that get a different name by the kernel, you have to do the lower layer configuration again, e.g. assigning the VLANs to the physical interfaces.
The upside:
Once that's done you just assign the "pfSense interfaces" back to their physical pendants and it all starts working again magically.
Cu
-
Hi @Grimeton,
Thank you very much for the prompt response and helpful explanation.
I think I understand everything you said however I'm still not sure how I would use the same VLAN on multiple interfaces. Specifically, wouldn't igb0.10 and igb1.10 require their own subnets? (In my ASA setup, VLAN 10 devices connected to ports 1, 2 and 3 are all on the same subnet.)
Thanks again!
-
@xyzzyz No.
The thing is the ASA is a mixture of Router and Switch. So you end up with three interfaces (l2) on the same subnet (l3) but they're not holding an IP-address on each. Instead, the ASA-CPU is connected to VLAN10 as well and is holding the IP-Address of the subnet on that connection.
So you create three interfaces igb0.10, igb1.10, igb2.10 and put them on a bridge (bridge0), that then gets the IP-address.
Another approach is to give all three interfaces the same IP-Address and then just let the kernel/switch figure it out based on the ARP requests/responses. Linux does that trick, with pfSense I don't know.
But as long as the ASA was just on the same VLAN for said interfaces because it became a bridge there, the solution is pretty obvious.
Cu
-
I see. I really appreciate your thorough explanation. Since pfSense has router and switch capabilities, I kinda assumed it would work the same way.
Like you said, if all 3 interfaces just used that one VLAN, a bridge would be the answer. However, in my ASA case, it's actually like this:
Interface 1 - Trunk with VLAN 10, 20 and 30
Interface 2 - VLAN 10
Interface 3 - VLAN 10Is there any way to accomplish this?
Thanks again!
-
Trunks are called "LAGGs" in pfSense.
But they have a lot of downsides. Traffic shaping is not supported, also it only supports only certain modes. A lot of people think you take four gbit connections, create a lagg and utilize the full four GBit. Sadly most modes use the source or destination MAC-address to decide over which port the packet is send. And this never varies, so one host can only utilize the speed of one connection...
Regarding the ASA-configuration I'd have to know what is on the other side of that configuration. If the ASA is a bridge in VLAN 10 and connects to three different switches that are not interconnected to each other or where the l2 way is blocked through (R)STP configuration, then that's a setup where a bridge on pfSense would work.
Also VLAN 10 can be assigned to each interface on its own, and each interface is connected to different switches that are not interconnected with eachother. So the 802.1q is basically split by using a mixture of port based and 802.1q VLAN where VLAN10 on each of the switches doesn't know of the others...
But unless I know that, I can't tell for sure.
Cu
-
With regards to trunks, I was actually referring to a VLAN trunk (a single logical pipe carrying multiple VLANs to my two 24 port switches). I'm familiar with LAGGs (bonding multiple physical connections into a single one for throughput and/or failover purposes) but that's not what I was looking for here.
With regards to the ASA config, I'm not sure how Cisco has that logic implemented internally. I certainly don't have anything advanced or special in the config. I simply have multiple ports specified with different sets of VLANs and each VLAN has its own subnet. Since the subnets apparently run at the VLAN level, I guess it automatically creates a bridge at the VLAN level for any interfaces that are assigned to that VLAN. It seems to do this even if there's a different mixture of VLANs on each interface.
I think the bottom line is that it appears with pfSense I'm limited to scenarios where I can bridge the interfaces together. IE, if I want the same VLAN (or same set of VLANs) on multiple interfaces, I can bridge them together. However, once I've "used" a VLAN on one of those, I can't have it appear in another one.
So, I could do this:
Bridge 1 - (Interfaces 1 & 2) - VLAN trunk with VLANs 10, 20 and 30
Bridge 2 - (Interfaces 3 & 4) - VLAN trunk with VLANs 40, 50 and 60
Bridge 3 - (Interfaces 5 & 6) - VLAN 70However, I couldn't, for example, add VLAN 30 to Bridge 2 or Bridge 3 or any other interfaces.
Or am I missing something here?
Thanks again for all your help. I really appreciate it.
-
@xyzzyz said in VLAN question for noob moving from Cisco ASA:
Or am I missing something here?
Yeah why and the hell are you bridging interfaces??
-
@johnpoz said in VLAN question for noob moving from Cisco ASA:
@xyzzyz said in VLAN question for noob moving from Cisco ASA:
Or am I missing something here?
Yeah why and the hell are you bridging interfaces??
To be honest, I wasn't really looking to bridge the interfaces but it seems like I might need to to accomplish what I'm trying to accomplish.
Or I guess I could go buy an extra switch or two but I'd rather not waste the empty ports on the PF box and have more power draw in the already toasty equipment closet.
-
They are not wasted, use them as uplinks for you vlans...
Looks like you have 7 vlans there.. I don't get the bridging of those interfaces...
-
@xyzzyz
Hi, you're mixing something up here.The bridge runs on top of A SINGLE VLAN. That's it. That has nothing to do with how many VLANs you can assign to a network interface. It's, from the bottom up: NIC -> VLAN -> Bridge.
What is still not clear here is the ASA configuration.
Let's go with one VLAN:
The 802.1q VLAN is on multiple interfaces. Are those interfaces connected to the same layer 2 network that is then managed through STP to not produce loops?
Are the ports tagged on the outside or untagged?
Does the ASA have an IP-Address per member in the same VLAN?
Does the ASA bridge the same VLAN together and have the IP on the bridge?
Once we know this, we can talk about LAGG and similar things.
Cu
-
@johnpoz Sorry, that was just a fictitious example to show how I cannot mix-and-match VLANs on different interfaces like I currently do my Cisco ASA. I was trying to avoid boring everyone with the gory details of the full setup. But just in case, here it is:
VLANs:
VLAN 2 - WAN
VLAN 10 - Work
VLAN 11 - Personal
VLAN 12 - Printers
VLAN 15 - Guest Wi-Fi (SSID #1)
VLAN 16 - Guest Wi-Fi (SSID #2)
VLAN 99 - Management (ie, for IPMI ports on servers, management interfaces on WAPs, etc)VLAN 10 can talk to 2, 11 and 12. My main workstation on VLAN 10 can also talk to 99.
VLAN 11 can talk to 2 and 12.
VLAN 12 can't talk to anything.
VLAN 15 can talk to 2.
VLAN 16 can talk to 2.
VLAN 99 can't talk to anything.Interfaces:
Int 1 (VLANs: 2) - This is the WAN connection.
Int 2 (VLANs: 10-12, 15, 16, 99) - Trunk to a 24-port switch which connects to another one. The vast majority of my network traffic stays on these 2 switches. Of the subset that goes through the ASA, this interface is the busiest.
Int 3 (VLANs: 99) - If I have problems with the switches, I an temporarily connect a laptop here.
Int 4 (VLANs: 15-16, 99) - WAP #1. I direct connected both WAPs to the ASA because (a) it has PoE, (b) I had open ports and didn't want to tie up the trunk. Wireless devices get assigned to IPs on VLANs 15 and 16. The management interface is on VLAN 99.
Int 5 (VLANs: 15-16, 99) - WAP #2. Same as above.I'm open to any suggestions!
-
So your not bridging interfaces... Good! ;)
Why would you put vlan on multiple physical interfaces? If you did that - that would be done on a lagg, not a bridge.
edit:
Wait you do have same vlans on multiple physical interfaces from what you show... WTF??? You don't have those all bridged do you? What box are you running pfsense on, does it have switch ports? Are those switch ports, or interfaces? -
@johnpoz said in VLAN question for noob moving from Cisco ASA:
So your not bridging interfaces... Good! ;)
Why would you put vlan on multiple physical interfaces? If you did that - that would be done on a lagg, not a bridge.
edit:
Wait you do have same vlans on multiple physical interfaces from what you show... WTF??? You don't have those all bridged do you? What box are you running pfsense on, does it have switch ports? Are those switch ports, or interfaces?The example I provided is the current setup on my Cisco ASA. While I didn't explicitly setup bridges, I suspect there's some sort of bridge happening automatically at the VLAN level. For example, if I connect a laptop to ASA Interface 3 using VLAN 99, it can talk to VLAN 99 devices connected to switches connected to ASA Interface 2 as if my laptop was connected to the same switch.
As for why the same VLAN would show up in different ports, the VLAN 99 for management is a good example why I would want that. I have dedicated IPMI ports for my servers connected to the switches downstream of ASA Interface 2. I also have management interfaces on the two WAPs, which each directly connect to ASA Interfaces 4 and 5. It's really nice having all of my management things on a single VLAN so I can lock it down.
Another good example would be the two WAPs. They both support the same two SSIDs (one is for 2.4 GHz, the other is for 5 GHz). The SSIDs use VLANs 15 and 16. So, I have both VLAN 15 and 16 on the two WAP ports.
I hope this makes sense. Thank for your help!
-
Hi,
sorry for my late reply. One has to sleep every now and then ;-P
As I thought those interfaces are bridged. That's usually something you should leave to the switches. As the ASA is both that's something that is ok in the current setup but should be avoided later on.
So what you should do is configure the VLANs on the other switches and interconnect them and then just connect your future firewall to those switches or to one switch with the VLANs all on one interface or distributed over multiple interfaces.
If you need link aggregation, try to keep it away from the firewall itself and use it on the switches. It's basically a math game that you have to play.
I got gigabit interwebtubez coming in on the firewall, so I don't need 2gbit/s out or even more. There's just no reason for it, as the slowest part here is the interwebz and it will never get faster than that.
If you want to have routing between the VLANs, then you can do that via the firewall and use multiple interfaces for it, Like one VLAN per interface. If you DO NOT NEED traffic shaping later on, e.g. for VoIP, then it's fine to use LAGGs.
Cisco switches are able to distribute the 802.1q VLANs via their own protocol. Same goes with trunks, where I'd use LACP.
I'd also setup some [R|V]STP on the switches/vlans so that loops are prevented. In bigger environments, e.g. going over multiple buildings, I usually create rings that are stopped at some point via [R|V]STP. But I do not know the whole setup you got there, so it's a bit hard to actually help you. It would also help to know how much traffic you actually got going on on each VLAN so that you could either distribute it via multiple GBit/s NICs or just get one or more 10GBit/s NICs to help you out.
I always prefer one 10GBit/s connection over a trunk of GBit/s connections, but again it's a math game you have to play.
In times where you get systems like this one [1] for a fair price, there's always a solution on the hardware side...
Cu
[1] - https://www.supermicro.com/products/system/1U/5018/SYS-5018A-FTN4.cfm
-
@Grimeton, thank you VERY much for taking the time to provide very detailed and helpful information.
If it wasn't for the two cases where I connect the WAPs to the ASA, my setup would basically the classic "router on a stick". My trunk feeds switch #1, which feeds switch #2. Pretty simple.
The current ASA has 100 mbit ports so putting the WAPs on one of the switches would be painful. Specifically, that trunk between the ASA and the 1st switch would be carrying the WAP-to-WAN traffic plus the local traffic routing between VLANs.
However, the pfSense box I'm building will have at least 2 SFP+ ports. As a result, I'll have plenty of bandwidth to put everything on the switches. So I think I'll take your advice and steer clear of bridges on pfSense.
With regards to [R|V]STP and its variants, I have a basic understanding of what they do. But if I have a relatively simple router -> switch -> switch setup, do you think I need to implement STP (beyond whatever might be automatically enabled)?
Thanks again!
-
I think the ASA does do some sort of bridging called BVI interface.
https://www.petenetlive.com/KB/Article/0001422
I used to have a ASA 5506 and used the "same-security-traffic permit inter-interface" command. Googling it I found the Kb article above.
So If you are using BVI interfaces with VLANs you'll be able to talk across the physical interfaces...
i just re-read 100Mbit + PoE , so you must have a 5505. However BVI is available on the 5505, too.
-
@xyzzyz You will only need some kind of STP if you create a ring.
It can be used to recognize topology change so that the switches can bridge packets over a different interface once one "route" (it's not an L3 route), goes down.
So you primarily need it to avoid loops.
If you have a simple setup, there's not really a need for it, especially as STP is rather slow and without RSTP you won't have a lot of fun.
Cu
-
@gcu_greyarea, @Grimeton - Thanks for both of your responses!
One (hopefully last) question...
On the ASA, the WAN port used VLAN 2. I think this was required on the ASA because the IPv4 info is set up at the VLAN level. Also keeping in mind the ASA's bridging model, I think this was needed to keep WAN traffic off the other ports. There definitely was not any ISP requirement that I use a specific VLAN.
My question: On my pfSense replacement for the ASA, is there any advantage to setting up a VLAN for the WAN port?
-
@xyzzyz No. Only if your ISP is using tagged vlans for different services like VoiP or IPTV, etc.
-
@xyzzyz said in VLAN question for noob moving from Cisco ASA:
My question: On my pfSense replacement for the ASA, is there any advantage to setting up a VLAN for the WAN port?
No.
