Multi-LAN and VLAN trunking

pstokman

I think this is the best location to ask my question.

My pfSense appliance has multiple interfaces, 6 in total of which 4 will be used for LAN. I want to combine all 4 interfaces as one, so I can connect our switches to the pfSense. The way I see it, this should allow inter-switch networking through pfSense (client 1 on switch 1 connects to client 2 on switch 2, using pfSense). I've already created an interface group of these four interfaces, but only one interface has an IP on it. I'm afraid that if that interface would be disconnected, none of the clients are able to connect to the internet. On Linux, there is something like bonding, putting multiple interfaces together as one to allow better connectivity. Switches have LAG or similar. I've seen LAGG in pfSense, but I'm not really sure how to configure that and if it's what I'm looking for, looking at the description. Taking another look made me think it over, see below.

We also have two VLANs, one untagged (default VLAN1) for all the clients and a separate VLAN for our VoIP network (VLAN2). The VoIP phones have two ports, allowing computers to be linked to the network through it, saving in uplinks on the work place. The switches are configured to accept both VLANs on all the ports, so essentially they are all in trunk mode.
As noted above, there are several LAN interfaces and I want to have VLAN trunking on all of them to allow VoIP and data to work together, but not route between one another.

Is it possible to create a LAGG which consists of both the physical interfaces and the two VLANs? One can only create a VLAN with a parent NIC, so I guess I create the LAGG from the four LAN interfaces and use that LAGG as parent interface for the VLANs. But that's where I come to a halt. In the documentation, it says to assign the VLAN to an interface as well, but how would I combine the two? Is it really required and can't I skip the interface assignment as the VLAN is already on the LAGG?

Please tell me if my train of thought is correct and I just need to create a LAGG, hook up the two VLANs on it as parent interface and be done.

clarknova

@pstokman:

Please tell me if my train of thought is correct and I just need to create a LAGG, hook up the two VLANs on it as parent interface and be done.

Did you try that? I'm not sure I follow your whole train of thought but it strikes me that this might be all you need to do. (keep in mind that some types of LAGG need to be configured on the switch as well).

If you're still having trouble it would be immensely useful to include a network diagram.

stephenw10

Though I've never tried it what you are suggesting sounds like exactly what you should be doing.
Some quick googling reveals plenty of people doing this in FreeBSD. In fact there seems to have been a patch that filtered back from pfSense to FreeBSD correcting a bug in this situation.

So, yes, create a lagg interface with your four real interfaces, use that as the parent for your VLANs.

Steve

wallabybob

@pstokman:

My pfSense appliance has multiple interfaces, 6 in total of which 4 will be used for LAN. I want to combine all 4 interfaces as one, so I can connect our switches to the pfSense. The way I see it, this should allow inter-switch networking through pfSense (client 1 on switch 1 connects to client 2 on switch 2, using pfSense).

This sounds to me more like an application for bridging than link aggregation which is (I believe) commonly used when you want to have multiple links between the same systems. (In this case the mention of switch 1 and switch 2 suggests the multiple links have different end points).

pstokman

@clarknova:

@pstokman:

Please tell me if my train of thought is correct and I just need to create a LAGG, hook up the two VLANs on it as parent interface and be done.

Did you try that? I'm not sure I follow your whole train of thought but it strikes me that this might be all you need to do. (keep in mind that some types of LAGG need to be configured on the switch as well).

If you're still having trouble it would be immensely useful to include a network diagram.

I haven't tried it, because I had my doubts about LAGG. I tried creating one, but the settings and their description made me think about it once more. Which setting would be best in such a case where switch configuration should be as little as possible? I'm a bit lost with all the explanations, and I'm not that bad in networking (I've followed and finished Cisco CCNA and had CCNP at school, done plenty of bonding on Linux and work with virtual machines a lot).

@wallabybob:

@pstokman:

My pfSense appliance has multiple interfaces, 6 in total of which 4 will be used for LAN. I want to combine all 4 interfaces as one, so I can connect our switches to the pfSense. The way I see it, this should allow inter-switch networking through pfSense (client 1 on switch 1 connects to client 2 on switch 2, using pfSense).

This sounds to me more like an application for bridging than link aggregation which is (I believe) commonly used when you want to have multiple links between the same systems. (In this case the mention of switch 1 and switch 2 suggests the multiple links have different end points).

I have now created a bridge and I do think this is the easiest solution to my situation. From what I've read about Link Aggregation when I configured the Netgear switches at our data center for HA is that it's used for inter-switching, making it better to have dual or better link between two switches preventing network looping.

The data center currently has only a single uplink to the router (soon be replaced with pfSense appliance) because of network looping. I think I would need a LAGG on that to avoid it. If one switch fails, the other would still provide an uplink to the internet. Now it doesn't: SPOF waiting to fail. LACP mode will do fine on that part, because the switch has the same name. At the office however, we have mixed brands and a 'dumb' switch, making it harder to figure out what setting is required on the switches and such.

Reminds me to fix their config too to allow a link between them, in case the pfSense appliance ever fails; would be bad to not be able to access the rest of the network that is on another switch than you are.

Thanks for your insights and expertise.

pstokman

Ok, I didn't check the VLAN interface assignment after I created the bridge and left for home. Just did and I can not select a bridge as VLAN parent interface. The appliance hasn't rebooted yet, which may be required for it to show up properly, but would it be ok? I'll make a crude drawing of the network I have.


Switch 1 ----------------,
  VoIP, data. PoE         |
                                    |
Switch 2 ----------------|-- pfSense appliance
  Data, optional VoIP   |
                                    |
Switch 3 ----------------|
  Data, optional VoIP   |
                                    |
Switch 4 ----------------'
  Data, unmanaged

Switches 1-3 are managed switches, of which I can only control switches 2 and 3. The PoE switch is from our VoIP supplier and I am unable to access it (don't know the IP, nor the username/password).

The end result should be that the managed switches can switch data through the pfSense and keep the VLAN data separate, so VoIP traffic won't bleed to the data network and vice versa, while being able to just hook up a VoIP phone with external power to any switch and still be able to make calls and use it's PC uplink.

clarknova

Why not do something like this?


Switch 1 -----
                       \
Switch 3 ----- Switch 2 ---- pfsense
                       /
Switch 4 -----

Supermule

Single point of failure….

clarknova

True, but he has to weigh the risk of that single point of failure against the possibility of degraded performance as a result of filtering everything through pfsense's bridge. That, and the possibility of substituting switch 3 in its place if switch 2 were to fail.

I'm not going to say that one way or the other is correct, only that there are obvious advantages to alternate setups, and the admin gets to decide which works for him.

podilarius

This definitely sounds like a job for bridge. If you have the IPs to spare, you can assign each an IP if you like. LAGG is more like Linux bonding and not Linux or Unix bridges. Using a bridge is more like turning pfsense into a smart switch that is capable of filtering or not. How ever, if all your computers don't have a NIC in at least 2 switches, then you still have a single point of failure for part of the network. It does allow for making sure that not everyone is down.

I find that rarely do switches fail. It is usually environmental (ie lightnight strike, fire, and the like) and that usually will take out all switches and computers in the net. Not saying they don't, because I have had it happen a couple of times. They were cheap though.

Also if you have smart switches, you can do STP in there and not even have to worry about the pfsense (except for creating its LAGG for failover) config.

Either way, pfsense and most all *nix systems can do what you need. A nice system will be needed if you want to push gigabit or faster line speeds.

pstokman

@clarknova:

Why not do something like this?


Switch 1 -----
                       \
Switch 3 ----- Switch 2 ---- pfsense
                       /
Switch 4 -----

I do plan to have all three switches hooked up to each other at some point in time, but I don't have the time to configure them for it now. Switch 4 is not in the same rack, it's a 16 ports switch hooking up the servers we have as backup to the normal LAN. So eventually, switch 1 will have a link to 2 and 3, 2 will have a link to 3 and thus completing a loop. With LAG on the switches, this will prevent looping of traffic and freaking out the switches. Until then, all should go through pfSense.

@podilarius:

This definitely sounds like a job for bridge. If you have the IPs to spare, you can assign each an IP if you like. LAGG is more like Linux bonding and not Linux or Unix bridges. Using a bridge is more like turning pfsense into a smart switch that is capable of filtering or not. How ever, if all your computers don't have a NIC in at least 2 switches, then you still have a single point of failure for part of the network. It does allow for making sure that not everyone is down.

I find that rarely do switches fail. It is usually environmental (ie lightnight strike, fire, and the like) and that usually will take out all switches and computers in the net. Not saying they don't, because I have had it happen a couple of times. They were cheap though.

Also if you have smart switches, you can do STP in there and not even have to worry about the pfsense (except for creating its LAGG for failover) config.

Either way, pfsense and most all *nix systems can do what you need. A nice system will be needed if you want to push gigabit or faster line speeds.

I do have IPs to spare, but I don't have any (only one) for the tagged VLAN2 network. But what will multiple IPs offer as advantage? How will it help with switching data from one system to another?

As for SPOF, there always will be one, Systems are connected to only one switch at a time, so if a switch fails, they are down. But the rest will be able to keep working. However, I didn't create this topic to talk about SPOFs, I created it to get the best configuration option possible to get traffic going between the switches with voice and data traffic in separate VLANs.

stephenw10

Hmm, this thread seems confusing. Reading your first post I had assumed you wanted to group your interfaces in a lagg in order to get increased bandwidth between vlans when they are routed through pfSense. Now it seems you just want greater availability.
You say you don't want a single point of failure but, as podilarius said, you will still have one if all your traffic is routed through your pfsense box.

Consider which is more likely to fail a switch or your pfSense box?

I would guess the switch will outlast almost any server/appliance.
If you are really that concerned about availability then consider having a CARP failover setup.

Steve

pstokman

It's not so much as higher bandwidth as higher availability, I just want to hook the switches together through pfSense. The normal network isn't that hard, bridge the physical interfaces or make it a group and all should go fine. But with our VoIP VLAN in the picture, it's a bit more tricky than that. And because there is also an unmanaged switch on it, I'm not entirely sure if LACP or CARP will work with it. The gateway IP has to be accessible on all the ports for the data network.

podilarius

Using an IP for each NIC was not to help routing, but to help with the management of the box. The bridge will do the data switching just fine.

pstokman

@podilarius:

Using an IP for each NIC was not to help routing, but to help with the management of the box. The bridge will do the data switching just fine.

Ok, so bridge it and no VLAN will get it done? Will do that. Worst that can happen is that VLAN2 isn't working properly, but that doesn't work now anyway.

Thanks everyone for your help.

podilarius

You should still be able to use a VLAN and set the parent as the bridge interface. I have not tried it, but it might work.

pstokman

You can't select a bridge interface as VLAN parent, only physical interfaces. So all I have is em0 - em5, no bridge0, opt6 or anything. I tried that already ;).

wallabybob

Perhaps you want to bridge your VLAN interfaces (create a bridge with members VLAN interfaces) rather than VLAN a bridge!

pstokman

Tough luck. Only physical interfaces can be bridged. I can't select virtual OPTx interfaces. And I would still have the issue of the parent interface, if that would get disconnected, the whole VLAN falls apart and fails. I'll be able to test the bridge this week or early next week, as my boss wants it in use before I go on vacation (which is in two weeks :)). I'll report back with the results once it's in production use.