Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
In your example the "Core Switch" is functioning as a Layer 2 switch.
You'll have to read the comment. Yes, that was the intent of this iteration. In this example the "core switch" is functioning as layer 2 switch (ignoring its L3 capabilities) with routing the occurring in PFsense as johnpoz suggested early on. My concern with this iteration is due to our physical layout requiring router>switch>switch>device.
Thanks for your suggestions and comments btw, I'm just trying to understands the pro's and con's of the different ways this can be accomplished.
-
All you have to do is tag the VLANs from the core down to the edge switches.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
There may be a problem with this layout below and potential broadcast storms? See below. We have fiber going between buildings on site, all the fiber between the buildings goes to central location where the core switch is....
Why do you think you will have a broadcast storm? Broadcast storms generally result from having loops. I don't see any loops and even if you had some, you'd use spanning tree protocol to block them. That the L2 switches are in another building is irrelevant. They'd behave exactly the same as if they were in the same rack. If you use L3 (routing) to separate the sites, then you will need separate IP address ranges for each VLAN on each site.
-
@JKnott said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Why do you think you will have a broadcast storm?
Someone who knows more about this stuff than me (but perhaps less than you all) suggested it could be a big issue here due to the chaining of switches. I also read about STP back then it would seem that is exactly what STP sets to mitigate....with STP traffic would find it's way through the trunks to the best known destinations?
-
In that configuration you should be more worried about the core failing and taking your whole network with it than loops.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
All you have to do is tag the VLANs from the core down to the edge switches.
Yeah I think I'm going to do it this way.... Trunking all the VLAN's through a 10GBE connection or two to a pfsense box.
Problems I see with the other solution, the "/30 to pfsense and L3 on the Cisco" solution, is if I use pfsense at all on the edge as firewall, then I am already there in that GUI managing rules for WAN and whatever else. Then I am also digging around, probably a lot, on the Cisco in not-quite-IOS (because it is Small Business/SG series) maintaining a bunch of firewallish rules as ACL's between the VLAN's. From what I've read there (Cisco ACL's on SG*), people really have a hard time accomplishing what they want between the vlans because what they really need is..... a firewall.
Long story short... I've ordered a 10GBE nic for the test pfsense box so that it can have a larger connection to network.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
@JKnott said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Why do you think you will have a broadcast storm?
Someone who knows more about this stuff than me (but perhaps less than you all) suggested it could be a big issue here due to the chaining of switches. I also read about STP back then it would seem that is exactly what STP sets to mitigate....with STP traffic would find it's way through the trunks to the best known destinations?
There are a couple of issues here. The first, loops, we can disregard as you don't have any (that I can see). The other is the more devices, the more broadcast/multicast traffic there will be. This is entirely dependent on the number of devices connected, not the number of switches. It was even a concern back in the dark ages, when Ethernet meant a coaxial cable linear bus and there were no switches. The problem with broadcasts, but not so much multicasts, is that every devices is interrupted by every broadcast packet on the network. This means each device has to stop what it's doing, receive the broadcast and then determine if it has to do anything. This is greatly reduced with multicasts, where a device only listens for the multicasts it's interested in. So, for example, a regular computer would not be listening to "all routers" multicasts. Also, some switches can filter multicasts, so that they only pass them to devices that are actually listening for those multicasts.
What spanning tree does is block any redundant paths, so that loops can't form. It will also respond to changes in the network, so that if one path fails a previously blocked redundant path is opened. It does this by constantly evaluating the various paths to a "root" switch and the best path is automatically enabled. All other paths are blocked. So, unless you have redundant paths, spanning tree does nothing.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
In that configuration you should be more worried about the core failing and taking your whole network with it than loops.
Definitely. Although, the SG550X we have is stackable. If only we have the money to purchase the other one. :) Soon. But yeah, that's the plan. In the mean time, I guess Cisco has a pretty quick replacement program for them if it were to fail. If it were to fail and we have to wait 24-72 hours for replacement, that'll be the day I guarantee we get the 2nd for stackable (and probably a HA setup for pfsense at the same time).
-
Depends on how much money the downtime would cost.
I would be more comfortable with a stacked core and LACP to each edge, HA on the firewall, and a cold spare for each.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Definitely. Although, the SG550X we have is stackable. If only we have the money to purchase the other one. :) Soon. But yeah, that's the plan
Then you will have potential loops and need spanning tree. You'll also need 2 connections to each building switch, 1 from each core switch. You'll also have to configure spanning tree priority to determine the root. I expect you'd make the 2 core switches root.
PfSense running on Qotom mini PC
i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
UniFi AC-Lite access pointI haven't lost my mind. It's around here...somewhere...
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
I would be more comfortable with a stacked core and LACP to each edge, HA on the firewall, and a cold spare for each.
Yes me too. LACP possible with these (SG550XG-8F8T) we'll just need to first hit those unexpected hiccups OR have revenue/operations enough to pay for HA and spares.
First "availability"... then "high availability". Budget conscious startups are hard sometimes. You can only lay out the risks and suggest what can be done ahead of time to avoid them. When SHTF, then they'll remember and break out the checkbook to buy equipment. Right now it's more like "Oh that's great that we'll have that option in the future." :) -
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
First "availability"... then "high availability". Budget conscious startups are hard sometimes. You can only lay out the risks and suggest what can be done ahead of time to avoid them. When SHTF, then they'll remember and break out the checkbook to buy equipment. Right now it's more like "Oh that's great that we'll have that option in the future." :)
They have to determine how long they can afford to be down for. One company I used to work for had spares handy, should it be necessary to replace a failed unit at a customer site. That wasn't needed very often. I don't recall ever having to replace a failed switch or router, in the 3 years I was there. This was with Adtran gear. There was some other equipment, used with phone systems, that failed occasionally.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
From what I've read there (Cisco ACL's on SG*), people really have a hard time accomplishing what they want between the vlans because what they really need is..... a firewall.
And there you go ;) What I said was reason back in he beginning of this thread.. You go into almost any decent sized corp and what you will find is when they do downstream routing - they are not doing any filtering between vlans... "To Much Work" ;)
How do you think these worms go nuts in a corp.. They don't put in proper controls... To be honest most corps should be running private vlans... Why does user A box need to directly be able to talk to user B box?? They don't!!
In every customer network have ever walked into - their L3 that are routing are not doing any acls, and all their different vlans be it server, user, infrastructure are all wide open between each other. if your going to run it that way - why are you even segmented?
As long as pfsense can handle the routing - then yes simple solution is let it do the routing... Put it a downstream to let it do your intervlan routing/firewalling and have another one for your edge, etc.
There are multiple ways to skin the cat..
-
@JKnott Then you will have potential loops and need spanning tree.
Don't use STP use LACP.
-
@johnpoz said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
From what I've read there (Cisco ACL's on SG*), people really have a hard time accomplishing what they want between the vlans because what they really need is..... a firewall.
And there you go ;) What I said was reason back in he beginning of this thread.. You go into almost any decent sized corp and what you will find is when they do downstream routing - they are not doing any filtering between vlans... "To Much Work" ;)
How do you think these worms go nuts in a corp.. They don't put in proper controls... To be honest most corps should be running private vlans... Why does user A box need to directly be able to talk to user B box?? They don't!!
I've seen the Cisco IOS configuration of a Fortune 100 type company with hundreds of sites.... thousands of Cisco devices. Basically super flat network per site. A handful of highly consolidated VLANS (i.e. VOIP) . Default passwords. Couldn't believe it. Talked to one of the main networking guys and he was like... "we have like 15 people. If the location/room the equipment is in isn't totally secure, we've got bigger problems. If the configurations get overly complicated and cause outages because someone fat fingers an ip or port and we lose money, someone fired. If we need to rush in replacement hardware and need to reconfigure.... etc. Keep it simple."
-
@johnpoz said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
As long as pfsense can handle the routing - then yes simple solution is let it do the routing... Put it a downstream to let it do your intervlan routing/firewalling and have another one for your edge, etc.Wait...what?
Internet
|
pfsense01.local edge firewall
|
pfsense02.local firewall
|
coreswitch??
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Keep it simple."
And then they wonder why 1 single idiot user clicking on an ad can take down their whole network... They all talk a big game up at the cso level, etc. But yeah some of these big corp networks are just a joke...
What the point of spending the bucks to put in this fancy cisco gear if what your doing amounts to the feature set of a "smart" switch...
Which is why you have to make the choice - if you want to easy firewall, then use pfsense to route between your vlans - be it you fire up another another one in the core, or just route at the edge.. Or are you going to take the time to actually do it correctly at your L3... If your not - then you might as well just do a big fat flat network and not have to worry about the routing at all.
-
@johnpoz said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Keep it simple."
Which is why you have to make the choice - if you want to easy firewall, then use pfsense to route between your vlans - be it you fire up another another one in the core, or just route at the edge.. Or are you going to take the time to actually do it correctly at your L3... If your not - then you might as well just do a big fat flat network and not have to worry about the routing at all.
In pfsense land, one thing I've read as a reason for segmenting, at the least in terms of provisioning IP's along CIDR/subnet lines, is if you're using IDS then you can filter and target logs better. Another is "network ACL's" ... but yeah.... it sounds great, but maintaining this doesn't seem like something a small shop would be doing very well beyond that first day or two they set it up. The IDS logs I think I would actually use a lot, or at least want to narrow things down quickly on alert.
