Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?
-
Mostly linux here. It's the MS stuff that gives me headache while I learn to navigate through it.... and networking planning at this level too, I suppose :P
Anyway I can manage the DHCP pools on Cisco through IOS CLI no problem... the SG CLI is actually slower/more of a PITA than CLI for that IMO. It just would have been nicer to manage all the DHCP pools through pfsense gui to save one more interface to have to log in to and navigate around in.
So here's the updated image... does the pink stuff by PFsense still look ok?
So I will:
- Let pfsense be super authorative top dog DNS for the overall network**
- Let pfsense manage OpenVPN ... is ok like this?
- Let pfsense manage http proxy / filter (squid?) for LAN traffic going out through WAN
- Let unifi controller manage radius
- Let unifi controller manage captive portal
**I originally wanted DNS at pfsense because I was thinking in terms of blacklists like DNSMASQ does, which is still good... but I forgot with pfsense (i.e. freebsd PF) I'm sure there is some plugin or script or something that leverages "blackhole". I forget how I used to do this long ago with freebsd and pf...something about tables. I'm sure there is some pfsense way of accomplishing this. Goal = kill traffic to known bad IP's. MS of course wants to manage DNS for everything in its "domain" so I let it do that, but I want MS to get it's DNS resolutions from pfsense.
-
I would, personally, avoid anything 10dot like the plague.
If you want a /20 I would use something random like 172.19.16.0/20
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
but I want MS to get it's DNS resolutions from pfsense.
Why? I would let the firewall be a firewall. If you want a DNS blacklist server I would put it on the inside (hint: it could be another pfSense software instance doing just that). I would also consider a separate node for squid. You could put it between the edge and the internal layer 3 switch on a couple transit interfaces.
Having everything on the edge device is a disease and leads to problems with VPN traffic and policy routing/multi-wan and other issues.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Let unifi controller manage radius
You mean run radius on the same box? The unifi controller software can not do radius..It can only point to one - you could run radius on the same box I guess..
Sure you can have any downstream dns being authoritative for your domain(s) just forward to pfsense and then let unbound resolve.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
It's the MS stuff that gives me headache
I'm allergic to it too.
-
Trying to wrap my head around this a bit more. Found some tutorials that talk about this solution and "simple, just use VLAN TRUNKING!"... but... VLAN TRUNK would conflict with the /30 static route between the L3 switch and PFSENSE we're talking about here, no? See image... it's one (VLAN TRUNK) or the other (STATIC /30 route) right?
Assume orange connections are 10GBE and blue are 1GBE.
I would presume if you VLAN trunk the 10GBE vlans like that, then potentially the pipe to pfsense could get flooded with traffic as well.
Feels like I'm putting a bicycle on top of a car and riding the bike. :(
-
You can use either - you can have pfsense route the vlans, ie trunk all of them up to pfsense... Now your switch is only L2 and not doing any L3..
You do can do either - there is not right or wrong way.. You use the one that best suites your needs..
As to being a problem with hairpin - depends on how much intervlan traffic you have... But you can always use more uplink to router if sharing 1 physical link is a bottle neck, etc. etc..
But yes a bottle neck in required intervlan traffic is normally a reason to use a downstream router.
BTW: your going to trunk all your vlans up to pfsense, then the transit becomes moot.
-
-
@johnpoz said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
You can use either - you can have pfsense route the vlans, ie trunk all of them up to pfsense... Now your switch is only L2 and not doing any L3..
You do can do either - there is not right or wrong way.. You use the one that best suites your needs..
I guess my overnight concern was if I use the /30 between L3 switch and PFSENSE, what's that going to look like on PFSENSE configuration side.
Will I still be able to define gateways for VLAN's (i.e. 10.0.10.1, 10.0.20.1, etc.) at/in pfsense? Will I see see the VLAN tagged traffic and be able to setup firewall rules per-device-in-a-vlan (i.e. a windows wsus.mydomain.com server in VLAN20 "SERVERS" can only go outbound to wsus.microsoft.com port 80, etc.)
-
Dude I already went over this your going to let your downstream do all the routing then there are NO VLANS on pfsense... all there is the transit, and routes... You do not create any vlan interfaces on pfsense when your using downstream router to get to those networks - it will NEVER see the tags..
Your gateways for vlans will be on your L3 doing routing via SVI setups on it..
And yes you can still firewall them, you would just do all the firewall on the transit interface - and you would not have any control over devices talking intervlan - only control of them going to the internet.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
VLAN TRUNK would conflict with the /30 static route between the L3 switch and PFSENSE we're talking about here, no?
Why would they conflict? They'd conflict only if you tried assigning the same addresses to multiple connections. Each VLAN would have it's own address range.
-
I dug into the Layer 3 CISCO SG550X we have as "Core Switch" and capabilities for layer 3 VLAN routing and not too excited about managing the intervlan routing there as layer 3 on the Cisco. So looking back now at your suggestion @johnpoz to pass everything up from core switch to Pfsense as L2 / tagged VLANs. If I have to I can beef up and/or upgrade PFsense appliance to support more traffic (Chelsio 10GBE nics are cheap!)
But then I remembered why we were planning to route VLANs via L3 at the core switch. There may be a problem with this layout below and potential broadcast storms? See below. We have fiber going between buildings on site, all the fiber between the buildings goes to central location where the core switch is....
Problem?
-
Please don't take offense. Do you even know what Layers 2 & 3 are?
Where do you want the routing to occur (Layer 3)? On the firewall or on the switch? Look carefully at the diagram I already posted. Pay particular attention to where the IP addresses (Layer 3 addresses) are on the various interfaces.
In your example the "Core Switch" is functioning as a Layer 2 switch.
You would typically make the determination to let the switch route the traffic for performance reasons. You would typically want the firewall to route between subnets if you need firewalling services between the segments, such as a DMZ. You can do a combination of both, such as in the diagram I posted above.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
In your example the "Core Switch" is functioning as a Layer 2 switch.
You'll have to read the comment. Yes, that was the intent of this iteration. In this example the "core switch" is functioning as layer 2 switch (ignoring its L3 capabilities) with routing the occurring in PFsense as johnpoz suggested early on. My concern with this iteration is due to our physical layout requiring router>switch>switch>device.
Thanks for your suggestions and comments btw, I'm just trying to understands the pro's and con's of the different ways this can be accomplished.
-
All you have to do is tag the VLANs from the core down to the edge switches.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
There may be a problem with this layout below and potential broadcast storms? See below. We have fiber going between buildings on site, all the fiber between the buildings goes to central location where the core switch is....
Why do you think you will have a broadcast storm? Broadcast storms generally result from having loops. I don't see any loops and even if you had some, you'd use spanning tree protocol to block them. That the L2 switches are in another building is irrelevant. They'd behave exactly the same as if they were in the same rack. If you use L3 (routing) to separate the sites, then you will need separate IP address ranges for each VLAN on each site.
-
@JKnott said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Why do you think you will have a broadcast storm?
Someone who knows more about this stuff than me (but perhaps less than you all) suggested it could be a big issue here due to the chaining of switches. I also read about STP back then it would seem that is exactly what STP sets to mitigate....with STP traffic would find it's way through the trunks to the best known destinations?
-
In that configuration you should be more worried about the core failing and taking your whole network with it than loops.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
All you have to do is tag the VLANs from the core down to the edge switches.
Yeah I think I'm going to do it this way.... Trunking all the VLAN's through a 10GBE connection or two to a pfsense box.
Problems I see with the other solution, the "/30 to pfsense and L3 on the Cisco" solution, is if I use pfsense at all on the edge as firewall, then I am already there in that GUI managing rules for WAN and whatever else. Then I am also digging around, probably a lot, on the Cisco in not-quite-IOS (because it is Small Business/SG series) maintaining a bunch of firewallish rules as ACL's between the VLAN's. From what I've read there (Cisco ACL's on SG*), people really have a hard time accomplishing what they want between the vlans because what they really need is..... a firewall.
Long story short... I've ordered a 10GBE nic for the test pfsense box so that it can have a larger connection to network.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
@JKnott said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Why do you think you will have a broadcast storm?
Someone who knows more about this stuff than me (but perhaps less than you all) suggested it could be a big issue here due to the chaining of switches. I also read about STP back then it would seem that is exactly what STP sets to mitigate....with STP traffic would find it's way through the trunks to the best known destinations?
There are a couple of issues here. The first, loops, we can disregard as you don't have any (that I can see). The other is the more devices, the more broadcast/multicast traffic there will be. This is entirely dependent on the number of devices connected, not the number of switches. It was even a concern back in the dark ages, when Ethernet meant a coaxial cable linear bus and there were no switches. The problem with broadcasts, but not so much multicasts, is that every devices is interrupted by every broadcast packet on the network. This means each device has to stop what it's doing, receive the broadcast and then determine if it has to do anything. This is greatly reduced with multicasts, where a device only listens for the multicasts it's interested in. So, for example, a regular computer would not be listening to "all routers" multicasts. Also, some switches can filter multicasts, so that they only pass them to devices that are actually listening for those multicasts.
What spanning tree does is block any redundant paths, so that loops can't form. It will also respond to changes in the network, so that if one path fails a previously blocked redundant path is opened. It does this by constantly evaluating the various paths to a "root" switch and the best path is automatically enabled. All other paths are blocked. So, unless you have redundant paths, spanning tree does nothing.
