Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Would rather do DHCP on Cisco than the MS stuff.
From someone that I take has never done dhcp on cisco ;) hehehe MS is the way to go - been doing this since before AD.. back in the days of 3.11... MS does dhcp just fine - and it makes management of your members easy, because the dhcp server can register your members in DNS..
When you run dhcp on something other than your AD - it just complicates it for no reason. The MS dhcp server is very feature reach and can handle 1000's and 1000's of clients with 100's of scopes..
For pfsense to allow your downstream network out - you will have to setup a gateway pointing to the IP of your cisco IP on the transit /30... Once you create routes to the downstream networks - you can do it with 1 if all your downstream fall under a single cidr... But you will have to adjust the firewall rules on the transit interface of pfsense to allow the downstream network to go where you want them to go on the internet - could be an any any, but source would have to be adjusted from the default "lan net"
Once you create the routes in pfsense - auto outbound nat will allow for them to be natted to pfsense wan IP.
-
Mostly linux here. It's the MS stuff that gives me headache while I learn to navigate through it.... and networking planning at this level too, I suppose :P
Anyway I can manage the DHCP pools on Cisco through IOS CLI no problem... the SG CLI is actually slower/more of a PITA than CLI for that IMO. It just would have been nicer to manage all the DHCP pools through pfsense gui to save one more interface to have to log in to and navigate around in.
So here's the updated image... does the pink stuff by PFsense still look ok?
So I will:
- Let pfsense be super authorative top dog DNS for the overall network**
- Let pfsense manage OpenVPN ... is ok like this?
- Let pfsense manage http proxy / filter (squid?) for LAN traffic going out through WAN
- Let unifi controller manage radius
- Let unifi controller manage captive portal
**I originally wanted DNS at pfsense because I was thinking in terms of blacklists like DNSMASQ does, which is still good... but I forgot with pfsense (i.e. freebsd PF) I'm sure there is some plugin or script or something that leverages "blackhole". I forget how I used to do this long ago with freebsd and pf...something about tables. I'm sure there is some pfsense way of accomplishing this. Goal = kill traffic to known bad IP's. MS of course wants to manage DNS for everything in its "domain" so I let it do that, but I want MS to get it's DNS resolutions from pfsense.
-
I would, personally, avoid anything 10dot like the plague.
If you want a /20 I would use something random like 172.19.16.0/20
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
but I want MS to get it's DNS resolutions from pfsense.
Why? I would let the firewall be a firewall. If you want a DNS blacklist server I would put it on the inside (hint: it could be another pfSense software instance doing just that). I would also consider a separate node for squid. You could put it between the edge and the internal layer 3 switch on a couple transit interfaces.
Having everything on the edge device is a disease and leads to problems with VPN traffic and policy routing/multi-wan and other issues.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Let unifi controller manage radius
You mean run radius on the same box? The unifi controller software can not do radius..It can only point to one - you could run radius on the same box I guess..
Sure you can have any downstream dns being authoritative for your domain(s) just forward to pfsense and then let unbound resolve.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
It's the MS stuff that gives me headache
I'm allergic to it too.
-
Trying to wrap my head around this a bit more. Found some tutorials that talk about this solution and "simple, just use VLAN TRUNKING!"... but... VLAN TRUNK would conflict with the /30 static route between the L3 switch and PFSENSE we're talking about here, no? See image... it's one (VLAN TRUNK) or the other (STATIC /30 route) right?
Assume orange connections are 10GBE and blue are 1GBE.
I would presume if you VLAN trunk the 10GBE vlans like that, then potentially the pipe to pfsense could get flooded with traffic as well.
Feels like I'm putting a bicycle on top of a car and riding the bike. :(
-
You can use either - you can have pfsense route the vlans, ie trunk all of them up to pfsense... Now your switch is only L2 and not doing any L3..
You do can do either - there is not right or wrong way.. You use the one that best suites your needs..
As to being a problem with hairpin - depends on how much intervlan traffic you have... But you can always use more uplink to router if sharing 1 physical link is a bottle neck, etc. etc..
But yes a bottle neck in required intervlan traffic is normally a reason to use a downstream router.
BTW: your going to trunk all your vlans up to pfsense, then the transit becomes moot.
-
-
@johnpoz said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
You can use either - you can have pfsense route the vlans, ie trunk all of them up to pfsense... Now your switch is only L2 and not doing any L3..
You do can do either - there is not right or wrong way.. You use the one that best suites your needs..
I guess my overnight concern was if I use the /30 between L3 switch and PFSENSE, what's that going to look like on PFSENSE configuration side.
Will I still be able to define gateways for VLAN's (i.e. 10.0.10.1, 10.0.20.1, etc.) at/in pfsense? Will I see see the VLAN tagged traffic and be able to setup firewall rules per-device-in-a-vlan (i.e. a windows wsus.mydomain.com server in VLAN20 "SERVERS" can only go outbound to wsus.microsoft.com port 80, etc.)
-
Dude I already went over this your going to let your downstream do all the routing then there are NO VLANS on pfsense... all there is the transit, and routes... You do not create any vlan interfaces on pfsense when your using downstream router to get to those networks - it will NEVER see the tags..
Your gateways for vlans will be on your L3 doing routing via SVI setups on it..
And yes you can still firewall them, you would just do all the firewall on the transit interface - and you would not have any control over devices talking intervlan - only control of them going to the internet.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
VLAN TRUNK would conflict with the /30 static route between the L3 switch and PFSENSE we're talking about here, no?
Why would they conflict? They'd conflict only if you tried assigning the same addresses to multiple connections. Each VLAN would have it's own address range.
-
I dug into the Layer 3 CISCO SG550X we have as "Core Switch" and capabilities for layer 3 VLAN routing and not too excited about managing the intervlan routing there as layer 3 on the Cisco. So looking back now at your suggestion @johnpoz to pass everything up from core switch to Pfsense as L2 / tagged VLANs. If I have to I can beef up and/or upgrade PFsense appliance to support more traffic (Chelsio 10GBE nics are cheap!)
But then I remembered why we were planning to route VLANs via L3 at the core switch. There may be a problem with this layout below and potential broadcast storms? See below. We have fiber going between buildings on site, all the fiber between the buildings goes to central location where the core switch is....
Problem?
-
Please don't take offense. Do you even know what Layers 2 & 3 are?
Where do you want the routing to occur (Layer 3)? On the firewall or on the switch? Look carefully at the diagram I already posted. Pay particular attention to where the IP addresses (Layer 3 addresses) are on the various interfaces.
In your example the "Core Switch" is functioning as a Layer 2 switch.
You would typically make the determination to let the switch route the traffic for performance reasons. You would typically want the firewall to route between subnets if you need firewalling services between the segments, such as a DMZ. You can do a combination of both, such as in the diagram I posted above.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
In your example the "Core Switch" is functioning as a Layer 2 switch.
You'll have to read the comment. Yes, that was the intent of this iteration. In this example the "core switch" is functioning as layer 2 switch (ignoring its L3 capabilities) with routing the occurring in PFsense as johnpoz suggested early on. My concern with this iteration is due to our physical layout requiring router>switch>switch>device.
Thanks for your suggestions and comments btw, I'm just trying to understands the pro's and con's of the different ways this can be accomplished.
-
All you have to do is tag the VLANs from the core down to the edge switches.
-
@Jpub said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
There may be a problem with this layout below and potential broadcast storms? See below. We have fiber going between buildings on site, all the fiber between the buildings goes to central location where the core switch is....
Why do you think you will have a broadcast storm? Broadcast storms generally result from having loops. I don't see any loops and even if you had some, you'd use spanning tree protocol to block them. That the L2 switches are in another building is irrelevant. They'd behave exactly the same as if they were in the same rack. If you use L3 (routing) to separate the sites, then you will need separate IP address ranges for each VLAN on each site.
-
@JKnott said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
Why do you think you will have a broadcast storm?
Someone who knows more about this stuff than me (but perhaps less than you all) suggested it could be a big issue here due to the chaining of switches. I also read about STP back then it would seem that is exactly what STP sets to mitigate....with STP traffic would find it's way through the trunks to the best known destinations?
-
In that configuration you should be more worried about the core failing and taking your whole network with it than loops.
-
@Derelict said in Planning to use PFsense with Cisco L3 core router and Unifi for L2... does this look ok? suggestions?:
All you have to do is tag the VLANs from the core down to the edge switches.
Yeah I think I'm going to do it this way.... Trunking all the VLAN's through a 10GBE connection or two to a pfsense box.
Problems I see with the other solution, the "/30 to pfsense and L3 on the Cisco" solution, is if I use pfsense at all on the edge as firewall, then I am already there in that GUI managing rules for WAN and whatever else. Then I am also digging around, probably a lot, on the Cisco in not-quite-IOS (because it is Small Business/SG series) maintaining a bunch of firewallish rules as ACL's between the VLAN's. From what I've read there (Cisco ACL's on SG*), people really have a hard time accomplishing what they want between the vlans because what they really need is..... a firewall.
Long story short... I've ordered a 10GBE nic for the test pfsense box so that it can have a larger connection to network.
