Layer 3 Cisco Switch & pfSense Design Assistance
-
I've got a Cisco 3750E 48 port PoE switch that I'm using at home. Currently, I'm on a flat VLAN and subnet. I'd like to create the following:
VLAN 10: Transit between pfSense and Cisco Switch (10.0.0.0/30)
VLAN 11: Home Server/Management Subnet (10.0.1.0/24)
VLAN 12: Home Automation (10.0.2.0/24)
VLAN 13: Home Media (10.0.3.0/24)
VLAN 14: Home LAN (10.0.4.0/24)
VLAN 15: Home WLAN (10.0.5.0/24)
VLAN 16: Home Phone (10.0.6.0/24)VLAN 666: Guests (192.168.254.0/24)
VLAN 999: Work IPSec (10.89.25.0/24)I'm envisioning a design where I have an interface on the pfSense box that's 10.0.0.1/30 and on the switch as 10.0.0.2/30. However, the trick is that I need a guest VLAN interface (192.168.254.1/24) on the pfSense device. This is confusing to me as it means I need some sort of trunked interface on pfSense and the Cisco switch. I'm not sure how to set that up.
I don't want any Guest WiFi devices to have the ability to talk to anything on the 10.0.0.0/20 supernet. I suppose I could forget the idea of a Guest Network interface on pfSense. I could implement the interface on the switch and make my life easier. Then I could add an ACL to that VLAN interface that blocks destination of RFC1918 addresses.
The other thing I'm trying to understand is pfSense's method of static routing. I've watched a couple YouTube videos and they're telling me I have to create a new "gateway" (and only one gateway?). I'm assuming this gateway is the IP of the Cisco switch? Then, I need to create static routes for all of my internal subnets pointing at that single gateway? Then NAT rules?
I'm looking for the easiest method of accomplishing my goal, but also something that is robust. I have OpenVPN clients running on my pfSense, so it would be nice to tunnel the Guest traffic out one of the clients. Also, I'm not sure how my rules would be affected. . I don't think pfSense has been updated to work as an "ip helper" for Cisco switches. I mean that, I can't have multiple subnet/VLAN DHCP requests forwarding to pfSense and pfSense having multiple DHCP pools for those devices on those particular subnets. I'd have to update static DNS entries as well (not a problem). Are there any other gotchas? I know for my Chromecast to work, I've got to do some fiddling with directed broadcasts in Cisco on certain VLAN interfaces.
Thank you!
-
Why do you need a guess VLAN on the pfsense box? I run a Cisco layer 3 switch and I run my guess VLAN on the layer 3 switch. I have access lists setup on the switch to limit access to and from the guess VLAN.
There is only one LAN on the pfsense box and it has an IPv4 /30 which talks to the layer 3 switch. With the pfsense being in a /30 network only traffic destined for the internet ever reaches the pfsense box. All local traffic stays local.
On pfsense you need to add a route statement for each network setup on the layer 3 switch except of course the /30 network because it is directly connected. You also need to add access lists on pfsense for all the networks on the layer 3 switch otherwise they will be blocked at the firewall.
-
You can trunk a layer 2 VLAN to the switch along with the layer 3 transit network (VLAN 10). Just don't enable layer 3 on the switch for VLAN 666 and all traffic will have to go through pfSense (and the firewall there) to access anything else.
Create VLANs 10 and 666 on the pfSense physical interface (ethX)
Assign GUEST to VLAN 666 on ethX
Assign TRANSIT to VLAN 10 on ethX
Configure your guest network as you'd like
Create a gateway for 10.0.0.2 called TRANSIT_GW
Create a static route for 10.0.0.0/20 to TRANSIT_GW
vlan database
vlan 10,11,666int vlan 10
name TRANSIT
ip address 10.0.0.2 255.255.255.252int vlan 11
name HOME_SERVER
ip address 10.0.1.1 255.255.255.0int gigabit 1
switchport mode trunk
switchport trunk allowed vlan add 10,11,666Connect that to ethX
int gigabit 2
name GUEST_PORT
switchport mode access
switchport access vlan 666ip default-gateway 10.0.0.1
Lots of ways to do it. That's a basic one.
No, pfSense cannot do DHCP for all the layer 3 VLANs. You probably want to use something else for that. And probably DNS as well if you want it to be auto-populated by the DHCP server.
I would stay away from 10.0.0.0/8 and 192.168.0.0/24 and 192.168.1.0/24. And probably 192.168.168.0/24.
Pick some other random /20 like 172.23.224.0/20
-
So why are you wanting a downstream router in a home setup? Does your home automation network need to talk at wire speed to home media vlan? And your pfsense box does not have enough umph to route and firewall it?
Does the L3 switch your using provide ACLs between these networks? While I completely agree with the breakup of home networks. I sure don't want my iot devices on the same L2 as my other devices I want them isolated with a firewall, and logging of their traffic to be honest.
I have multiple segments in my own home network. Sure there is a guest wifi that is isolated from the rest of everything for my guests to use, etc. And stuff like my nest are on their own segment, etc. etc. And while the nest can talk to the internet they can not talk to anything else on my network, etc.
But you have a great easy to use firewall at your finger tips, that routes - other than sure packet speed between segments why would you not just leverage pfsense as your router/firewall between all your network?
You atleast understand that you need a transit to your downstream so that is good. And as already mentioned sure you can bring a specific network up to your pfsense for routing you don't have to do all your routing at your L3 switch.
As Derelict correctly points out doing such a setup also means you need something else for dhcp, etc. While this is great lab exercise I really don't see the point of such a setup in a home network.. Isn't it just much easier to let your pfsense box do this? If you have devices that need switching speed to each other.. What is the security concern of putting those devices on the same layer 2, like your PC to your storage box, etc.
-
So why are you wanting a downstream router in a home setup? Does your home automation network need to talk at wire speed to home media vlan? And your pfsense box does not have enough umph to route and firewall it?
The reason I'm looking at this design is that this is how most corporations implement their networks. My HomeAutomation network includes Amcrest IP webcams. I do have an app called "Tiny Cam Pro" installed on my cell phone and on my Android TV that can access the video and audio stream. I can setup a screen with multiple cameras at the same time. Granted, this isn't 1Gb wire speed, but I don't want any potential sources of lag. And my current pfSense hardware isn't exactly beefy. It's an Atom 330 with 2GB of RAM. Not a power house. The other reason to have the bulk of the network L3 switched on the switch is for pfSense upgrades. An upgrade shouldn't take down my ability to stream that webcam video internally, or my ability to stream music or video internally.
Does the L3 switch your using provide ACLs between these networks? While I completely agree with the breakup of home networks. I sure don't want my iot devices on the same L2 as my other devices I want them isolated with a firewall, and logging of their traffic to be honest.
The Cisco 3750E that I'm running does have ACL capabilities. It also has SPAN capabilities so I can Wireshark the VLAN interface for the HA network. I do have an ACL setup right now that blocks the HA network's ability to hit the internet. A device in that subnet/VLAN can hit other IPs internally (except for the phone network). That's needed so that I can stream the video and control HomeSeer from my home network and over my VPN.
I have multiple segments in my own home network. Sure there is a guest wifi that is isolated from the rest of everything for my guests to use, etc. And stuff like my nest are on their own segment, etc. etc. And while the nest can talk to the internet they can not talk to anything else on my network, etc.
I also have a Nest. It will be segmented into the HA network. You just reminded me that the ACL I have in place will need to be adjusted to allow the Nest access to the internet. I really don't like the Nest needing a cloud connection. I think the Ecobee thermostat can work entirely internally. I'll also need to create a DHCP scope with a reserved DHCP IP for the Nest so I can make that ACL work. More to think about…
But you have a great easy to use firewall at your finger tips, that routes - other than sure packet speed between segments why would you not just leverage pfsense as your router/firewall between all your network?
You atleast understand that you need a transit to your downstream so that is good. And as already mentioned sure you can bring a specific network up to your pfsense for routing you don't have to do all your routing at your L3 switch.
As Derelict correctly points out doing such a setup also means you need something else for dhcp, etc. While this is great lab exercise I really don't see the point of such a setup in a home network.. Isn't it just much easier to let your pfsense box do this? If you have devices that need switching speed to each other.. What is the security concern of putting those devices on the same layer 2, like your PC to your storage box, etc.
It always comes back to DHCP ease. The Cisco switch can handle all the DHCP duties I need. It can do static DHCP IPs and multiple pools. However, you're correct that having DHCP on pfSense is easy. I just wish pfSense could handle "ip helper-address" requests like Windows can. There was a thread on it, but I haven't seen any movement to upstream that work back to pfSense mainline.
The reason I want to segment all this stuff is security. There was a Black Hat talk on the vulnerability of IP cams in the world. I don't have my IP cameras forwarding to my outside address, but that doesn't mean I'm not paranoid. So if I can limit the device's ability to talk to the outside world, or limit a device to only talking to the outside world, I'd be much happier. Obviously I can do this with pfSense, but in order to keep my network up during pfSense upgrades, etc, having a separate "router" is the way to go.
You can trunk a layer 2 VLAN to the switch along with the layer 3 transit network (VLAN 10). Just don't enable layer 3 on the switch for VLAN 666 and all traffic will have to go through pfSense (and the firewall there) to access anything else.
Create VLANs 10 and 666 on the pfSense physical interface (ethX)
Assign GUEST to VLAN 666 on ethX
Assign TRANSIT to VLAN 10 on ethX
Configure your guest network as you'd like
Create a gateway for 10.0.0.2 called TRANSIT_GW
Create a static route for 10.0.0.0/20 to TRANSIT_GW
vlan database
vlan 10,11,666int vlan 10
name TRANSIT
ip address 10.0.0.2 255.255.255.252int vlan 11
name HOME_SERVER
ip address 10.0.1.1 255.255.255.0int gigabit 1
switchport mode trunk
switchport trunk allowed vlan add 10,11,666Connect that to ethX
int gigabit 2
name GUEST_PORT
switchport mode access
switchport access vlan 666ip default-gateway 10.0.0.1
Lots of ways to do it. That's a basic one.
No, pfSense cannot do DHCP for all the layer 3 VLANs. You probably want to use something else for that. And probably DNS as well if you want it to be auto-populated by the DHCP server.
I would stay away from 10.0.0.0/8 and 192.168.0.0/24 and 192.168.1.0/24. And probably 192.168.168.0/24.
Pick some other random /20 like 172.23.224.0/20
This is great! Thank you for explaining it better to me. I wasn't prepared for the pfSense method of creating an object/group for everything, even the "gateway" for a static route. I was mixing up Cisco and pfSense. Hopefully, this weekend, I can make these changes! Thank you!
-
"The other reason to have the bulk of the network L3 switched on the switch is for pfSense upgrades. An upgrade shouldn't take down my ability to stream that webcam video internally, or my ability to stream music or video internally."
While I agree with this for sure, what happens when you want to upgrade your switch firmware? ;) what is being used for internal dns? While you might not have an issue while your streaming a movie or music and you reboot pfsense.
When do you do your pfsense upgrades? I do them after hours or before household hours because I am up early.. Or sometimes whenever.. Reboot of pfsense never takes more than a couple of minutes, etc. If someone was watching a movie I wouldn't do the update then ;)
"The reason I want to segment all this stuff is security."
Completely and utterly agree with you 100% I just do not see doing it at the switch, which clearly while it has some basic ACL functionality does not have the ease of creating the exact firewall rules and logging of hits on these rules like your switch is going to have.As to how large companies do it - sure they have core L3 switches, I have supported many a large company.. They rarely firewall between their segments, even though they should!! Most often I see a large core switch, say a nexus 7k but there are no ACLs between segments. Sure they will have their services that are open to the public internet behind a firewall and isolated from their core network..
I really don't see that as any sort of reason to do a downstream in your home setup.. If your pfsense box can not handle the wire speed you need between segments, prob better to get a faster pfsense box ;) heheeh It will make your life much easier that is for sure. I have toyed with putting my sg300 in L3 mode and doing a downtream setup.. This would for sure give me way faster speeds between my segments. But the thing is I have my segments isolated for security. The ports I do have open between segments like printing, access to my plex. Pfsense can more than handle the speed needed.
More than happy to help you work out the details of such a setup, I just don't see the actual value in doing it is all ;)
