PfSense with L3 switch



  • Hi guys, I recently picked up a new L3 switch and am wondering if there is anyway (or if it's worthwhile) to have the switch handle the intervlan routing yet still be able to enforce the firewall rules between some, if any, vlans. My goal is to use less resources on pfSense, provide intervlan capabilities (preferably through the switch), and still be able to use the VPN on certain IPs/vlans when needed. If it ends up being too much of a kludge setup I'll just setup the new switch as a L2 (bought it for more ports, was a good deal).



  • Leaving VLAN processing to a hardware switch is probably best. Your switch is a managed switch?



  • If you want to control (firewall) any traffic between VLANs then you need to either:
    a) Check exactly what L3 features are available in the managed switch - if it has the control you need, then fine.
    or
    b) Keep the VLANs that you want to isolate from each other going to pfSense and let pfSense route and firewall between them. Obviously that uses pfSense resources, particularly if there are a few VLANs that will have a lot of traffic between them.



  • In the simplest term, 3 things:

    • The switch's default gateway should be the firewall.

    • All VLANs that need to communicate with each other in an unrestricted fashion should have a VLAN interface IP address defined on the L3 switch.  Hosts use it as their default gateway.

    • All VLANs that need to be isolated from each other need to have an interface IP defined on the firewall (pfSense).  All hosts on these VLANs use it as their default gateway.  Do not create a VLAN interface IP on the switch



  • @awebster:

    In the simplest term, 3 things:

    • The switch's default gateway should be the firewall.

    • All VLANs that need to communicate with each other in an unrestricted fashion should have a VLAN interface IP address defined on the L3 switch.  Hosts use it as their default gateway.

    • All VLANs that need to be isolated from each other need to have an interface IP defined on the firewall (pfSense).  All hosts on these VLANs use it as their default gateway.  Do not create a VLAN interface IP on the switch

    This seems more like the rules I was looking for. Am I still able to relay DHCP from the switch to pfSense or does that conflict? Also, am I able to prevent the hosts on the switch from communicating with SOME of the isolated VLANs such as for management purposes or to access the NAS? (both being isolated on different VLANs)



  • What would prevent you from using Switch ACLs to regulate this things such as you want?
    If this is a Layer3 LAN Switch I would more tend to let them do the inter VLAN routing because
    mostly that will be then nearly wire speed on all switch ports and often the pfSense box must be
    then really powerful to route this by it self. And why setting upt the DHCP twice in a network or
    in shorter words why should the pfSense offers beside of the Layer3 Switch?


  • LAYER 8 Global Moderator

    Keep in mind that if your going to now have a downstream router (l3 switch) that you would connect that with a transit network (network that connects devices that no clients are in)..  If not your most likely going to run into a asynchronous routing problem..

    And I can promise you the ACL features of the switch are not going to be as good as firewall in pfsense, so this switch is actually layer 4?  If your going to do any filtering on ports.. like blocking or allowing say http or ssh, etc.

    Unless your are moving tons of traffic between segments, there is little reason to not just let pfsense route all the traffic and keep your firewall rules in place.

    The most common issue I see when connecting a downstream router is the thinking they can leverage a segment that clients are in as connection for the new router, and they end up with asynchronous connection issues with these clients and the other clients on the downstream router.



  • Here's one way to look at it:

    
                                                          L3 Switch
                                                  +----------------------+
                                                  |                      |
                                                  |          VLAN2(u)+IP +--------------+
                                                  |                      |
                      pfSense                     |                      |
                 +--------------+                 |          VLAN3(u)+IP +--------------+
                 |              |                 |                      |
                 |              |      Trunk      |                      |
                 |    LAN(u)+IP +=================+ VLAN1(u)+IP          |
                 |  VLAN5(t)+IP |                 | VLAN5(t)             |
                 |  VLAN6(t)+IP |                 | VLAN6(t)             |
                 |              |                 |          VLAN4(u)+IP +--------------+
                 |              |                 |                      |
                 |              |                 |             VLAN5(u) +--------------+
                 |              |                 |                      |
    +------------+ WAN+IP       |                 |             VLAN6(u) +--------------+
                 |              |                 |                      |
                 +--------------+                 +----------------------+
    
      Static routes:                               Static routes:
      VLAN2 subnet Via L3 switch VLAN1 IP          Default gateway: pfSense LAN IP
      VLAN3 subnet Via L3 switch VLAN1 IP
      VLAN4 subnet via L3 switch VLAN1 IP
    
      Interfaces with addresses                    Interfaces with addresses
      WAN                                          VLAN1
      LAN                                          VLAN2
      VLAN5                                        VLAN3
      VLAN6                                        VLAN4
    
      (t) = Tagged
      (u) = Untagged
    
    

    Caveat:
    In this diagram, you cannot set DHCP relay on the switch and have pfSense do the DHCP for VLAN 2,3,4.  This is because this functionality is missing from pfSense web GUI.  >:(



  • @johnpoz:

    Keep in mind that if your going to now have a downstream router (l3 switch) that you would connect that with a transit network (network that connects devices that no clients are in)..  If not your most likely going to run into a asynchronous routing problem..

    And I can promise you the ACL features of the switch are not going to be as good as firewall in pfsense, so this switch is actually layer 4?  If your going to do any filtering on ports.. like blocking or allowing say http or ssh, etc.

    Unless your are moving tons of traffic between segments, there is little reason to not just let pfsense route all the traffic and keep your firewall rules in place.

    The most common issue I see when connecting a downstream router is the thinking they can leverage a segment that clients are in as connection for the new router, and they end up with asynchronous connection issues with these clients and the other clients on the downstream router.

    Yeah the idea was to avoid the switch's ACL because (as you said) pfSense's firewall is superior. awebster's diagram is exactly the setup I was envisioning and what I wanted to do in hopes of saving some resources on pfSense.

    @awebster:

    Caveat:
    In this diagram, you cannot set DHCP relay on the switch and have pfSense do the DHCP for VLAN 2,3,4.  This is because this functionality is missing from pfSense web GUI.  >:(

    That's basically what I had in my head and what I wanted to go with (nice diagram!). It's a shame that it's missing this functionality (currently) I see that there have been several attempts to merge the pull request. If this functionality was implemented, would pfSense still handle the traffic between VLAN5 and VLAN2 but not the traffic between VLAN3 and VLAN4?



  • @Elegant:

    … If this functionality was implemented, would pfSense still handle the traffic between VLAN5 and VLAN2 but not the traffic between VLAN3 and VLAN4?

    That's correct.


  • LAYER 8 Global Moderator

    So have have your switch provide dhcp for those vlans on the switch..  Never seen a L3 switch that didn't do dhcp..

    Keep in mind you can not put clients on lan/vlan1 in his diagram, this now becomes your transit network.

    How much traffic is flowing between these downstream vlans to be curious, that you think you need to remove this traffic from the pfsense interface(s)



  • @johnpoz:

    So have have your switch provide dhcp for those vlans on the switch..  Never seen a L3 switch that didn't do dhcp..

    Yes, but it would be nice to have a single place to configure ALL your network's DHCP scopes.

    @johnpoz:

    Keep in mind you can not put clients on lan/vlan1 in his diagram, this now becomes your transit network.

    A wise choice!  Asynchronous routing problems are hard to spot and diagnose.

    @johnpoz:

    How much traffic is flowing between these downstream vlans to be curious, that you think you need to remove this traffic from the pfsense interface(s)

    Elegant didn't specify in his original question what the interface speeds were on his pfSense box, but a scenario where pfSense is running on an old clunker with 100mbps interfaces, which is fine if you've got a 10mbps internet connection, would not satisfy the needs of someone wanting to move data between clients and servers on different segments, like for a NAS, hence the L3 switch makes good sense here.


  • LAYER 8 Global Moderator

    your just speculating on a reason there awebster.. Yes I would agree not route all your traffic through pfsense if its slow interfaces, etc..

    But if that is not the case, then doing this is just complicating the network for the sake of complication..  Never a good thing..  You might say the better course of action would be to update the pfsense box/nics other than running a downstream router, now not being able to filter between vlans.  Not having central dchp, etc.



  • @johnpoz:

    your just speculating on a reason there awebster.. Yes I would agree not route all your traffic through pfsense if its slow interfaces, etc..

    But if that is not the case, then doing this is just complicating the network for the sake of complication..  Never a good thing..  You might say the better course of action would be to update the pfsense box/nics other than running a downstream router, now not being able to filter between vlans.  Not having central dchp, etc.

    Of course it is just speculation.  There are several use case scenarios where you'd want to use a downstream L3 switch: a) learning about routing and more complex networks; b) isolating voip or wifi traffic from the rest of the network; c) performance reasons; d) isolating potentially harmful traffic from pfSense.
    I don't think there is a hard and fast rule about how complex the network should be other than follow the KISS principle where possible when dealing with enterprise production systems.


  • LAYER 8 Global Moderator

    Agreed running a downstream router can be a huge learning experience.

    Not sure what harmful traffic you would want/need to isolate from your firewall?  If you mean blocking clients from access to gui, ssh or even services your running on pfsense sure..  But its meant to be your firewall, so I would think you would want to filter all your traffic through it all it to filter/isolate clients and services from each other.

    The filtering capabilities are going to be much better using pfsense than any l3 switch I can think of ;)

    Isolation of traffic like voip and data sure can be done with vlans or even physical segments routing the traffic through pfsense.

    To be honest the only reason I can think of a downstream router in a home setup would be learning experience or if you need to move traffic between segments that require none or very min filtering, And the interfaces of pfsense are not just up to it.  I would look to updating the pfsense box and or its interfaces if this is the case as better option other than just the pure learning aspect of the downstream router.

    I have toyed with turning on my sg300 l3 mode, since my pfsense runs in vm and can not really push full gig that I have seen in testing.  I could for sure update my vm host to accomplish this.  But the simpler solution (KISS) was to just put those devices that move lots of data on the same segment so they don't go through pfsense at all.  Since these devices don't really need filtering.. Where I need/want filtering is between my wifi and my lan - no real need for 100% util of gig there - wifi just can not do it anyway, and my internet connection is no where close to gig..  Come on google fiber in Chicagoland ;)  Chicago is on the list…

    And I isolate my wired stuff like TVs, and console devices on their own segments..  They again don't really need full gig speeds..  I want full gig between my workstation and my vm host/nas/etc  So I just put them on the same segment keep pfsense out of the mix for traffic between these devices.

    I took the OP to being on a home/lab sort of setup and not production, maybe I was mistaken in this?


Log in to reply