Installing pfSense with a layer 3 switch


  • LAYER 8 Netgate

    I've been meaning to do this one for a while:




  • I use pfsense for a fast router firewall NAT device.  I like the idea of snort.

    I want to keep all my VLAN networks on the switch.  When I create VLANs I always assign an IP network to each VLAN.  ACLs provide enough control for me and my network.  I  have 3 Cisco WAP321 which dumps users into a guess network or a LAN network.  This works great for me. I am able to feed multiple VLANs to one router VLAN on the layer3 switch to feed the router. The SG300-28 has a 17 GIG backplane so it can run faster than any internet connection I would have.  I only have a 300megabit connection.  I just want the fastest firewall I can run on the front door to the internet.  pfSense is on the table now.

    I am not saying pfsense has a problem with a /30 mask. It could be the SG300-28 switch.  It is why I asked johpoz whether he had tried it as he owns an Cisco SG300 switch also.  I have posted a question on the Cisco small business web site to ask if there is an issue.

    I took a working pfsense router 192.168.10.1/24 mask plugged into a router VLAN on the SG200-28 switch defined with a /24 mask. It was working and is my current config I am using.  I changed the VLAN IP to 192.168.10.2/30 on the switch and I changed the default route to 192.168.10.2/30 also. I then ran 2 on the console for pfsense to change the LAN IP to 192.168.10.1/30.  I could not ping from the switch using ping on the switch to pfsense.  I could not ping from pfsense to the switch.  The switch port used is an access port defined to the router VLAN. It is the same port used for the /24 mask which works.  This sounds basic so I was looking for confirmation from john since he recommended using a /30 mask on this thread.


  • LAYER 8 Netgate

    Neither the Cisco SG nor pfSense has any problems with a /30 netmask. Guaran-effing-teed. You did something wrong.

    And it doesn't matter one bit what netmask you use. All that matters is that there are no hosts on the transit network that you expect to route properly unless you maintain a proper routing table on that host.



  • The net mask does make a difference.  It works with a /24 mask and does not work with /30 mask.  I do not run any clients in my router VLAN.  The idea of the router VLAN is to isolate from the broadcast domain, default VLAN, Windows chatter, etc. I do not want the router waiting for any local traffic not destined for the internet.


  • LAYER 8 Netgate

    Dude. You did something wrong. No way the Cisco or pfSense has a problem with /30. If it was /31 maybe. /30 no way. Look at your config again.



  • When I have another free evening.  I spent about 5 hours going over everything.  I even reinstalled pfsense.


  • LAYER 8 Netgate

    Chasing ghosts. And that was after changing the netmask on both ends from /29 to /30 - no reboots just change and apply.

    ![Screen Shot 2016-01-31 at 4.25.48 PM.png](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.25.48 PM.png)
    ![Screen Shot 2016-01-31 at 4.25.48 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.25.48 PM.png_thumb)
    ![Screen Shot 2016-01-31 at 4.25.38 PM.png](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.25.38 PM.png)
    ![Screen Shot 2016-01-31 at 4.25.38 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.25.38 PM.png_thumb)
    ![Screen Shot 2016-01-31 at 4.27.39 PM.png](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.27.39 PM.png)
    ![Screen Shot 2016-01-31 at 4.27.39 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2016-01-31 at 4.27.39 PM.png_thumb)



  • You need to be changing the LAN side em1 not em0.  This means all your local networks are routed on the LAN side. And you need static routes for all local traffic. I guess it could be an alias.


  • LAYER 8 Netgate

    em0 is my LAN side. Neither device has a problem with /30, regardless.



  • Sounds good.  I don't know why they will not link up with /30 mask for me.  It works with a /24 mask so there is not a rush.


  • LAYER 8 Global Moderator

    You can use whatever mask you want for the transit - the point is there can be no clients on the transit or you will end up with problems unless you create host routes on them for the networks in 2 different directions.

    The fact that you think a /30 is a problem for these devices is beyond nonsense.. As derelict said you did something wrong..



  • @johnpoz:

    You can use whatever mask you want for the transit - the point is there can be no clients on the transit or you will end up with problems unless you create host routes on them for the networks in 2 different directions.

    The fact that you think a /30 is a problem for these devices is beyond nonsense.. As derelict said you did something wrong..

    The /24 mask is more convenient so if you need to change the gateway to the static routes you can plug in a machine and make the change.  With a /30 mask there basically is no access to pfsense if something happens to your routing other than console from what I see.

    The question about the /30 mask can be answered by you since you have one of these SG300 switches.  Just set it up.  Please post the config on the SG300 and I will compare it to mine.


  • LAYER 8 Netgate

    Pertinent parts.

    vlan database
    vlan 1000,2000

    ip dhcp server
    ip dhcp pool network TEST_LAYER3
    address low 172.28.1.65 high 172.28.1.254 255.255.255.0
    default-router 172.28.1.1
    dns-server 192.168.223.1
    exit

    interface vlan 1000
    name TRANSIT
    ip address 192.168.230.2 255.255.255.252
    !

    interface vlan 2000
    name TEST_LAYER3
    ip address 172.28.1.1 255.255.255.0
    !

    interface gigabitethernet46
    description ROUTER_LAN
    switchport mode general
    switchport general allowed vlan add 1000 tagged
    switchport general acceptable-frame-type tagged-only
    !

    Not a lot to it. On pfSense I just created interface TRANSIT on vlan1000 as 192.168.230.1/30, a gateway for 192.168.230.2, and a static route for 172.28.0.0/16 to the gateway.

    Then I passed ICMP any source any dest TRANSIT address on the TRANSIT interface. Could ping across in both directions and from a host on vlan 2000.


  • LAYER 8 Netgate

    @coxhaus:

    The /24 mask is more convenient so if you need to change the gateway to the static routes you can plug in a machine and make the change.  With a /30 mask there basically is no access to pfsense if something happens to your routing other than console from what I see.

    That's what management VLANs are for.



  • @Derelict:

    Pertinent parts.

    vlan database
    vlan 1000,2000

    ip dhcp server
    ip dhcp pool network TEST_LAYER3
    address low 172.28.1.65 high 172.28.1.254 255.255.255.0
    default-router 172.28.1.1
    dns-server 192.168.223.1
    exit

    interface vlan 1000
    name TRANSIT
    ip address 192.168.230.2 255.255.255.252
    !

    interface vlan 2000
    name TEST_LAYER3
    ip address 172.28.1.1 255.255.255.0
    !

    interface gigabitethernet46
    description ROUTER_LAN
    switchport mode general
    switchport general allowed vlan add 1000 tagged
    switchport general acceptable-frame-type tagged-only
    !

    Not a lot to it. On pfSense I just created interface TRANSIT on vlan1000 as 192.168.230.1/30, a gateway for 192.168.230.2, and a static route for 172.28.0.0/16 to the gateway.

    Then I passed ICMP any source any dest TRANSIT address on the TRANSIT interface. Could ping across in both directions and from a host on vlan 2000.

    I think I have found at least one difference my switch port is defined as an access port. The idea was to keep all the routing on the L3 switch.  The port adds the tags as data comes in and strips tags as data flows out.


  • LAYER 8 Netgate

    If you're only doing one VLAN between pfSense and the switch an access port is fine but you can't define the VLANs on pfSense - it has to just be emX, not vlan X on emX.

    When talking to a managed switch I always tag it even if it's only one so you can add a vlan without screwing up what's already working.



  • @Derelict:

    Pertinent parts.

    vlan database
    vlan 1000,2000

    ip dhcp server
    ip dhcp pool network TEST_LAYER3
    address low 172.28.1.65 high 172.28.1.254 255.255.255.0
    default-router 172.28.1.1
    dns-server 192.168.223.1
    exit

    interface vlan 1000
    name TRANSIT
    ip address 192.168.230.2 255.255.255.252
    !

    interface vlan 2000
    name TEST_LAYER3
    ip address 172.28.1.1 255.255.255.0
    !

    interface gigabitethernet46
    description ROUTER_LAN
    switchport mode general
    switchport general allowed vlan add 1000 tagged
    switchport general acceptable-frame-type tagged-only
    !

    Not a lot to it. On pfSense I just created interface TRANSIT on vlan1000 as 192.168.230.1/30, a gateway for 192.168.230.2, and a static route for 172.28.0.0/16 to the gateway.

    Then I passed ICMP any source any dest TRANSIT address on the TRANSIT interface. Could ping across in both directions and from a host on vlan 2000.

    I´ve followed this discussion with great interest since I have a similar SG300 layer 3 setup as Coxhaus and have just started to look into replacing my Linksys LRT224 router with a pfSense firewall.

    When using a transit network for the routing between the switch and the firewall as in your nice example, I guess there must be a separate management interface/IP address used for logging into the pfSense Web GUI?



  • @Derelict:

    @coxhaus:

    The /24 mask is more convenient so if you need to change the gateway to the static routes you can plug in a machine and make the change.  With a /30 mask there basically is no access to pfsense if something happens to your routing other than console from what I see.

    That's what management VLANs are for.

    I´m totally new to pfSense and am collecting information on how to set it up and administer it. Is the default management VLAN in pfSense VLAN 1 and is the IP address configured for the LAN interface the address of the management VLAN?


  • LAYER 8 Netgate

    All interfaces in pfSense are untagged by default, with WAN configured to obtain an address via DHCP and LAN as 192.168.1.1 with an active DHCP server starting at .100. You should really start another thread with a specific question. Doesn't sound like this has much to do with this one.



  • @Derelict:

    Pertinent parts.

    vlan database
    vlan 1000,2000

    ip dhcp server
    ip dhcp pool network TEST_LAYER3
    address low 172.28.1.65 high 172.28.1.254 255.255.255.0
    default-router 172.28.1.1
    dns-server 192.168.223.1
    exit

    interface vlan 1000
    name TRANSIT
    ip address 192.168.230.2 255.255.255.252
    !

    interface vlan 2000
    name TEST_LAYER3
    ip address 172.28.1.1 255.255.255.0
    !

    interface gigabitethernet46
    description ROUTER_LAN
    switchport mode general
    switchport general allowed vlan add 1000 tagged
    switchport general acceptable-frame-type tagged-only
    !

    Not a lot to it. On pfSense I just created interface TRANSIT on vlan1000 as 192.168.230.1/30, a gateway for 192.168.230.2, and a static route for 172.28.0.0/16 to the gateway.

    Then I passed ICMP any source any dest TRANSIT address on the TRANSIT interface. Could ping across in both directions and from a host on vlan 2000.

    Did you define a default route on the L3 switch pointing to 192.168.230.1, or isn´t this necessary?


  • LAYER 8 Netgate

    Yes if you want all traffic without a better route to go to 192.168.230.1.



  • @Derelict:

    All interfaces in pfSense are untagged by default, with WAN configured to obtain an address via DHCP and LAN as 192.168.1.1 with an active DHCP server starting at .100. You should really start another thread with a specific question. Doesn't sound like this has much to do with this one.

    Well, Coxhaus asked the same question, how to access/manage the pfSense other than the console when setup with a /30 address and you said "That's what management VLANs are for.".

    I take your point and will start a new thread with specific questions when I start setting up pfSense with my SG300 L3 switch.


  • LAYER 8 Netgate

    Out-of-band management of your firewall gets tricky. Cisco ASAs have the same problem. It would be really nice to have an interface that, by default, wasn't in the firewall's main routing table and wasn't accessible via the other normal interfaces, yet listened on ssh and webgui.

    I would settle for forcing management interfaces (ssh, webgui, snmp, etc) to only listen on a specific interface's IP address.

    As it is you have to create a VLAN interface. It will also be listening on management services.
    Block access to all management ports/addresses on unfriendly interfaces.


  • LAYER 8 Global Moderator

    What does the transit network size have to do with management??

    You do understand you can get to the webgui or ssh on any IP in pfsense as long as your rules allow it.  Even from the wan side if you allow it via rules.



  • @johnpoz:

    What does the transit network size have to do with management??

    Obviously nothing.

    You do understand you can get to the webgui or ssh on any IP in pfsense as long as your rules allow it.  Even from the wan side if you allow it via rules.

    I did not know and that is why I asked. I understand now that you´ve explained it. I´m totally new to pfSense and have just started to read the documentation and information found on the forum.

    I´m well aware that silly questions from n00bs like me might irritate expert users like yourself. You understand, we all have to start somewhere.  ;)


  • LAYER 8 Global Moderator

    Silly questions do not irritate me that is for sure, what can get frustrating is the same questions over and over and over and over again..  Without searching for the information yourself before asking ;)

    But even whne the questions are "silly" I still answer them or point to where they are answered… So ask away your questions..  That is what we are here for.


  • LAYER 8 Netgate

    People think this stuff is easy. And it is with a grasp of everything in the ISO model.



  • @johnpoz:

    Silly questions do not irritate me that is for sure, what can get frustrating is the same questions over and over and over and over again..  Without searching for the information yourself before asking ;)

    But even whne the questions are "silly" I still answer them or point to where they are answered… So ask away your questions..  That is what we are here for.

    Thank you very much, sir. I really appreciate the great effort and help expert users like yourself and Derelict provide in the community forum.

    Coxhaus and myself are both old farts. I´m 70 and retarded .. sorry retired  ;D and unlike Coxhaus who I believe worked professionally with Cisco stuff in the old days, I just started with setting up a home network based on separate components (SG300-10, SG200-08, Cisco WAP371, Linksys LRT214) a few months ago. It´s the LRT214 I´m planning to replace with a pfSense firewall.

    I´m moving in to a new 90 m2 apartment in a couple of months where I´ll put my home network in production. For sure I would do just fine with a small consumer router (Asus, Netgear etc.) or even the ISP provided one. I´m doing all this for fun, it´s an excellent pensionist exercise.  :)


  • LAYER 8 Global Moderator

    Consumer router?  Whats the fun in that ;) hehehe and they all pretty much suck anyway..

    With the use of pfsense, some smart switches and a real AP your on your way to very stable and robust network with lots of room for learning and play..

    More than happy to help anyone learn no matter the age, I myself am no spring chicken anymore at 51..  Been in IT for 30+ years..  Before there was even tcp/ip hehehe.. We use to use ipx/spx, remember changing from old thinnet/thicknet cable to UTP… Rocking cat 3 ;) what a project that was..  Having to add the tcp stack to the windows 3.1 boxes running on 486 with math co processor installed...  We were on the bleeding edge of tech heheeh ;)



  • oletuv after you get your pfsense setup with your Cisco layer 3 switch you will want to supply NTP from pfsense to your switch for time.  It seems to work real well.



  • @coxhaus:

    oletuv after you get your pfsense setup with your Cisco layer 3 switch you will want to supply NTP from pfsense to your switch for time.  It seems to work real well.

    Hi Cox,

    Nice to see you on the forums again!

    Thanks for the tip. I think I´ll setup a pfSense box in a couple of months, after moving in to my new apartment. :)

    Ole



  • Before there was even tcp/ip hehehe.. We use to use ipx/spx, remember changing from old thinnet/thicknet cable to UTP… Rocking cat 3 ;) what a project that was..  Having to add the tcp stack to the windows 3.1 boxes running on 486 with math co processor installed...

    Oh gawd, the days of "expensive" ISA NIC cards with undocumented IRQ dependencies - plug in a local LPT printer and the network goes down (shudder)….



  • You can not write network prefixes using shortcuts like that, you have to spell them out fully. Try 192.168.0.0/16 in the destination network field and it should work.



  • @johnpoz:

    Also you don't need a /24 as a transit, you could just use a /30 - if you made it say 172.16.0/30 you could then just use a simple summary route 192.168/16 route to your networks on your L3 switch.  Then no matter what 192.168 vlan you add to that switch you never have to touch your routes again.

    When trying to add a 192.168/16 summary route I´m getting an "A valid IPv4 or IPv6 destination network must be specified." error.

    I´m using the latest 2.3.2-DEVELOPMENT (amd64) version.

    ![Summary route .png_thumb](/public/imported_attachments/1/Summary route .png_thumb)
    ![Summary route .png](/public/imported_attachments/1/Summary route .png)


  • LAYER 8 Netgate

    Try 192.168.0.0 with a /16 netmask not 192.168



  • @Derelict:

    Try 192.168.0.0 with a /16 netmask not 192.168

    Thanks, working now.


Log in to reply