• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Replace L3 switch/router by routing with Pfsense firewall

Scheduled Pinned Locked Moved Routing and Multi WAN
19 Posts 3 Posters 3.5k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • T
    tech.swim
    last edited by May 5, 2017, 1:40 AM May 4, 2017, 11:57 PM

    I want to replace an old layer 3 switch by moving it's routing functions to the Pfsense firewall. Everything is working well as it is I just want to simplify my network and eventually setup CARP.

    Current network:

    Pfsense Settings
    WAN 01 (Windstream)
    IP - 70.x.x.x /28
    Gateway - 70.x.x.x

    WAN 02 (Comcast)
    IP - 50.x.x.x/28
    Gateway - 50.x.x.x

    LAN
    IP - 172.16.0.3 (Pfsense)
    Gateways
    IP - 172.16.0.1 (Windstream MPLS)
    IP - 172.16.0.2 (Windstream Internet)
    Static Routes
    10.0.0.0/8      Gateway 172.16.0.2
    172.16.8.0/22      Gateway 172.16.0.2
    172.16.12.0/22  Gateway 172.16.0.2
    172.16.16.0/24  Gateway 172.16.0.2
    172.16.24.0/21    Gateway 172.16.0.2
    192.168.2.0/24    Gateway 172.16.0.1
    192.168.3.0/24  Gateway 172.16.0.1
    192.168.4.0/24  Gateway 172.16.0.1
    192.168.6.0/23  Gateway 172.16.0.1

    Old L3 Core Router

    VE 2
    Destination 172.16.2.0/24
    IP on subnet 172.16.2.2
    Gateway 172.16.2.1
    VE 3
    Destination 10.0.0.0 /8
    IP on subnet 10.0.0.8
    Gateway 10.0.0.253
    VE 12
    Destination 172.16.12.0 /22
    IP on subnet 172.16.12.2
    Gateway 172.16.12.1
    VE 16
    Destination 172.16.16.0/24
    IP on Subnet 172.16.16.2
    Gateway 172.16.16.1
    VE 18
    Destination 172.16.18.0/24
    IP on Subnet 172.16.18.2
    Gateway 172.16.18.1
    VE 19
    Destination 172.16.19.0/24
    IP on Subnet 172.16.19.2
    Gateway 172.16.19.1
    VE 28
    Destination 172.16.24.0/21
    Harlin IP on Subnet 172.16.24.2
    Gateway 172.16.24.1
    DHCP Relay 10.0.0.2 (Server DC1)
    VE 167
    Destination 172.16.32.0/21
    IP on Subnet 172.16.24.2
    Gateway 172.16.32.1
    DHCP Relay 10.0.0.3 and 10.0.0.4
    VE 172
    Destination 172.16.0.0/29
    IP on Subnet 172.16.0.2
    DHCP Relay 10.0.0.255

    L2 Switch Vlans
    Vlan 2 (VOIP)
    Network 172.16.2.0 Subnet 255.255.255.0 Gateway 172.16.2.1
    DHCP handled by Free PBX server
    Host Range 172.16.2.11 - 172.16.2.254
    Summary Address 172.16.2.0/24
    Vlan 3 (Servers)
    Network 10.0.0.0 Subnet 255.0.0.0 Gateway 10.0.0.253
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 10.0.3.0-10.0.255.255
    Excluded Range 10.0.5.0-10.0.5.255
    Summary Address 10.0.0.0/8
    Vlan 12 (Faculty)
    Network 172.16.12.0 Subnet 255.255.252.0 Gateway 172.16.12.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.12.11 - 172.16.15.254
    Excluded Range 172.16.15.201-172.16.15.254
    Summary Address 172.16.12.0/22
    Vlan 16 (Management)
    Network 172.16.16.0 Subnet 255.255.255.0 Gateway 172.16.16.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.16.11 - 172.16.16.254
    Summary Address 172.16.16.0/24
    Vlan 17 (Technology)
    Network 172.16.17.0 Subnet 255.255.255.0 Gateway 172.16.17.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.17.11 - 172.16.17.254
    Summary Address 192.168.2.0/24
    Vlan 18 (Security)
    Network 172.16.18.0 Subnet 255.255.255.0 Gateway 172.16.18.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.18.11 - 172.16.18.254
    Summary Address 172.16.18.0/24
    Vlan 19 (Sports Video)
    Network 172.16.19.0 Subnet 255.255.255.0 Gateway 172.16.19.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.19.11 - 172.16.19.254
    Summary Address 172.16.19.0/24
    Vlan 28 (Student)
    Network 172.16.24.0 Subnet 255.255.248.0 Gateway 172.16.24.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.24.11 - 172.16.31.254
    Excluded Range 172.16.31.240-172.16.31.254
    Summary Address 172.16.24.0/21
    Vlan 167(GUESTWIRELESS)
    Network 172.16.32.0 Subnet 255.255.248.0 Gateway 172.16.32.1
    DHCP Handled by Windows Servers 10.0.0.2
    Host Range 172.16.32.11 - 172.16.39.254
    Summary Address 172.16.32.0/21
    Vlan 168 (GUESTWIRELESS)
    Network 172.16.08.0 Subnet 255.255.252.0 Gateway 172.16.8.1
    DHCP Handled by Windows Servers Or not working because previous setup was brocade wireless controllers
    Host Range 172.16.8.11-172.16.11.254
    Summary Address 172.16.8.0/22
    Vlan 172 (PERIMETER)
    Network 172.16.0.0 Subnet 255.255.255.248 Gateway 172.16.0.1
    DHCP N/A
    Host Range 172.16.0.1 - 172.16.0.6
    Summary Address 172.16.0.0/29

    1 Reply Last reply Reply Quote 0
    • J
      jahonix
      last edited by May 5, 2017, 9:51 AM

      And where is your question or what isn't working?

      1 Reply Last reply Reply Quote 0
      • T
        tech.swim
        last edited by May 5, 2017, 2:42 PM

        Sorry I wasn't clear.

        My goal is to move all routing to the Pfsense firewall and pull the old layer 3 router out.

        Thank you for asking! :  )

        1 Reply Last reply Reply Quote 0
        • J
          johnpoz LAYER 8 Global Moderator
          last edited by May 5, 2017, 5:10 PM

          "My goal is to move all routing to the Pfsense firewall and pull the old layer 3 router out. "

          Ok - then do that, what is your question on doing that?

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          1 Reply Last reply Reply Quote 0
          • J
            jahonix
            last edited by May 5, 2017, 7:28 PM

            ;D

            john, that's not how "good cop, bad cop" is working. One of us has to change sides.  8)

            1 Reply Last reply Reply Quote 0
            • T
              tech.swim
              last edited by May 5, 2017, 7:35 PM

              Sorry, let me be more specific. How do I translate these brocade setting from the old router into Pfsense:

              Old Router

              VE 2
              Destination 172.16.2.0/24
              IP on subnet 172.16.2.2
              Gateway 172.16.2.1
              VE 3
              Destination 10.0.0.0 /8
              IP on subnet 10.0.0.8
              Gateway 10.0.0.253
              VE 12
              Destination 172.16.12.0 /22
              IP on subnet 172.16.12.2
              Gateway 172.16.12.1
              VE 16
              Destination 172.16.16.0/24
              IP on Subnet 172.16.16.2
              Gateway 172.16.16.1
              VE 18
              Destination 172.16.18.0/24
              IP on Subnet 172.16.18.2
              Gateway 172.16.18.1
              VE 19
              Destination 172.16.19.0/24
              IP on Subnet 172.16.19.2
              Gateway 172.16.19.1
              VE 28
              Destination 172.16.24.0/21
              Harlin IP on Subnet 172.16.24.2
              Gateway 172.16.24.1
              DHCP Relay 10.0.0.2 (Server DC1)
              VE 167
              Destination 172.16.32.0/21
              IP on Subnet 172.16.24.2
              Gateway 172.16.32.1
              DHCP Relay 10.0.0.3 and 10.0.0.4
              VE 172
              Destination 172.16.0.0/29
              IP on Subnet 172.16.0.2
              DHCP Relay 10.0.0.255

              You can see from the setting that I have static routes set up currently in Pfsense. What do I need to add to the Pfsense firewall to route traffic without the router?

              1 Reply Last reply Reply Quote 0
              • J
                johnpoz LAYER 8 Global Moderator
                last edited by May 5, 2017, 7:49 PM

                "What do I need to add to the Pfsense firewall to route traffic without the router?"

                Nothing!  If the networks are directly attached to pfsense.. Only thing you would have to do is put in the firewall rules to allow the traffic you want.

                An intelligent man is sometimes forced to be drunk to spend time with his fools
                If you get confused: Listen to the Music Play
                Please don't Chat/PM me for help, unless mod related
                SG-4860 24.11 | Lab VMs 2.8, 24.11

                1 Reply Last reply Reply Quote 0
                • T
                  tech.swim
                  last edited by May 5, 2017, 7:53 PM

                  I tried that and it didn't work. Let me be more specific.

                  How do you recreate what Brocade calls an ip helper in pfsense?

                  If I want to continue to use my current gateway 10.0.0.253 how do I get that through my firewall?

                  1 Reply Last reply Reply Quote 0
                  • J
                    johnpoz LAYER 8 Global Moderator
                    last edited by May 5, 2017, 7:56 PM

                    Is this 10.0.0.253 an IP on pfsense, or a gateway pfsense is connected to.. Your really going to need to draw your current network, and then draw what you want your network to look like.

                    So you can use a dhcp relay in pfsense to send dhcp discovery packets to your dhcp server.

                    An intelligent man is sometimes forced to be drunk to spend time with his fools
                    If you get confused: Listen to the Music Play
                    Please don't Chat/PM me for help, unless mod related
                    SG-4860 24.11 | Lab VMs 2.8, 24.11

                    1 Reply Last reply Reply Quote 0
                    • T
                      tech.swim
                      last edited by May 5, 2017, 8:31 PM

                      Here is our current Network setup (attached). I want to remove the router 10.0.0.8 and move those routing functions to Pfsense 172.16.0.3.

                      10.0.0.253 is a virtual IP in the brocade router.

                      Thank you for looking at this! Any advice is appreciated. :  )

                      ![Current Network Diagram.png](/public/imported_attachments/1/Current Network Diagram.png)
                      ![Current Network Diagram.png_thumb](/public/imported_attachments/1/Current Network Diagram.png_thumb)

                      1 Reply Last reply Reply Quote 0
                      • J
                        johnpoz LAYER 8 Global Moderator
                        last edited by May 6, 2017, 9:53 AM

                        What does 10.0.0.253 have to do with anything???  That your using 10.0.0/8 for what here?  Your loopback?

                        Dude connect your vlans to pfsense and be done with it.  What are you going to do with your mpls connection?  Do you still want that connected to the router?

                        There is nothing special you have to do here.. Create your vlans on pfsense, get rid of its routes and connect your L2 switch to pfsense.  Then using a transit network to connect to your router to get to the mpls networks it has routes for.  Or juts connect mpls direct to pfsense.

                        Your drawing is messed up.. How is pfsense using 172.16.0.1 as gateway when its hung off your router?  Was that meant to be drawing going to pfsense?  Or is there a switch there?

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        1 Reply Last reply Reply Quote 0
                        • T
                          tech.swim
                          last edited by May 6, 2017, 1:26 PM

                          Thanks Johnpoz! I appreciate your helping my ignorance! I was afraid of not providing enough information and I did make a mess, my apologies.

                          You're right about the 172.16.0.1(internet) and 172.16.0.2(MPLS) are both on the Windstream router. The MPLS connects to another site. I hung it off the Pfsense box to represent static routes on the Pfsense box. I can see how that doesn't make senses, sorry. I will probably replace it with a site to site VPN between Pfsense boxes.

                          I was trying to make this change without a major network configuration change, but I think you're right. I will follow your advice and simplify my network.

                          Thanks again for wading through my mess and providing sage advice!

                          1 Reply Last reply Reply Quote 0
                          • J
                            johnpoz LAYER 8 Global Moderator
                            last edited by May 6, 2017, 1:38 PM

                            You have some large networks hanging off the L3 currently.. /21, /22 how many nodes/clients are we talking?  How much intervlan traffic do you have?

                            How many interfaces does pfsense have?  Routing all your intervlan traffic through vlans on 1 physical interface on pfsense - even if its beefy enough to do all the routing at wire speed is going to force all your intervlan traffic to be shared and hairpinned off those vlan interfaces on pfsense.

                            While it will buy you ease of firewall rules between vlans - it does come at a price of available bandwidth between your vlans.  If they do not do a lot of intervlan then it prob not an issue.  But when redesign your network you need to take this into account or your going to get complaints from users that stuff is slower..

                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                            If you get confused: Listen to the Music Play
                            Please don't Chat/PM me for help, unless mod related
                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                            1 Reply Last reply Reply Quote 0
                            • T
                              tech.swim
                              last edited by May 6, 2017, 11:30 PM

                              At most, we have about 2000 users and devices at this location and 500 at the other location connected by MPLS. Pfsense is running on a physical server with 8cores and 12GB of RAM. We have 6 physical interfaces, one LAN and two WAN in use. Currently, CPU and RAM stays below 10-12%. We load balance with a 1Gbps copper and a 250Mbps fiber connection. During production, we maintain 250Mbps and peak around 350Mbps or a little higher.

                              You are right on we could do everything we need with one subnet and VLANs which is probably what we'll look at implementing this summer. We inherited this current setup. It was designed by an engineer who worked for a healthcare company.

                              I will re-evaluate our planning based on your advice.

                              Thank you!

                              1 Reply Last reply Reply Quote 0
                              • J
                                jahonix
                                last edited by May 7, 2017, 3:07 PM

                                @tech.swim:

                                We have 6 physical interfaces, one LAN and two WAN in use.

                                How do you spread 1x LAN and 2x WAN on 6 physical interfaces?

                                You have 11 VLANs on your L2 switch. Could prove beneficial to use 3x 3 VLANs + 1x 2 VLANs on 4 hardware interfaces to distribute load and avoid blocking. OR put them all on one LAGG?
                                I would really like to hear other opinions about this!

                                1 Reply Last reply Reply Quote 0
                                • J
                                  johnpoz LAYER 8 Global Moderator
                                  last edited by May 7, 2017, 6:33 PM

                                  Maybe he has 2 physical and rest all on 4 interfaces.. Really to make the best call need to know which vlans do the most intervlan.

                                  Lagg gets him nothing then throwing it all in 1 lump and having the ability for one of the connections to go down.. Doesn't remove the problem of shared bandwidth with possible hairpin of connections, etc.  When you have lots of intervlan traffic is normally when you move them to a switch, and you only need to send traffic to your router/firewall that is somewhere else and actually needs to route..

                                  Without understanding the amount of intervlan traffic its impossible to say what would be the best configuration.  But he does have a few interfaces to play with, maybe his wan should share 1 physical interface since its less bandwidth then 1 physical interface is capable anyway.  Then he could split up his 5 interfaces, or maybe he reworks his network segments - there was some /24 and /21 etc.. so maybe he can combine some of those /24 that do a lot of talking between them.. That he is not worried about firewall rules with, etc.

                                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                                  If you get confused: Listen to the Music Play
                                  Please don't Chat/PM me for help, unless mod related
                                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                                  1 Reply Last reply Reply Quote 0
                                  • J
                                    jahonix
                                    last edited by May 7, 2017, 10:40 PM

                                    @johnpoz:

                                    Maybe he has 2 physical and rest all on 4 interfaces..

                                    Sorry, I don't get you.
                                    We know he has 6 physical interfaces (with 1x LAN and 2x WAN currently).
                                    Two WAN probably stay untouched and 1x LAN will be one or several trunks holding all 11 VLANs. I'm unsure about how to spread them across the available interfaces.
                                    However, I'm all with you that knowing inter-VLAN traffic would help.

                                    @johnpoz:

                                    Lagg gets him nothing … Doesn't remove the problem of shared bandwidth

                                    According to the LAGG docs there are more protocols than LACP, including LoadBalance.
                                    Couldn't that be a desired behaviour?
                                    (but I always thought LACP would be round-robin, which doesn't seem to be the case)

                                    @johnpoz:

                                    …maybe his wan should share 1 physical interface since its less bandwidth then 1 physical interface is capable anyway.

                                    1Gbps copper & 250Mbps fiber  ;)  but traffic would fit:
                                    @tech.swim:

                                    we maintain 250Mbps and peak around 350Mbps

                                    Let me say once more: I have no clue what's considered best practice.
                                    My best bet is to ask in this forum and wait for the educated responses.

                                    1 Reply Last reply Reply Quote 0
                                    • J
                                      johnpoz LAYER 8 Global Moderator
                                      last edited by May 8, 2017, 9:30 AM

                                      Lagg is great if you need more bandwidth as uplink and you have lots of clients going back and forth over this link..  Each mac pair could be using different path in the lagg.. But for a specific device lagg gets you nothing since your not going to use the different paths in the pair..

                                      Lagg into a router on a stick, which is what you get when you hairpin an interface is never an optimal setup.

                                      Lagg is never 1+1=2, it is just 1 and 1..  If you were going to use lagg why not just use 1 for vlan A, and other 1 for vlan B and this way you are sure you never have hairpin.  if you have more than 2 vlans then try and put the 2 that do not talk to each other much on the same physical interface.  Lagging the interfaces up just really put you in the dark on what traffic will take what path, etc.

                                      We do not know his network - only he would.  Maybe one of those segments is users and the other is servers.. And users love to put and pull stuff from servers..  Users normally don't talk to each other, and should have zero reason too.. So put the user segments on one uplink, and server vlans on other uplink, etc..

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        tech.swim
                                        last edited by May 8, 2017, 2:43 PM

                                        Wow! I really appreciate this!

                                        Thank you!

                                        1 Reply Last reply Reply Quote 0
                                        2 out of 19
                                        • First post
                                          2/19
                                          Last post
                                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                          This community forum collects and processes your personal information.
                                          consent.not_received