Replace L3 switch/router by routing with Pfsense firewall
-
I want to replace an old layer 3 switch by moving it's routing functions to the Pfsense firewall. Everything is working well as it is I just want to simplify my network and eventually setup CARP.
Current network:
Pfsense Settings
WAN 01 (Windstream)
IP - 70.x.x.x /28
Gateway - 70.x.x.xWAN 02 (Comcast)
IP - 50.x.x.x/28
Gateway - 50.x.x.xLAN
IP - 172.16.0.3 (Pfsense)
Gateways
IP - 172.16.0.1 (Windstream MPLS)
IP - 172.16.0.2 (Windstream Internet)
Static Routes
10.0.0.0/8 Gateway 172.16.0.2
172.16.8.0/22 Gateway 172.16.0.2
172.16.12.0/22 Gateway 172.16.0.2
172.16.16.0/24 Gateway 172.16.0.2
172.16.24.0/21 Gateway 172.16.0.2
192.168.2.0/24 Gateway 172.16.0.1
192.168.3.0/24 Gateway 172.16.0.1
192.168.4.0/24 Gateway 172.16.0.1
192.168.6.0/23 Gateway 172.16.0.1Old L3 Core Router
VE 2
Destination 172.16.2.0/24
IP on subnet 172.16.2.2
Gateway 172.16.2.1
VE 3
Destination 10.0.0.0 /8
IP on subnet 10.0.0.8
Gateway 10.0.0.253
VE 12
Destination 172.16.12.0 /22
IP on subnet 172.16.12.2
Gateway 172.16.12.1
VE 16
Destination 172.16.16.0/24
IP on Subnet 172.16.16.2
Gateway 172.16.16.1
VE 18
Destination 172.16.18.0/24
IP on Subnet 172.16.18.2
Gateway 172.16.18.1
VE 19
Destination 172.16.19.0/24
IP on Subnet 172.16.19.2
Gateway 172.16.19.1
VE 28
Destination 172.16.24.0/21
Harlin IP on Subnet 172.16.24.2
Gateway 172.16.24.1
DHCP Relay 10.0.0.2 (Server DC1)
VE 167
Destination 172.16.32.0/21
IP on Subnet 172.16.24.2
Gateway 172.16.32.1
DHCP Relay 10.0.0.3 and 10.0.0.4
VE 172
Destination 172.16.0.0/29
IP on Subnet 172.16.0.2
DHCP Relay 10.0.0.255L2 Switch Vlans
Vlan 2 (VOIP)
Network 172.16.2.0 Subnet 255.255.255.0 Gateway 172.16.2.1
DHCP handled by Free PBX server
Host Range 172.16.2.11 - 172.16.2.254
Summary Address 172.16.2.0/24
Vlan 3 (Servers)
Network 10.0.0.0 Subnet 255.0.0.0 Gateway 10.0.0.253
DHCP Handled by Windows Servers 10.0.0.2
Host Range 10.0.3.0-10.0.255.255
Excluded Range 10.0.5.0-10.0.5.255
Summary Address 10.0.0.0/8
Vlan 12 (Faculty)
Network 172.16.12.0 Subnet 255.255.252.0 Gateway 172.16.12.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.12.11 - 172.16.15.254
Excluded Range 172.16.15.201-172.16.15.254
Summary Address 172.16.12.0/22
Vlan 16 (Management)
Network 172.16.16.0 Subnet 255.255.255.0 Gateway 172.16.16.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.16.11 - 172.16.16.254
Summary Address 172.16.16.0/24
Vlan 17 (Technology)
Network 172.16.17.0 Subnet 255.255.255.0 Gateway 172.16.17.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.17.11 - 172.16.17.254
Summary Address 192.168.2.0/24
Vlan 18 (Security)
Network 172.16.18.0 Subnet 255.255.255.0 Gateway 172.16.18.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.18.11 - 172.16.18.254
Summary Address 172.16.18.0/24
Vlan 19 (Sports Video)
Network 172.16.19.0 Subnet 255.255.255.0 Gateway 172.16.19.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.19.11 - 172.16.19.254
Summary Address 172.16.19.0/24
Vlan 28 (Student)
Network 172.16.24.0 Subnet 255.255.248.0 Gateway 172.16.24.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.24.11 - 172.16.31.254
Excluded Range 172.16.31.240-172.16.31.254
Summary Address 172.16.24.0/21
Vlan 167(GUESTWIRELESS)
Network 172.16.32.0 Subnet 255.255.248.0 Gateway 172.16.32.1
DHCP Handled by Windows Servers 10.0.0.2
Host Range 172.16.32.11 - 172.16.39.254
Summary Address 172.16.32.0/21
Vlan 168 (GUESTWIRELESS)
Network 172.16.08.0 Subnet 255.255.252.0 Gateway 172.16.8.1
DHCP Handled by Windows Servers Or not working because previous setup was brocade wireless controllers
Host Range 172.16.8.11-172.16.11.254
Summary Address 172.16.8.0/22
Vlan 172 (PERIMETER)
Network 172.16.0.0 Subnet 255.255.255.248 Gateway 172.16.0.1
DHCP N/A
Host Range 172.16.0.1 - 172.16.0.6
Summary Address 172.16.0.0/29 -
And where is your question or what isn't working?
-
Sorry I wasn't clear.
My goal is to move all routing to the Pfsense firewall and pull the old layer 3 router out.
Thank you for asking! : )
-
"My goal is to move all routing to the Pfsense firewall and pull the old layer 3 router out. "
Ok - then do that, what is your question on doing that?
-
;D
john, that's not how "good cop, bad cop" is working. One of us has to change sides. 8)
-
Sorry, let me be more specific. How do I translate these brocade setting from the old router into Pfsense:
Old Router
VE 2
Destination 172.16.2.0/24
IP on subnet 172.16.2.2
Gateway 172.16.2.1
VE 3
Destination 10.0.0.0 /8
IP on subnet 10.0.0.8
Gateway 10.0.0.253
VE 12
Destination 172.16.12.0 /22
IP on subnet 172.16.12.2
Gateway 172.16.12.1
VE 16
Destination 172.16.16.0/24
IP on Subnet 172.16.16.2
Gateway 172.16.16.1
VE 18
Destination 172.16.18.0/24
IP on Subnet 172.16.18.2
Gateway 172.16.18.1
VE 19
Destination 172.16.19.0/24
IP on Subnet 172.16.19.2
Gateway 172.16.19.1
VE 28
Destination 172.16.24.0/21
Harlin IP on Subnet 172.16.24.2
Gateway 172.16.24.1
DHCP Relay 10.0.0.2 (Server DC1)
VE 167
Destination 172.16.32.0/21
IP on Subnet 172.16.24.2
Gateway 172.16.32.1
DHCP Relay 10.0.0.3 and 10.0.0.4
VE 172
Destination 172.16.0.0/29
IP on Subnet 172.16.0.2
DHCP Relay 10.0.0.255You can see from the setting that I have static routes set up currently in Pfsense. What do I need to add to the Pfsense firewall to route traffic without the router?
-
"What do I need to add to the Pfsense firewall to route traffic without the router?"
Nothing! If the networks are directly attached to pfsense.. Only thing you would have to do is put in the firewall rules to allow the traffic you want.
-
I tried that and it didn't work. Let me be more specific.
How do you recreate what Brocade calls an ip helper in pfsense?
If I want to continue to use my current gateway 10.0.0.253 how do I get that through my firewall?
-
Is this 10.0.0.253 an IP on pfsense, or a gateway pfsense is connected to.. Your really going to need to draw your current network, and then draw what you want your network to look like.
So you can use a dhcp relay in pfsense to send dhcp discovery packets to your dhcp server.
-
Here is our current Network setup (attached). I want to remove the router 10.0.0.8 and move those routing functions to Pfsense 172.16.0.3.
10.0.0.253 is a virtual IP in the brocade router.
Thank you for looking at this! Any advice is appreciated. : )
 -
What does 10.0.0.253 have to do with anything??? That your using 10.0.0/8 for what here? Your loopback?
Dude connect your vlans to pfsense and be done with it. What are you going to do with your mpls connection? Do you still want that connected to the router?
There is nothing special you have to do here.. Create your vlans on pfsense, get rid of its routes and connect your L2 switch to pfsense. Then using a transit network to connect to your router to get to the mpls networks it has routes for. Or juts connect mpls direct to pfsense.
Your drawing is messed up.. How is pfsense using 172.16.0.1 as gateway when its hung off your router? Was that meant to be drawing going to pfsense? Or is there a switch there?
-
Thanks Johnpoz! I appreciate your helping my ignorance! I was afraid of not providing enough information and I did make a mess, my apologies.
You're right about the 172.16.0.1(internet) and 172.16.0.2(MPLS) are both on the Windstream router. The MPLS connects to another site. I hung it off the Pfsense box to represent static routes on the Pfsense box. I can see how that doesn't make senses, sorry. I will probably replace it with a site to site VPN between Pfsense boxes.
I was trying to make this change without a major network configuration change, but I think you're right. I will follow your advice and simplify my network.
Thanks again for wading through my mess and providing sage advice!
-
You have some large networks hanging off the L3 currently.. /21, /22 how many nodes/clients are we talking? How much intervlan traffic do you have?
How many interfaces does pfsense have? Routing all your intervlan traffic through vlans on 1 physical interface on pfsense - even if its beefy enough to do all the routing at wire speed is going to force all your intervlan traffic to be shared and hairpinned off those vlan interfaces on pfsense.
While it will buy you ease of firewall rules between vlans - it does come at a price of available bandwidth between your vlans. If they do not do a lot of intervlan then it prob not an issue. But when redesign your network you need to take this into account or your going to get complaints from users that stuff is slower..
-
At most, we have about 2000 users and devices at this location and 500 at the other location connected by MPLS. Pfsense is running on a physical server with 8cores and 12GB of RAM. We have 6 physical interfaces, one LAN and two WAN in use. Currently, CPU and RAM stays below 10-12%. We load balance with a 1Gbps copper and a 250Mbps fiber connection. During production, we maintain 250Mbps and peak around 350Mbps or a little higher.
You are right on we could do everything we need with one subnet and VLANs which is probably what we'll look at implementing this summer. We inherited this current setup. It was designed by an engineer who worked for a healthcare company.
I will re-evaluate our planning based on your advice.
Thank you!
-
We have 6 physical interfaces, one LAN and two WAN in use.
How do you spread 1x LAN and 2x WAN on 6 physical interfaces?
You have 11 VLANs on your L2 switch. Could prove beneficial to use 3x 3 VLANs + 1x 2 VLANs on 4 hardware interfaces to distribute load and avoid blocking. OR put them all on one LAGG?
I would really like to hear other opinions about this! -
Maybe he has 2 physical and rest all on 4 interfaces.. Really to make the best call need to know which vlans do the most intervlan.
Lagg gets him nothing then throwing it all in 1 lump and having the ability for one of the connections to go down.. Doesn't remove the problem of shared bandwidth with possible hairpin of connections, etc. When you have lots of intervlan traffic is normally when you move them to a switch, and you only need to send traffic to your router/firewall that is somewhere else and actually needs to route..
Without understanding the amount of intervlan traffic its impossible to say what would be the best configuration. But he does have a few interfaces to play with, maybe his wan should share 1 physical interface since its less bandwidth then 1 physical interface is capable anyway. Then he could split up his 5 interfaces, or maybe he reworks his network segments - there was some /24 and /21 etc.. so maybe he can combine some of those /24 that do a lot of talking between them.. That he is not worried about firewall rules with, etc.
-
Maybe he has 2 physical and rest all on 4 interfaces..
Sorry, I don't get you.
We know he has 6 physical interfaces (with 1x LAN and 2x WAN currently).
Two WAN probably stay untouched and 1x LAN will be one or several trunks holding all 11 VLANs. I'm unsure about how to spread them across the available interfaces.
However, I'm all with you that knowing inter-VLAN traffic would help.Lagg gets him nothing … Doesn't remove the problem of shared bandwidth
According to the LAGG docs there are more protocols than LACP, including LoadBalance.
Couldn't that be a desired behaviour?
(but I always thought LACP would be round-robin, which doesn't seem to be the case)…maybe his wan should share 1 physical interface since its less bandwidth then 1 physical interface is capable anyway.
1Gbps copper & 250Mbps fiber ;) but traffic would fit:
@tech.swim:we maintain 250Mbps and peak around 350Mbps
Let me say once more: I have no clue what's considered best practice.
My best bet is to ask in this forum and wait for the educated responses. -
Lagg is great if you need more bandwidth as uplink and you have lots of clients going back and forth over this link.. Each mac pair could be using different path in the lagg.. But for a specific device lagg gets you nothing since your not going to use the different paths in the pair..
Lagg into a router on a stick, which is what you get when you hairpin an interface is never an optimal setup.
Lagg is never 1+1=2, it is just 1 and 1.. If you were going to use lagg why not just use 1 for vlan A, and other 1 for vlan B and this way you are sure you never have hairpin. if you have more than 2 vlans then try and put the 2 that do not talk to each other much on the same physical interface. Lagging the interfaces up just really put you in the dark on what traffic will take what path, etc.
We do not know his network - only he would. Maybe one of those segments is users and the other is servers.. And users love to put and pull stuff from servers.. Users normally don't talk to each other, and should have zero reason too.. So put the user segments on one uplink, and server vlans on other uplink, etc..
-
Wow! I really appreciate this!
Thank you!
