Separate Subnets out via 1 LAN Address
-
First post here. But, I've been reading and learning a lot about pfSense over the last few weeks and I have finally decided to bite the bullet and implement it here in our network. I'm having a small problem that is probably easy to fix issue.
I have 3 subnets behind our main layer 3 switch. I have 10.24.0.0/16, 10.28.0.0/16, and a 10.100.0.0/16 network. The layer 3 switch and the pfSense box are in the 10.100.0.0/16 network. I have all of my WAN addresses configured correctly. When I remove our out dated Barracuda and put in the pfSense, the 10.100.0.0/16 network can get on the internet just fine. However, none of the computers in the 10.24.0.0/16 or the 10.28.0.0/16 networks can. I can't seem to figure out how to get all of the subnets to work.
I've given the pfSense the same LAN address that the Barracuda had 10.100.10.1 (the barracuda is not connected while I'm testing the pfSense box) so none of the switch configurations need to be changed.
The edge switches at the 10.24.0.0 and 10.28.0.0 subnets are set to route their traffic to the 10.100.10.3 switch here at the main office. This is the layer 3 switch connected to the pfsense box. The 10.100.10.3 switch here has a default route of 10.100.10.1 (the pfsense box). With all this in place, the 10.100.0.0/16 network has internet, the 10.24.0.0/16 and 10.28.0.0/16 networks do not....
I have set up an alias named "TrustedLAN" and it lists the 10.24.0.0/16, 10.28.0.0/16, and 10.100.0.0/16 networks in it. I have a LAN rule that sends IPv4* from the source (TrustedLAN), any port, any destination out the WAN Gateway. That didn't help.
I'm stumped.... why does the 10.100.0.0/16 network have internet but neither of the other networks do?
Any insight you could give would be most appreciated.
-
Well your 10.100/16 becomes the transit network - when you have a downstream router (your L3 switch).
For this to work.. Yes you would have to allow your downstream networks on your transit firewall interface.
You would also need to create routes on pfsense so it knows how to get to these downstream networks. Unless your downstream is natting them?
You would also need to modify your outbound nats, so these downstream networks are natted to the public internet IP (pfsense wan IP).
If you create gateway and routes to these downstream networks. And using automatic outbound nat - the outbound nat for these downstream networks would auto be created. If you had switched your outbound nat to manual. Then you would have to manually adjust here as well..
Please tell me you don't have hosts on this 10.100/16 network.. Or you going to run into asymmetrical routing problems for sure if your downstream router is not natting.
-
I do have clients in the 10.100/16 network... all of them work fine behind pfsense. None of the 10.28/16 or 10.24/16 clients are able to get on the internet via the pfsense box.
Where do I created the routes on pfsense so it will know about the downstream networks? When I go to System>Routing>Static Routes, I don't see where/how to add the other local networks...
I've tried going to Firewall>NAT>Outbound and changing it to Manual Outbound NAT Rule Generation and adding the other networks in there, but that didn't resolve the no internet issue for them.
Do you have any links/reads/videos to do what you are suggesting?
-
@fjackson said in Separate Subnets out via 1 LAN Address:
all of them work fine behind pfsense.
Yeah they will - and if you have no need to talk to other devices behind your downstream that can work. But if devices on your transit and downstream networks want to talk to each other. And your not doing host routing on the devices in the transit, or your not natting at the downstream router - your asymmetrical..And that is not good and will have issues at some point that is for sure.
You need to create a gateway first.. Whatever the 10.100 IP of your downstream switch is.. Then create your static routes for your downstream networks pointing there.
If devices on the 10.100 will want to talk to devices on your downstream networks. It would be better to create another transit network for pfsense and the downstream router to talk via. /30 would be fine for this. Then you will never have asymmetrical traffic flow.
There are many threads that discuss this topic... Here is a drawing that derelict had posted many moons ago that is still valid
edit: Another problem with downstream router and having hosts on the transit. Unless your downstream router is doing firewalling as well. While your asymmetrical setup might stop downstream from creating a conversation with hosts on the transit. Because pfsense would not pass the syn,ack back to them (no state) even if it has a route. This is not really good isolation of your networks.
Whenever your going to connect routers together.. There should be a true transit network connecting them, ie no hosts on this network.. Only routers belong on a transit network.
-
@johnpoz Thank you for your advice. I'll keep digging.
Each of the networks (10.24/16 and 10.28/16) have static routes pointing to 10.100/16 and all clients can talk to each other, even when the pfsense firewall is running. The only issue is the 10.24/16 adn 10.28/16 doesn't get out to the internet through the pfsense...
On the 10.24/16 edge L3 switch with an IP address of 10.24.0.1/16, I have route 0.0.0.0 0.0.0.0 10.100.10.3
On the 10.28/16 edge L3 switch with an IP address of 10.28.0.1/16, I have route 0.0.0.0 0.0.0.0 10.100.10.3
On the 10.100/16 edge L3 switch (the switch physically connected to the pfsense box) with an IP address of 10.100.10.3/16, I have ip route 0.0.0.0 0.0.0.0 10.100.10.1 (the IP address of the pfsense box LAN interface).I will add the 10.24.0.0/16 and the 10.28.0.0/16 to the Firewall>NAT>Outbound rules and see if that makes a difference. I can't test it right now as there are several utilizing the internet, so I'll have to test it this evening. I know I'm close to the solution.... LOL
Thank you again for all of your help so far... Information and Advice, never gets wasted on me... LOL
-
@fjackson said in Separate Subnets out via 1 LAN Address:
Each of the networks (10.24/16 and 10.28/16) have static routes pointing to 10.100/16 and all clients can talk to each other,
Huh?? Makes no sense.. How would 10.24 have a static route to 10.100.. This would be default on the L3.. Since its connected to 10.100
Where would you point a 10.24 client to for its route, other than the 10.24 svi on the L3 switch..
I could see 10.100 clients having routes allowing this to work.. point to 10.100.ip of your L3 for say 10.24/16
No offense dude - but your doing it wrong if your doing it this way.. Pfsense and your downstream router should be connected via a transit (no hosts on it).. Now all issues problems go away. No host routing needed, not downstream natting needed. Clients just point to their default gateway on their segments.
You have no chance of asymmetrical traffic... You have complete control of what downstream networks can talk to your upstream network.. No routes needed on your downstream other than its default, etc. etc..
Per the drawing I posted above is the correct way to use a downstream router..
-
I apologize for the confusion. I was explaining it in very simplified terms without getting into vlans and such.. The 3 networks are 3 separate buildings on the same campus connected 10 gig by fiber. The computers in each building gets their computers name based off their IP address scheme (10.24.x.x, 10.28.x.x, and 10.100.x.x). I inherited this network when I took over the task of maintaining it. Each edge L3 switch haa 3 vlans on them with routing enabled. Every other switch in the buldings does not have routing enabled and use the edge switch as their default gateway. The vlans on the edge switches are as follows:
Vlan1 for Management with an IP address 172.17.x.1/24 (the x being 24, 28, 100 based off the building it's in). This vlan is also the connection vlan for the other, non-routing switches in the building.
Vlan100 is the DATA vlan and is the vlan that all computers are in. It has the IP Address scheme of 10.x.0.1/24 (the x being 24,28,100 based off location). All computers get this address scheme too.
And finally each edge (routing) L3 switch has Vlan 255, the CORE vlan.... it is only for uplinking to the main L3 switch that is connected to pfsense box and has an IP Add scheme of 172.16.1.x/24 (the x being 24,28,100 based off location)....
I was trying to be very simple in my explaination earlier and I apologize for that. .. Here is how the L3 Routing switches are set up:
(10.24.0.0/16 Switch)
VLAN 1 (Management) - 172.17.24.1/24
VLAN 100 (Data) - 10.24.0.1/16 <<<<<---- Default Gateway for all computers in this building
VLAN 255 (Core) - 172.16.1.24/24 <<<<<---- fiber connection to the main L3 switch
ip route 0.0.0.0 0.0.0.0 172.16.1.100(10.28.0.0/16 Switch)
VLAN 1 (Management) - 172.17.28.1/24
VLAN 100 (Data) - 10. 28.0.1/16 <<<<---- Default gateway for all computers in this building
VLAN 255 (Core) - 172.16.1.28/24 <<<<<----- fiber connection to main L3 switch
ip route 0.0.0.0 0.0.0.0 172.16.1.100(10.100.0.0/16 Main Switch)
VLAN 1 (Management) - 172.17.100.1/24
VLAN 100 (Data) - 10.100.10.3/16 <<<----- Default gateway for all computers in this building
VLAN 255 (Core) - 172.16.1.100/24 <<<<--- fiber connection from both other L3 switches
ip route 0.0.0.0 0.0.0.0 10.100.10.1 <<<----- pfsense land port/ip addressAgain, I apologize to be so confusing before...
-
So your 172.16.1 your using as your transit for all your downstream L3s
And your 10.100 hands off your main (core) L3, that does routing for to your other L3s..
Ok that makes more sense..
But still seems like you hosts on this 10.100 transit.. Which is directly connected to pfsense. But your clients point to the L3 as their default. This is still asymmetrical - and for sure not an optimal configuration..
You run into this..
So just like how you have transit network for your other downstream L3s, you should have a transit to get to pfsense from your main L3..
Any network that connects routers is a transit, and should have no hosts on it.
Such setups normally occur as companies expand from more than a single network, and start adding downstream routers.. They don't remove the hosts from their networks, as it changes to a transit when a downstream router is added to it.
Do this..
Where the transit that connects your pfsense to your main L3 doesn't overlap with any of your other networks.
-
Thank you for the info.... I'll let you know the results.
-
Got it working.
First adding the transit switch/30 didn't work either.
To get it working:
I had to go under System>Routing>Gateways and created a LAN Gateway pointing to the 10.100.10.3 device
After that, I went to System>Routing>Static Routes and created 2 static routes… one for the 10.28.0.0/16 and one for the 10.24.0.0/16, setting both of them to use the 10.100.10.3 LANGateway I created above…
Then bam!!! all 3 networks, 10.100.0.0, 10.28.0.0, 10.24.0.0 can get on the internet… Now to move on to port forwarding. So far, this too has been a failure.... LOL
Hopefully this helps someone in my situation
-
Well yeah, which I clearly stated beginning of this thread that you would need a gateway and routes to your downstream networks
https://forum.netgate.com/post/965347
I had to go under System>Routing>Gateways and created a LAN Gateway pointing to the 10.100.10.3 device
So your still using 10.100 as your transit?? Again this is wrong!!
