LAN with multiple subnets
-
I'm not sure if this is setup correctly or if i'm experiencing a bug:
Setup
pfSense 2.2.2
VMWare ESXi 6.0, e1000 adapters1 LAN interface
4 WAN interfacesLan interface has 10.1.0.0/16 range, which i'm slowly migrating to various /24 subnets (10.2.1.0/24 10.2.2.0/24, etc) and for them to have internet routing i had to add both a routing entry for the subnet (adding a gateway for the LAN interface) and rules in the firewall.
Now, the problem i have is that from time to time something gets screwed and it seems like pfSense makes LANGW the default gw! (attempting to ping internet results in ttl expired, a trace shows the packet bouncing between the switch, which is the inter-subnet-router and the pfsense box). Also during this time pfSense's GUI becomes unresponsive, ping works and local console also works.I really don't want to have to add an interface for each subnet nor have to add vlan interfaces on pfSense.
-
I really don't want to have to add an interface for each subnet
What you say? A gateway for the LAN interface? An interface for each subnet? What a novel idea!
Draw a picture.
-
"I really don't want to have to add an interface for each subnet nor have to add vlan interfaces on pfSense."
What?? How are you going to have different subnet without an interface be it physical/virtual or a vlan sitting on a interface?
As to gateways?? You don't set gateways on pfsense lan interfaces.. It becomes a wan connection then, when you set a gateway on an interface your saying, hey if you want to get off your own network go to this IP. that is not something a lan interface in pfsense does. It is used for pfsense to talk to that segment, and to get traffic from that segment that it routes out its WAN interface(es) or its other lan interfaces if dest is directly connected to it.
if you have a downstream router connected to a lan interface, then you create route(s) to that down stream router pfsense should use to get to those networks. Not a gateway on the interface.
Only thing that should have a "gateway" on your lan should be your devices - pointing to pfsense to get off that network, ie other lan segments or wan/internet via pfsense.
If you don't want vlans or interfaces than just use your 10.1/16 – which come on dude really how many nodes to you have that you need a /16 JFC... 65K addresses, are you anywhere close to that??
-
I'm not using a lot of addresses. what i'm doing is having separate collision domains for each part of the company.
We have several labs, some customer machines housed (which have their own set of subnets) and of course visitors get their own vlan where they can get to the internet but not the other subnets.I'm not sure a picture is needed, i have a Dell 5548 switch with 3 units stacked. that switch is the default gateway for each and every vlan. The switch has it's own default gateway set to the pfSense box.
But if pfSense doesn't have routes loaded for every subnet it doesn't know how to reach them AND the only way i've found to add said routes is under:
System->Routing->Routes, to add a route there i HAVE to create a gateway, it doesn't allow me to just set the next hop like it should, but forces me to create a GW.I understand that this should be as simple as adding a route for 10.2.1.0/24 via 10.1.255.254 but i haven't found it on the gui (or to be fair, it is there but it forces the creation of a GW)
This part of the comment is exactly what i'm trying to do:
"if you have a downstream router connected to a lan interface, then you create route(s) to that down stream router pfsense should use to get to those networks. Not a gateway on the interface." -
If you don't want to have a bunch of routes in pfSense it is up to you to route a supernet to the layer 3 switch and subnet that network to each segment.
Make a small transit network between pfSense and the switch on its own VLAN.
pfSense 172.24.56.1/24
Switch 172.24.56.2/24
Set the default gateway on the switch to 172.24.56.1Supernet: 10.2.0.0/16
Make a gateway (ie SWITCH_GW) for the switch in System > Routing fat 172.24.56.2
Make a static route for 10.1.0.0/16 with a destination of SWITCH_GW
You can then create layer 3 interfaces on each VLAN in the switch for:
10.1.1.0/24
10.1.2.0/24
10.1.3.0/24
10.1.4.0/24
…Automatic NAT is pretty smart but you want to make sure there are outbound NAT rules for the entire 10.1.0.0/16. You will also need to DHCP for your segments in the switch or with helpers to something other than pfSense (Like your DC). It doesn't have the hooks for multi-scope DHCP on one interface.
-
Thanks Derelict,
I forgot to add. All of this is currently working, i have multi-scope DHCP from my AD controllers, the switch has an interface on each VLAN (that is how it routes after all), all of what you are saying is basically how i've got it set up right now. It's just the part where you say "Make a gateway (ie SWITCH_GW)…." that breaks my config. I've had cases where for some reason pfSense tries to use THAT GW as the default gw and breaks all outbound connections. I mean, it's working perfectly right now, but in the last month i had 3 separate occurences of pfSense trying to route back thru the switch, IF there was some way to have the routes loaded WITHOUT defining a gateway the multiwan module wouldn't try to do that, that's what i'm trying to find out.
-
pfSense doesn't do that. You have something wrong. If it's not marked as the default gateway it will not use it as the default gateway.
Post your config. Gateways, Routes, interface configurations.
Post a drawing.
Don't tell us what you think you have done. Show us exactly what you HAVE done.
If you had configured it like that it would be working. Always.
-
It might have something to do with the finicky-ness of the internet links down here, at least one of the times this ocurred i had reconfigured the main switch therefore causing pfSense to mark all gateways as down.
Attached are the VM config, where each WAN has a vlan and it's interface, the interface tab and the route tab.
Network config is:
VMCluster -> bladecenter stacked switch - > main switch (in trunk mode, ESXi deals with untagging the vlans)
WAN1-4 -> main switch (with the ports correctly set as general, untagged and the corresponding vlan id)Like i said, it works. it just so happens that for some reason it suddenly wants to route thru the LANGW. nothing in the logs saying why or how it does that (at least not after a reboot)
-
What is the interface address and subnet mask configured on pfSense LAN?
-
10.222.254.251 /16, in the old non-subnetted range.
-
What else do you have anywhere that starts with 10.222 ?? besides the switch at 10.222.255.254
No idea why you have that on a /16 but it's your network.
-
It's a /16 because i was young and stupid at the time (might be stupid still…) and didn't know anything about networking, vlans, subnets and didn't have hardware that supported it. The separate ranges are for each lab and it's people to be able to configure PXE, BOOTP, DHCP, and all the other multicast stuff they might want without crashing the rest of the network.
The stuff still on the 10.222.0.0/16 are some of the labs that i haven't been able to move to VLANs yet and one of the DCs that is awaiting new licenses for a migration.
-
So everything's working now and the issue is why, as you say, pfSense uses LANGW as the default gateway right?
Did you do something silly like make a failover gateway group with WAN and LANGW as members?
-
No, only the real WAN GWs are included in the gateway groups. i have 3 with different tiers for different needs (adsl for the visitors, cablemodem for users, best one for the servers, etc)
-
Well, pfSense doesn't just arbitrarily route out LAN so I guess I don't know what to tell you.
-
Damm, i hoped i had something misconfigured. Well, i'll try to delve deeper and see if it happens again.
Thanks! -
You might but I have no idea where to tell you to look next. Grep WANGW in /tmp/rules.debug and see if it's somewhere it shouldn't be. Take a good, long look at what's really happening next time it blows up. At least get a copy of Diagnostics > Routes.
