PFSEnse 2.4.1 - DISASTER !!!
-
Attached is the schematic of my setup. This also applies to other 7 offices with the exact layout. All of them are currently configured like this - WAN interface is the default gateway.
They all work ok. The issues started with v.2.4.0/1.
Any question let me know.
Thanks for your prompt reply.
-
Hun? Why does your lan side of pfsense have a gateway of .190 while pfsense IP on this interface is .150/26
Are you saying you have a vip setup on pfsense lan of .190? That your devices on this 10.50.0.128/26 network use as their gateway?
So your natting between these two rfc1918 networks?
Or are you saying you have a GW setup on pfsense LAN interface em1? Pointing to some 10.50.0.190/26 IP?
Can you please post your pfsense routes and gateways under system\routing menu.
-
LAN side .150 is connected to a firewall with the interface - .190 - this is DMZ_INSIDE subnet.
WAN side .240 is connected to a firewall with the interface - .240 - this is DMZ_OUTSIDE subnet.
Such setup works with no issues on 2.3.4Attached is the routing tab as requested - this was taken on the VM running on PFSense 2.3.4.
Once it's upgrade to PFSense 2.4.0/1 we can't reach that box at all.
-
And where are you routes to these downstream networks you reach via the 10.50.0.190 gateway?
Where exactly can you not reach what from what? So your natting between this 10.50.0.128/26 network and your 10.50.0.192/26 network?
Vs saying it worked on 2.x and not working on 2.4 lets actually understand how your network is setup..
What traffic flow is not working from where to where? From the internet to something downstream of this 10.50.0.190 router? From something the wan or lan of pfsense going where as dest, with what as the gw on this client?
You say you loose access to the gui of pfsense.. So you loose access to 10.50.0.150 from where are you trying to access it? From something on this 10.50.0.128/26 network with 10.50.0.150 (pfsense lan IP) as its gateway? A downstream network on the other side of the 10.50.0.190 router? From the wan network network of pfsense pointing to upstream router as its gateway?
edit:
What VM host are you run this on? You understand that 2.4 moves to freebsd 11.1 from 10.3, this could have a support issue on your VM hosting software? You sure your vm interfaces didn't change order in the VM on the update. You validated the vmnic macs are still the same connected to the network they are suppose to be connected to (wan/lan). How do you have the pfsense VM connected to the physical world.. you mention em for interfaces. So these are non vmx interfaces on esxi? How are the vswitches pfsense is connected to the real world? What is the switching environment in the real world? Are these networks vlans on the same smart switch, or completely different physical switches? -
If it's a VM what is on the console?
-
I've been doing the testing on several sites now and everytime ther are huge issues. I'm giving up. It's not worth upgrading to 3.4.0/1
Just tried from 3.3.5 to 3.4.1 and lost acces to that VM. Routes are in place though. I'm accessing it from 10.1.1.171 and the route is there.
So far the worst release I've ever seen. I've lost 4 nights on the upgrades and all ended up with no WEB gui or ssh access. It used to be ok in past.
Again I'd like to emphisize there's been no re-configuration on LAN/WAN side. The only thing that was done - PFSense upgrade to 2.4.0/1.#
Extremely frustrating. -
Unless you provide information asked above, there's really no easy way to help you.
-
Attached are screenshots from the box that is on 2.3.4 - its LAN IP - 10.50.0.147 and the one on 2.4.1 - LAN IP - 10.50.0.151.
The first one has two additional routes added manually to reach 192.168.X.X and 172.16.X.X networks.
By the looks of the one that I can't reach can access the Internet OK. Again - there's been no changes on the network. The issue seems to be only once I'm on 2.4.1
What other information do you need?Thanks
-
What kind of route is 10.0.0.0 ? Where is the mask on that network route? Even if was a host route it would have /32 on it..
Here I create a route then looking at it pfsense you see the mask.. yours is just 10.0.0.0 with no mask how is that a route?
As to other questions - are you natting? And all the other questions asked.. There is a list of them in previous post.
-
I added it manually using a VM console with that command:
route add -n 10.0.0.0/8 10.50.0.190
The output from netstat -r looks identical on both boxes - the accessible and the one I can't reach.
-
Well its not showing as valid since you see there is no mask for the network in your netstat -r command.
You added it via that command in pfsense console?
route add -n 10.0.0.0/8 192.168.9.2
route: bad keyword: nNotice it doesn't like that command, atleast in current version of pfsense 2.4.1 - I would have to fire up a 2.3.4 version to see if 10.3 of freebsd too it.
but if I add the route via
[2.4.1-RELEASE][root@pfsense.local.lan]/root: route add -net 10.0.0.0/8 192.168.9.2
add net 10.0.0.0: gateway 192.168.9.2it works and then shows route with mask.
[2.4.1-RELEASE][root@pfsense.local.lan]/root: netstat -rn
Routing tablesInternet:
Destination Gateway Flags Netif Expire
default 192.168.9.253 UGS em1
10.0.0.0/8 192.168.9.2 UGS em1
127.0.0.1 link#5 UH lo0So you clearly have something wrong in your current setup of 2.3.4 that might be working but is not liked in the 2.4 release.. Why would you not just add the route via the gui static route tab?
Since once I add that route via that command… It is not listed in the gui.. While it does list it in the diag route output.. I would have to assume on reboot that route would be gone. Are you then adding it via console once you update to 2.4?
On a bit of a side not, not a fan of overlapping route commands.. Ie the part where you saying to get to 10/8 use a 10.x network your currently attached too. I would be clearer and more direct to route specific to the networks that are downstream without the overlap network to what your attached too. While the most direct route should be used for your directly attached 10 networks. Your making a statement that to get to 10 anything you should talk to the gateway. Which is really not true since you have directly attached 10 networks that you would not talk to that gateway to get too. Its better practice to not route over your existing attached networks. Not saying it won't work - but its not as clear cut when looking at the routing table as it could be. And for sure the lack of mask is going to be a problem. I would create your routes via the gui.. Then attempt your upgrade.
-
Success!!!
Finally I know what was wrong. It must be a bug.Before upgrade all static routes are visible in Web GUI e.g. 10.0.0.0/8.
After the upgrade to 2.4.1 they are gone. Then I added tchem manually via shell:route add -net 10.0.0.0/8 10.50.0.190 (in my case the LAN interface on PFSense box)
Once done Web GUI becomes accessible. However theose static routes are not present there. They are only seen via command line - netstat -rn
Then I added them using GUI and bounced the box. All back to normal.
Thanks for the effort and pointing me to the right direction. -
pfsense is designed to be administered via the gui… Doing stuff like route add at the command line - you can not be sure it will survive a reboot/upgrade because doing it that way does not put the settings into the xml files that gets loaded, etc.
As to being a bug... There is something wrong if when you do a netstat -rn you don't see a mask on your route that is for sure.. What was causing that can not be sure. Could not duplicate it.. either manually adding route or adding route via gui showed the mask.
-
You need to (All under System > Routing):
1. Create a gateway on the LAN interface (em1) for 10.50.0.190.
2. Create static routes for 192.168.0.0 /16, 172.16.0.0 /12, and 10.0.0.0 /8 with that gateway as the destination.
3. If those routed subnets need to make connections into the em1 interface, the firewall rules there must allow those sources.
No other way of creating static routes is correct or supported and if you are playing around manually adding routes in the shell it is not really any surprise you had trouble when you upgraded.
Nothing here changed between 2.3.4 and 2.4.X. Gateways and static routes all upgrade just fine.
