Disappearing static routes
We have a problem where the static routes configured in pfsense disappear every few days.
Our environment is as follow:
Pfsense version *2.1-RELEASE * acting as the main router, inside our lan there is an openvpn server configured with site-to-site connections to our other locations, and while there are multiple WAN connected to the pfsense there is a firewall rule to make sure all site-to-site traffic will only exit out of one of the WAN connections, there is also static routes set so that all traffic directed to the IP addresses that belong to our other locations will be redirected to the openvpn server inside the LAN.
While for the most part the connections work well every now and then the static routes in pfsense (set in system>routing>routes) vanishes from the routing table, pfsense logs under system>routing are empty but there seems to be some connection to whenever any of the WAN fails, but it should be noted that even when all WAN connections are reconnected the static routes remain missing and that not every WAN failure results in this problem.
A temporary fix I’ve found is to make any change in the web console that requires to press the apply changes button (such as switching what is the default gateway), any suggestions about a more permanent fix?
Thanks in Advance.
openvpn routes should NEVER be set as static routes under system>routing>routes.
you should use the built-in "Advanced configuration" field available in the openvpn config to set or push the required routes.
Thanks for the replay heper but i think you don't undrstand the problem
The pfsense is not the openvpn server and this is not an openvpn route.
There is a diffrent openvpn server on the network that is in charge of handling all site to site traffic, that server works (and he has it's routes configed in a fashion similar to what you suggested).
The problem is that as pfsense is the defualt gateway it needs a static route to redirect the ip addresses of our other enviroments to that server, (which exits to the internet through a diffrent router and not pfsense) it is the route that redirects traffic to the other enviroments through that server that vanishes.
There is no site to site vpn configed on pfsense of any kind.
What you are describing should work - pfSense does not even know (or need to know) that the remote private-IP LANs are across a VPN. It just knows that the static route destination is the way to get to those networks. This has been working OK for me, to route to other private-subnets not directly connected to the main pfSense.
During 2.1 development there were some issues where the default route went missing when it needed to change, got removed but then not added back. But those issues were fixed an I haven't seen anything like this on 2.1-RELEASE.
I wonder what relevant differences there are in your configuration to an "ordinary" one?
Sorry to not be any direct help!
phil this sounds exactly like the problem i am having, i guess it's possible that there is a scenario where the problem remained unfixed and i happen to stumble into it (i am using 2.1-release), but as for what's different then the typical configuration it can be a lot of things, using multiple VPN's, multi-wan, virtual ip's, custom firewall rules… it's a complex network so any help narrowing it down will be appreciated.
I have had a dual-WAN site today that has had plenty of ISP issues - both WAN links going up and down a lot and sometimes both down at the same time (so much for redundancy!). It also has an internal router with other networks behind that router. There is a static route from the main router (e.g. 10.49.32.250) for 10.51.0.0/16 to the internal router at 10.49.32.252 - that route is still there.
This is a 2-WAN, 3-local-LAN using VLANs and with about 10 site-to-site OpenVPN servers (on the front-end system in this case) that each have an incoming site-to-site client office connected.
It is physically a 4-NIC box with 2GB of RAM and 4GB CF card - it doesn't run out of physical memory.
There are gateway groups and LAN1,2,3 rules selecting and passing traffic into appropriate gateway groups, normal type of block rules,…
This much complex a system is working, and I don't remember any time I have lost the static route since 2.1-RELEASE.
That covers a fair range of things, so I am struggling to think exactly how to narrow this down if the logs don't have anything interesting. I'll let my brain mull it over, but anyone else feel free to help.
is the static routes there set in the firewall rule gateway option or did you set it under system>routes and left the firewall rule as the default *?
for me it's currently set as the latter but that might be the difference between our two networks.
The Firewall Rule/s pass the traffic to the other internal router/networks without specifying a gateway (appears as "*" in the gateway column). Thus the traffic passes through to the ordinary routing table.
System->Routing->Gateways has an entry for the internal router (gateway) - note this gateway is NOT (and must not) be set as the gateway in Interfaces->LAN (the gateway field there should be "none")
System->Routing->Routes has an entry pointing to the internal router (gateway) (which sits on the LAN).
Same as you, by the sound of it.
exactly the same, which only makes this problem weirder.