pfSense ignores static routes
-
I am deploying pfSense 2.4.4 in a High Availability mode and facing a peculiar problem. The pfSense firewall seems to ignore the static routes configured.
The firewall has the following configuration:
WAN IP Addresses - 10.10.10.3 (Primary) and 10.10.10.4 (Secondary)
WAN CARP VIP - 10.10.10.2LAN IP Addresses - 172.16.100.3 (Primary) and 172.16.100.4 (Secondary)
LAN CARP VIP - 172.16.100.1Default Gateway - 10.10.10.1
SYNC IP Addresses - 172.16.1.1 (Primary) and 172.16.1.2 (Secondary)
Gateway for 192.168.0.0/16 - 10.10.10.100
There are a number of Virtual IP addresses configured on the WAN which get 1:1 NATed to the LAN IP addresses. There is another set of internal networks (192.168.0.0/16) which are behind a router (10.10.10.100) connected to the same switch as the pfSense WAN Interfaces. The traffic from these networks is selectively allowed to access the servers behind pfSense. I have added a static route for 192.168.0.0/16 on pfSense with 10.10.10.100 as the gateway but pfSense ignores that gateway and sends the return traffic destined for 192.168.0.0/16 networks also to 10.10.10.1 (default gateway).
I have tried using a single pfSense standalone system but in that case also the problem persists. However, when I put the Master node in maintenance mode and make the Secondary as the Master, the traffic for 192.168.0.0/16 network starts going back through the specified gateway (10.10.10.100) but after some time it again starts hitting the default gateway. Similar thing happens when the first node is made Master again. The return traffic for 192.168.0.0/16 starts going back through 10.10.10.100 but after some time it starts going towards the default gateway
As suggested in different discussion in forums, I have added the following Floating Rule on the top:
Action: Pass
Interface: WAN
Quick: Checked
Direction: Out
Any
Destination: 192.168.0.0/16But this has also not made any difference. Any suggestions for resolving this issue?
-
Couple of questions is this 10.10.10 a transit - or are there hosts on it? .100 seems like a LARGE transit.. Why are you doing 1:1 nat or any nat for that matter on rfc1918 networks?
2nd ? Your not policy routing are you.. On your lan what is your rule - are you sending traffic out a gateway or gateway group, etc. Is so such rules bypass all routes..
if you want traffic to just use pfsense routing then you would want a rule on lan side that allows traffic to this 192.168/16 network so it can just use pfsense routing.
Outbound rule on floating not goning to do shit - routing has already been determined by time your leaving a nic..
-
There aren't any directly accessible hosts on 10.10.10.0/24 and 10.10.10.100 is the gateway for multiple 192.168.x.0/24 networks.
The traffic from 192.168.0.0/16 for 10.10.10.0/24 leaves 10.10.10.100 without NAT.
There is an update here. The actual IP addresses being used on the WAN side are from a /24 public IP address block but for privacy reasons I have substituted them with 10.10.10.x while everything else remains the same.
There is no policy routing. The LAN rules don't specify a gateway.
I had come across some earlier posts in the forums which suggested using this Floating Rule to make sure that the outgoing traffic does not ignore the static routes.
In the rule on the LAN side suggested by you, do we need to specify any Gateway or should we leave it at default ?
Thanks,
-
The actual IP addresses being used on the WAN side are from a /24 public IP
The traffic from 192.168.0.0/16 for 10.10.10.0/24 leaves 10.10.10.100 without NAT.
There aren't any directly accessible hosts on 10.10.10.0/24So your using public as transit network? And then not natting rfc1918 space over public space... But you clearly stated your doing 1:1 nat etc..
So this is clear as MUDD!!
Please post up your lan rules.. Andy your routing tabble Sorry but what your saying makes zero sense... If you have a route, that says to get to network 192.168/16 use gateway 10.10.10.100, then that is where the traffic would be sent.. Since that is the ROUTE to get there... So unless your using some policy that forces traffic out some other path...
-
I fixed it. I just had to check "Disable reply-to" in System --> Advanced --> Firewall & NAT.
Thanks.
-
Where did you mention anything about multi wan?
Oh yeah you setup another gateway for this route.. But not using an actual transit, etc. And your natting it seems... Glad you got it sorted but still clear as mud to your setup.
-
Not using Multi WAN but even then this option worked for me.
-
Well, since no one understands what exactly was fixed and how, just by changing disable reply to (which is a very sane setting IMHO) you are setting yourself up for failure next time anything breaks.
And certainly pf doesn't ignore static routes
Glad this worked for you, but still, this worked out of luck (and you will be needing it)Regards
-
Prob has something to do with his vips and 1:1 natting.. Which not exactly clear on the how and why of such a setup..
If you hit vip address, your reply should come from that vip.. If he doesn't want it to come from that vip there is where the reply-to is coming into play.
I concur with you @technophile not actually understanding what fix it and how is not a good solution...
Tech: So what did you do to fix your problem?
User: Oh I clicked on some shit
Tech: Rolleyes!!!The network (as explained) seems borked out of the gate to me.. If his transit is public.. Why is he routing over this public space back to rfc1918 space to a different router? If he wanted to obfuscate his public space he should of called that out in his OP.. and prob used the documentation nets 192.0.2/24, 198.51.100.0/24 or 203.0.113.0/24
If the space he is using as transit is clearly public and the reason for the nats to his servers on rfc1918 space then he should use a different transit network to get to other rfc1918 space where this traffic is not natted, or use a tunnel to transverse the public space which amounts to a different transit, etc. etc.
My GUESS is some sort of asymmetrical routing problem where say rfc1918 space hits the public IP without the source being natted to some public transit being used and now needs to get back to the rfc1918 space via different route.
You could go down a deep rabbit hole trying to figure out what is actually going on, etc. But without the details from him just guessing.. For now the solution is "clicked some shit" ;)
An intelligent man is sometimes forced to be drunk to spend time with his fools
If you get confused: Listen to the Music Play
Please don't Chat/PM me for help, unless mod related
SG-4860 24.11 | Lab VMs 2.7.2, 24.11 -
@johnpoz said in pfSense ignores static routes:
Tech: So what did you do to fix your problem?
User: Oh I clicked on some shit
Tech: Rolleyes!!!Much better:
Patient calls doctor:
Patient: My Finger hurts.
Doctor: Can you explain exactly where it hurts and what you did before.
Patient: Never mind, I cut the finger of. It hurts no more.15 Minutes later the patient calls again:
Patient: Now my hand hurts.
... -
hehehhe - good one!
-
As I had explained originally, pfSense was ignoring the static route for 192.168.0.0/16 networks and instead of routing the traffic to the specified gateway for 192.68.0.0/16 it was directing the traffic to the default gateway. Since the problem got rectified I believe the reply to setting was overruling the static route setting.
-
The users on the 192.168.0.0/16 network need to access the servers behind pfSense using their actual IP addresses so that the access can be logged. Otherwise I could have conveniently done NAT for the entire block and this issue of specifying a static route and gateway won't have arisen.
There is no asymmetrical routing. It is RFC 1918 space leaving the public IP without a source NAT and trying to get back to the RFC space via the SAME route, which wasn't happening even after specifying the static route.
-
In the forums, similar cases of pfSense ignoring the static routes have been discussed in the past. In many cases, people mentioned using a floating rule to override this behaviour which did not work for me.
-
Hi.
I had this problem in 2.5.1 and 2.5.2, i tried ANYTHING, even opnsense but i had similar issues;
i found a permanent fix by removing all static routes and migrating them to firewall rules.Let's say we have three connection WAN1, WAN2 and WANX, we want to use WAN1 and WAN2 in load balancing and WANX ONLY for some routes (no internet is available here!):
1- remove all static routes, they will be ignored anyway and may give error on UI
2- Configure your gateways and create a gateway group for WAN1 and WAN2, in this example we call it LoadBalance
3- Create like usual a firewall rule in LAN to redirect internet traffic to LoadBalance gateway
4- Create one or multiple firewall rules under LAN with source LAN NET and with destination the nework of your static rules
5- Put the "static route" rules on TOP and the load balancing rule on the bottom, save and apply
Everything should now work fineExample: (note: networks in routes are redacted for privacy)
-
I had this same issue and what worked for me is creating a floating rule on the downstream PfSense to allow WAN to LAN connections. YMMW.
