pfSense Rewrites Source IP for ICMP Errors Breaking Traceroute
-
@johnpoz said in pfSense Rewrites Source IP for ICMP Errors Breaking Traceroute:
What is your outbound nat..
Im using
Automatic outbound NAT rule generation. (IPsec passthrough included):
Here are my nat rules I see in
/tmp/rules.debug:# Outbound NAT rules (automatic) # Subnets to NAT table <tonatsubnets> { 192.168.1.10/32 127.0.0.0/8 ::1/128 10.10.10.0/24 10.10.5.0/24 10.10.20.0/24 10.10.30.0/24 10.10.40.0/24 10.10.50.0/24 10.10.60.0/24 10.10.70.0/24 10.10.80.0/24 10.10.90.0/24 10.10.100.0/24 10.10.255.1/24 } nat on $WAN inet from <tonatsubnets> to any port 500 -> x.x.x.x/32 static-port nat on $WAN inet6 from <tonatsubnets> to any port 500 -> (lagg0.4090) static-port nat on $WAN inet from <tonatsubnets> to any -> x.x.x.x/32 port 1024:65535 nat on $WAN inet6 from <tonatsubnets> to any -> (lagg0.4090) port 1024:65535500port rule I think is coming fromIPSec:
Sorry Im new to NAT in general (thats among the reasons I got pfSense since I want to learn networking better). If my understanding is correct NAT should rewrite the source IP on outbound connections, add entry of the flow table, and finally for inbound connections, change the destination IP address so the packet can be redirected to an IP inside LAN. If so I dont fully understand why source IP is rewritten on the inbound connection.
Also keep in mind that responses from the hops would be ICMP but if you using linux for example the actual data sent doing the trace would be UDP packets.. Or sure you could even send TCP... But default to udp on linux, and windows would be icmp sent, etc..
Thanks! Looks like in this case
tracerouteis usingUDPwithICMPerror responses. -
So yes the nat would send the traffic back into your sender of the traceroute.. So I am just thinking you are confused to how nat works in general... But seeing missing spots in your trace that do not answer is quite normal these days of everyone blocking shit ;)
Are you doing anything odd on your lan side interface for rules or policy routing?
I sure an the hell can not duplicate your issue, and can traceroute from any machine on my network just fine using either icmp, udp or even tcp...
$ tcptraceroute -n www.google.com 443 Selected device ens3, address 192.168.2.12, port 50519 for outgoing packets Tracing the path to www.google.com (172.217.8.196) on TCP port 443 (https), 30 hops max 1 192.168.2.253 0.493 ms 0.332 ms 0.324 ms 2 50.4.135.1 11.620 ms 12.213 ms 11.700 ms 3 76.73.191.106 10.921 ms 12.476 ms 11.911 ms 4 76.73.164.142 9.586 ms 18.234 ms 12.953 ms 5 76.73.164.154 20.807 ms 20.399 ms 20.238 ms 6 76.73.191.242 14.286 ms 11.709 ms 14.869 ms 7 143.59.95.224 28.425 ms 12.980 ms 13.262 ms 8 75.76.35.8 11.605 ms 22.702 ms 10.050 ms 9 72.14.211.145 11.355 ms 31.802 ms 18.859 ms 10 108.170.243.225 23.928 ms 13.804 ms 14.214 ms 11 72.14.232.187 12.821 ms 12.967 ms 13.243 ms 12 172.217.8.196 [open] 21.723 ms 31.557 ms 17.055 ms -
Didnt do anything crazy as far as I know:
Ill keep digging on my end. Thanks for your help.
-
@miki725 Is it the same problem documented here? https://redmine.pfsense.org/issues/9263
-
You're using traffic (ICMP) shaping ?
-
Sure looks like that is his problem... @Alex-Atkin-UK
-
ooooh. I forgot about the limiter:
can confirm that when I disabled floating rule with the queue, traceroute works as expected inside LAN.
Thank a lot for linking to that. Now would be curious why a limiter would have that affect.
-
@miki725 said in pfSense Rewrites Source IP for ICMP Errors Breaking Traceroute:
Now would be curious why a limiter would have that affect.
It shouldn't - therefore the bug report ;)
You should of brought up the limiter, when asked doing anything odd on lan side rules (floating would be considered lan side) hehehe.. I will make sure to always mention floating rules going forward..
-
@johnpoz said in pfSense Rewrites Source IP for ICMP Errors Breaking Traceroute:
You should of brought up the limiter, when asked doing anything odd on lan side rules (floating would be considered lan side) hehehe.. I will make sure to always mention floating rules going forward..
My apologies. Forgot about them. Makes perfect sense that they are as legitimate LAN side rule as any other.
Thanks everyone for helping even with limited provided information. Really appreciate everyones time and effort. Stay safe!
-
@miki725 I remembered it as I had the same problem.
As I recall you have to add a Pass floating rule for the WAN interface at the top of the list for echo reply/requests with Quick ticked, so it bypasses the limiter rule.
-
-
Thanks for the suggestion. Didnt even think of bypassing it.
Where is a standard place to add limiter rules? Mine is currently is a floating rule. Even with the ICMP bypass rule with
Quickabove it, traceroute is still impacted:
I tried removing the
Quickfrom the limiter rule but that does not seem to help. Only thing which helps is disabling the limiter altogether. Im probably not doing something correctly if this works for you. -
Like me, you added floating rules to do something about buffer bloat.
These are all my floating rules :
You're missing one IPv4 rule : the "direction : IN" rule.
You have the "Out" rule, the one with the gateway (WAN_DHCP).I also use IPv6 - so that adds 3 more rules.
-
In your ICMP rule setting direction: out, interface: WAN and change default to your actual gateway in advanced > Gateway
should do the job. -
This got me today. I can confirm the floating rule for ICMP solves the issue.
