Connection refused vs timeout

johnpoz

well depends on the firewall rules.. If the firewall sent back info via tcp rst or with udp a icmp port not available with reject setup you would get that refused.. Or if service listening rejected you for some reason then sure refused. If the packet was just dropped you get a time out..

So for example pfsense not listening 7777.. If I telnet to it get timed out. I then added a reject rule and instant refused

rejecttimeout.png_thumb

Grrruk

Hmmm. I tried to make a floating rule that rejects any UDP in 33434-33534 range. Still UDP-based traceroute does not work…

How to change default deny rule?

doktornotor

I totally fail to see how's changing block to reject going to make your traceroute "work"…

KOM

How to change default deny rule?

You can't. Instead, you create your custom rule and put it at the very bottom. Rules are processed top-down, first-match, so putting it at the very bottom will catch anything not explicitly allowed. The real Default Deny rule is not visible in the GUI but you could envision as being at the very bottom of your ruleset in every interface.

Grrruk

Ok, i'll simplify the question:

[2.2.4-RELEASE][admin@censored]/root: traceroute 192.168.0.11
traceroute to 192.168.0.11 (192.168.0.11), 64 hops max, 52 byte packets
^C

Local network card has 192.168.0.11. I can't traceroute even to local ip address inside pfSense box. How to allow UDP-based traceroute?

johnpoz

can you post up your firewall rules.. I am not having any issues with tracerouting using UDP which is linux default or using icmp after su to root.

traceroute to 192.168.2.11 (192.168.2.11), 30 hops max, 60 byte packets
1 pfSense.local.lan (192.168.9.253) 0.355 ms 0.267 ms 0.272 ms
2 uc.local.lan (192.168.2.11) 2.013 ms 2.011 ms 2.039 ms
user@clean:~$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 pfSense.local.lan (192.168.9.253) 0.466 ms 0.170 ms 0.173 ms
2 96.120.24.113 (96.120.24.113) 9.191 ms 15.255 ms 15.251 ms
3 te-0-5-0-8-sur04.mtprospect.il.chicago.comcast.net (68.85.180.133) 15.249 ms 15.248 ms 15.250 ms
snipped
user@clean:~$ traceroute -I 8.8.8.8
You do not have enough privileges to use this traceroute method.
socket: Operation not permitted
user@clean:~$ sudo su
[sudo] password for user:
root@clean:/home/user# traceroute -I 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 pfSense.local.lan (192.168.9.253) 0.459 ms 0.443 ms 0.461 ms
2 96.120.24.113 (96.120.24.113) 9.927 ms 14.704 ms 15.646 ms
3 te-0-5-0-8-sur04.mtprospect.il.chicago.comcast.net (68.85.180.133) 16.442 ms 17.900 ms 17.895 ms
snipped
root@clean:/home/user# traceroute -I 192.168.2.11
traceroute to 192.168.2.11 (192.168.2.11), 30 hops max, 60 byte packets
1 pfSense.local.lan (192.168.9.253) 1.674 ms 1.561 ms 1.534 ms
2 uc.local.lan (192.168.2.11) 4.926 ms 4.922 ms 4.901 ms
root@clean:/home/user#

Grrruk

Ok, I solved the problem. I added UDP reject rule for box destination IP ports 33434-33534 and placed it as FIRST rule on ALL incoming interfaces! And with several openvpn links it's not an easy task!

Still it's very annoying that there is no "allow this interface to be traceroutable target" button.

I totally fail to see how's changing block to reject going to make your traceroute "work"…

Because UDP-based traceroute packet must rejected from destination host, while pfSense silently drops it.

johnpoz

I do not have such rule in place and as you can see pfsense answer my traceroute..

Those are the rules on my lan 192.168.9.253 in pfsense, which you clearly see answer traceroute both to another segment off pfsense and stuff past pfsense on the internet.

lanrulesfortracepost.png_thumb

Grrruk

No need for any firewall rules. Just boot fresh new pfSense 2.2, log in and type:

traceroute 127.0.0.1

and you'll see timeouts.

Now I found why: it happens NOT because of default deny rule but because of this:


[2.2.4-RELEASE][admin@censored]/root: sysctl net.inet.udp.blackhole
net.inet.udp.blackhole: 1

Setting net.inet.udp.blackhole=0 in loader.conf.local solves all traceroute problems without additional firewall rules (and opens some questions about DDOS protection). I see no easy way to turn on blackhole for multiple WAN links and turn it off for multiple LANs and VPNs.

Grrruk

root@clean:/home/user# traceroute -I 192.168.2.11
traceroute to 192.168.2.11 (192.168.2.11), 30 hops max, 60 byte packets
 1  pfSense.local.lan (192.168.9.253)  1.674 ms  1.561 ms  1.534 ms
 2  uc.local.lan (192.168.2.11)  4.926 ms  4.922 ms  4.901 ms
root@clean:/home/user#

Here. You just used -I. I can use it too… while on linux. I can't use it from cisco routers (and we have a lot of them).

cmb

@doktornotor:

I totally fail to see how's changing block to reject going to make your traceroute "work"…

It won't.

Sending a RST in response to a connection attempt to a closed port, or sending a RST/ICMP unreachable instead of silently blocking traffic, indeed has no relation to whether traceroute is going to work. You have to pass UDP 33434-33534 for UDP traceroute to work.

Grrruk

Sending a RST in response to a connection attempt to a closed port, or sending a RST/ICMP unreachable instead of silently blocking traffic, indeed has no relation to whether traceroute is going to work.

It works. At least that's what worked in my case: I made a rule to reject UDP to 33434-33545 for one link address and immediately timeouts disappeared – for that address. But it's way too cumbersome to write lots of such rules.

You have to pass UDP 33434-33534 for UDP traceroute to work.

It's not enough to pass these ports because of default sysctl setting in pfSense. It defaults to blackhole for any refused connect. So packets come through and silently die. It's not a firewall issue after all!!!

BTW, /boot/loader.conf.local does not work in this case, I had to change net.inet.udp.blackhole in system -> advanced -> system tunables, then it survives reboot. Grrr.

cmb

@Grrruk:

It's not enough to pass these ports because of default sysctl setting in pfSense. It defaults to blackhole for any refused connect. So packets come through and silently die. It's not a firewall issue after all!!!

If you're trying to traceroute to an IP on the box itself from a directly-connected network, yes. The normal circumstance of tracerouting through the system, that's not applicable.

@Grrruk:

BTW, /boot/loader.conf.local does not work in this case, I had to change net.inet.udp.blackhole in system -> advanced -> system tunables, then it survives reboot. Grrr.

Of course, it's not a loader tunable, it's a sysctl. It'll be fine in the tunables list.

johnpoz

Again what??

So here is my pfsense
[2.2.4-RELEASE][root@pfSense.local.lan]/root: sysctl net.inet.udp.blackhole
net.inet.udp.blackhole: 1
[2.2.4-RELEASE][root@pfSense.local.lan]/root:

And again – as you saw I can traceroute to pfsense and through pfsense just fine using UDP not ICMP.. Well both work actually.. I have no idea what you did to your system.. But out of the box it works..

Here is am tracing from my cisco sg300

sg300#traceroute ip 192.168.2.11
Tracing the route to 192.168.2.11 (192.168.2.11) from , 30 hops max, 18 byte packets
Type Esc to abort.
1 192.168.9.253 (192.168.9.253) <40 ms <20 ms <20 ms
2 192.168.2.11 (192.168.2.11) <20 ms <20 ms <20 ms

Trace complete.

sg300#traceroute ip 8.8.8.8
Tracing the route to 8.8.8.8 (8.8.8.8) from , 30 hops max, 18 byte packets
Type Esc to abort.
1 192.168.9.253 (192.168.9.253) <20 ms <20 ms <20 ms
2 96.120.24.113 (96.120.24.113) <20 ms H <20 ms
3 H 96.120.24.113 (96.120.24.113) <20 ms H
4 68.85.180.133 (68.85.180.133) <20 ms <20 ms <20 ms
5 68.86.187.193 (68.86.187.193) <20 ms <20 ms <20 ms

cmb

@johnpoz:

And again – as you saw I can traceroute to pfsense and through pfsense just fine using UDP not ICMP.. Well both work actually.. I have no idea what you did to your system.. But out of the box it works..

Not in the circumstance he's describing, where you're tracerouting to an IP on the same subnet as the source host (why you'd want to do that, I don't know..what path are you expecting on the local subnet?).

When you traceroute to, say, your LAN IP from the LAN subnet, the first packet is destined to the host with TTL=1. It's not going to reply with a TTL exceeded because it's not. It's not going to reply that the port's closed because of the blackhole sysctl. It's the same as trying to connect to the UDP port on the host in that scenario and only that scenario. But it's a pointless use case of traceroute.