Can't ping the same IP from multiple devices

[patient0]

@SteveITS sound a lot like FreeBSD Bug 283795 - ICMP echo requests from Windows hosts dropped when NAT'ed.

If I run ping -e 1 8.8.4.4 on two Linux machines on LAN, I hit the same issue.
If instead I run ping -e 8 8.8.4.4 the port (not sure that is the correct term) on WAN is set to random and it works.

There is another bug (long) report (can't find it right now) in which it's mentioned that it is based on changes imported from OpenBSD. Unfortunately it sounded like ThisSense against TheOtherSense (which stated a possible reason for the bug) and it won't get fixed.

Addition: the other (fixed, long-read) bug Bug 280701 - FreeBSD-SA-24:05 fix breaks ICMP/ICMP6 states handling in pf firewall (ping, traceroute)
Addition 2: still broken on latewt 25.03-BETA

[SteveITS]

@patient0 Interesting so we’re not crazy. It’s 1am so I’ll test more tomorrow. Not sure I tried pinging all our routers.

It’s something few would notice I expect but we of course diagnose connection issues for clients, and we have monitoring which pings everything in and on the way to our data center. Seems like we could trigger that by pinging a server ourselves.

[patient0]

@SteveITS said in Can't ping the same IP from multiple devices:

Not sure I tried pinging all our routers.

I don't run into the issue since I don't have any Windows clients. And it seems that ping on Windows sets ICMP ID to 1 while on Linux it is set to a random id.

FreeBSD (erroneously) set the WAN ICMP ID to the same as the source ICMP ID. And only if you set ICMP ID to 8 FreeBSD picks a random ID for the WAN to ping-destination. And since ICMP type ECHO_REQUEST numerical value is 8 ... mmhh, I'm no programmer.

[Bob.Dig]

@patient0 Interesting. Just checked with the OtherSense, it shows this too, but you already gave that away.
Edit: And checked with another Router, FreshTomato: behaved the same.

[patient0]

@Bob-Dig said in Can't ping the same IP from multiple devices:

@patient0 Interesting. Just checked with the OtherSense, it shows this too, but you already gave that away.

Yes and it still does work with CE 2.7.2/FreeBSD 14. I think the feature/bug got introduced somewhere along FreeBSD 14.1/14.2 (in pf, ipfw doesn’t have the issue)

[patient0]

@Bob-Dig said in [Can't ping the same IP from multiple devices]

Edit: And checked with another Router, FreshTomato: behaved the same.

Mmh, that odd since it’s Linux based. I did check with VyOS back then and didn’t hit it. Have to recheck later this week.

[Bob.Dig]

@patient0 said in Can't ping the same IP from multiple devices:

that odd since it’s Linux based

But the host was Windows, so I guess, it is somewhat expected. It probably was nice of *Sense, that they had a "mitigation" for this Windows behavior.

[stephenw10]

Yeah if you test from something that's not Windows you'll probably find it works fine. For some reason Windows uses the same ID for all pings. So if you have 1:1 NAT (or static ports outbound NAT) then only one internal system can open a unique state. Linux uses incremental IDs. BSD uses random IDs.

[Bob.Dig]

@stephenw10 said in Can't ping the same IP from multiple devices:

So if you have 1:1 NAT (or static ports outbound NAT)

It doesn't seem to be related to that. It has worked in the past (according to the thread) and doesn't right now.

[SteveITS]

@SteveITS FWIW it's also an issue pinging the outer/building router from our LAN, so doesn't need to go past the second router.

I suppose, it mostly only matters as a colossal time waster while troubleshooting, if you don't know of the bug, since it's probably uncommon to see it (more common, the larger the company, I suppose).

My first time, pinging from a Linux VM, then from Windows, the Windows pings failed. After that I can't seem to reproduce that failure.

At least that implies we maybe can't trigger a false failure on our monitoring if we happen to ping something at the same time as our monitoring software.

@stephenw10 We did have static outbound set on the outer/building router...the rule is timestamped 2018 so I don't recall now why I set that. :) But turning that off last night did not change the behavior. It was not set on the inner one.

[stephenw10]

1:1 NAT implies static ports so if you have that set you would still hit this.

[SteveITS]

@stephenw10 ah ha, did not realize/remember that.

Edit: OK so then few would see this. And in theory port forwarding all ports and configuring outbound NAT, for that VIP, would bypass it?

[stephenw10]

Yes it would. Though it only affects icmp from Windows so.... it mostly doesn't matter.

I remember that blowing my mind when I first saw it. Mostly because Linux clients were unaffected.

[SteveITS]

I seem to use this pic a lot lately.

[SteveITS]

@Bob-Dig said in Can't ping the same IP from multiple devices:

It doesn't seem to be related to that. It has worked in the past (according to the thread) and doesn't right now.

Actually I think you are right, as least as worded. I tried from two Windows PCs at home and can repro it there. Automatic outbound NAT, not static, no 1:1.

One can see where the first ping expired:

Pinging 8.8.4.4 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 8.8.4.4: bytes=32 time=18ms TTL=116
Reply from 8.8.4.4: bytes=32 time=22ms TTL=116
Reply from 8.8.4.4: bytes=32 time=19ms TTL=116
Reply from 8.8.4.4: bytes=32 time=19ms TTL=116
Reply from 8.8.4.4: bytes=32 time=21ms TTL=116
Reply from 8.8.4.4: bytes=32 time=21ms TTL=116

[Bob.Dig]

@SteveITS said in Can't ping the same IP from multiple devices:

I think you are right

I tried it with both Senses and with FreshTomato, without any special OutboundNAT, the outcome was every time the same.