Yet another ping problem with Virtual IPs

stephenw10

Having looked into this further I'm certain it's a state issue.

If you look at the failed packet capture the incoming ping requests are all using ICMP identifier number '1'. pf uses that in the state it opens in the same way it does ports for TCP/UDP. It can't open a state on LAN with the same source, destination and identifier as one that exists so nothing passes until the previous state times out.

If you test remotely from behind a difference pfSense it will randomise the ICMP identifier when it outbound NATs the traffic. That means when it arrives at the port forward the two pings have different identifiers and two states can be created. No problem.

It appears that when you are testing and seeing failures the client you are testing from is using the same identifier for all pings. And the router you are testing behind is not randomising the identifier on the way out.

Nothing we can do about that in the pfSense forwarding those pings.

Interesting. Never hit that before.

Steve

stephenw10

Hmm, you could maybe have it outbound NAT the traffic to a different internal IP for ICMP only. That would randomise the identifier and allow it.
Ugly but...might work.

rebi

@stephenw10
Aaah, that makes sense now! Spot on!!! :)

Could it be because of a double NAT? ... When I think about it, I have a GPON that handles fibre optic and an IPTV box plus my router are connected to that GPON. My router does have an internal IP on the WAN interface. I don't have access to that GPON ... at least legally (I know the account though), as it's controlled by my TV & Net provider. Probably the guys whom I asked to test do have similar setup.

Anyway, I'm extremely grateful that you, guys, put a lot of effort into dealing with my problem!

Thanks and have a great week!

stephenw10

No problem. Interesting case.
I'd be interested to see if that outbound NAT workaround allows the pings. That would confirm it

You would see the identifier numbers changed in the state table.

Steve

rebi

I didn't quite get the workaround idea (BTW using "Automatic outbound NAT rule generation" at the moment). What rule do I have to create and will it take precedence, i.e. do I need to switch the Outbound NAT Mode?

As for the pings, I've just confirmed it's definitely the ICMP identifier as I downloaded something called hrping (googled it) which allows setting the ICMP Id. Whenever I set one and the same ICMP Id I have the problem and once they're different (hrping does this by default) there's no problem ... bloody routers! ... though one could blame Windows as well! Here's an excerpt from Wikipedia I've just read: "The Identifier and Sequence Number can be used by the client to match the reply with the request that caused the reply. In practice, most Linux systems use a unique identifier for every ping process, and sequence number is an increasing number within that process. Windows uses a fixed identifier, which varies between Windows versions, and a sequence number that is only reset at boot time."

stephenw10

Yes, blame all around there.

So pf can't create the state because everything is the same. If we add an outbound NAT rule on the LAN that then chnages the states to include the NAT but more importantly pf randomises the identifier making it unique.

So first switch outbound NAT to hybrid mode so you can add manual rules. The add a rule like:

It would be better to specify the source or use !this_firewall but that's not available. As it is if you try to ping the proxy VM from the firewall itself it will NAT that too. But that may not be an issue, certainly not as a test.

Steve

rebi

@stephenw10
Just tested this and it's the same. I've changed the NAT mode and added this rule:

Grimson

@rebi said in Yet another ping problem with Virtual IPs:

@stephenw10
Just tested this and it's the same. I've changed the NAT mode and added this rule:

@stephenw10 said in Yet another ping problem with Virtual IPs:

If we add an outbound NAT rule on the LAN that then chnages the states to include the NAT but more importantly pf randomises the identifier making it unique.

He said on LAN and you created it on WAN.

rebi

@Grimson my bad ... here it is but still doesn't work:

stephenw10

What happens when you try to ping? Any change at all?

Check the states in Diag > States, filter by icmp.

Steve

rebi

@stephenw10

Nope, exactly the same as before. Until the old state is cleared, the next ping doesn't go through. hrping works flawlessly but it changes ID on every ping request.

Anyway, now that I know what the problem is, I'll simply adapt to the reality and will use another internal IP address for the second ICMP port forward.

I'm happy to test any ideas though ... out of curiosity what would make it work :)

stephenw10

I'd be interested to see the states on the LAN side. I expect it to show the NAT and the identifier chaging.

But, yeah, it may simply not be possible in which case just move forward with what works.

Steve

rebi

States were not very clear about what exactly was going on so I got for you some packet capture results from LAN:

without the rule
ping IP1 - ICMP Identifier is always 1
ping IP2 - nothing until state is cleared
ping IP2 - ICMP Identifier is always 1

with the rule
ping IP1 - one and the same ICMP Identifier which is a random number, e.g. 3654
ping IP2 - nothing until state is cleared
ping IP2 - one and the same ICMP Identifier which is a random number but different from the one above, e.g. 8346

So, how I understand this ... your idea works but probably too late as the state is already being created with whatever came first to WAN.

Here's why I think it's too late:

ping from my home network - whatever I do, it always creates the state as "<my home ip>:1 -> 192.168.101.2:1"
ping from my office network - it always creates states with random id for each ping "batch", e.g. "<my office ip>:60473 -> 192.168.101.2:60473"
hrping - it sends each ping with a different ID but the state is always created with the ID of the first ping request ... hence I think it's probably too late for the rule to take place.

stephenw10

Mmm, interesting. Two states are created in the firewall, one on WAN and one on LAN.

It could be the WAN state still giving a problem since the NAT happens before the ACL there so both have the same destination. However the NAT is included in the state so I expect it to still be unique.

Clearly something is still conflicting.

Not really anything else we can do there.

Steve