Performance regression 2.7.2 to 2.8

fathead

So how exactly does NAT64 fail?

Native v6 traffic is normal.
Outside NAT64 so far is not working with a VIP set on wan.
The VIPs them selves are reachable example 64:ff9b::10.0.0.3

WAN	ipv6-icmp	10.0.0.4:1 (fdbb::8[1]) -> 77.47.127.138:8 (64:ff9b::100:1[1])	NO_TRAFFIC:NO_TRAFFIC	2 / 2	160 B / 120 B

What are you using that VIP for?

v4 VIPs for ping, v6 VIPs for DNS.

stephenw10

@fathead said in Performance regression 2.7.2 to 2.8:

Outside NAT64 so far is not working with a VIP set on wan.

OK so you are setting that VIP just as something to ping from a V6 only client device?

Can I assume it still responds to ping from an internal IPv4 client?

fathead

@stephenw10

OK so you are setting that VIP just as something to ping from a V6 only client device?

Can I assume it still responds to ping from an internal IPv4 client?

Yes and yes.

fathead

Furthermore 64:ff9b::c0a8:100/120 is passed on the lan, at some point in the past two days this stopped working.

stephenw10

Oh so dhcpv6 stopped working you mean by passed?

fathead

@stephenw10
Firewall rule set to pass LAN subnets to 64:ff9b::c0a8:100/120
64:ff9b::c0a8:101(pfSense it self) was the only address replying.

tcpdump on pfSense show the wrong ip(the VIP) and not the interface IP(192.168.1.1).
Removing all v4 VIPs from lan restores reachabilit. v4 VIPs on localhost are different some how.
Before:

16:52:35.520044 IP 10.0.0.1 > 192.168.1.100: ICMP echo request, id 179, seq 1, length 64

After v4 VIPs removed:

17:08:30.209222 IP 192.168.1.1 > 192.168.1.100: ICMP echo request, id 182, seq 10740, length 64
17:08:30.209351 IP 192.168.1.100 > 192.168.1.1: ICMP echo reply, id 182, seq 10740, length 64

The 64:ff9b::c0a8:100/120 rule is above the 64:ff9b::/96 rule.
NAT64 and v4 VIPs alias on wan and/or lan are not playing nicely.

marcosm

The issue is that the "Automatic" NAT64 source option in the rule leaves things up to the OS itself. Hence the source tends to be the first address on the interface (e.g. 10.0.0.1). To address that issue a GUI option exists to override the source. For example:

fathead

@marcosm The source has always been set to "LAN subnets".

stephenw10

It's the 'Address Family Translation' source that must be set to 'LAN address' rather than the default 'Automatic' where it's choosing the VIP.

fathead

@stephenw10
'Address Family Translation' is also set to 'LAN address'.

stephenw10

Hmm, then maybe it's not matching the test traffic somehow?

fathead

@stephenw10

Hmm, then maybe it's not matching the test traffic somehow?

How to see what it is matching with?

For testing, made a do nothing OPT1, setting v4 VIPs on loopback and OPT1 does not seem break lan to lan or lan to wan.

stephenw10

Mmm, I'd only expect the VIP to make any difference on LAN. For some reason it's choosing to use the first IP on the interface (the VIP) as the source for the translation.

If you run pfctl -vss you can see the states with the rules that created them.

Then if you run pfctl -vvsr you can see the ruleset with numbers to see what the rule is.

However since this is a NAT64 rule I'm not 100% sure how it will appear!

fathead

@stephenw10

Mmm, I'd only expect the VIP to make any difference on LAN.

Exactly, 64:ff9b::c0a8:100/120 is affected by ipv4 LAN VIPs.

pfctl -vss

igb1 ipv6-icmp 10.0.0.4:1 (fd22::8[1]) -> 192.168.1.100:8 (64:ff9b::c0a8:164[1])       NO_TRAFFIC:NO_TRAFFIC
   age 00:00:18, expires in 00:00:07, 4:4 pkts, 320:240 bytes, rule 117

pfctl -vvsr

@117 pass in quick on igb1 inet6 from fd22::/64 to 64:ff9b::c0a8:100/120 flags S/SA keep state (if-bound) label "USER_RULE" label "id:1748759142" ridentifier 1748759142 af-to inet from (igb1)
  [ Evaluations: 760       Packets: 52        Bytes: 3640        States: 1     ]

stephenw10

Ok I think we see a problem here. Digging...

marcosm

Thanks for testing! A redmine has been created for this issue:
https://redmine.pfsense.org/issues/16250

A patch is available there to test with the System Patches package; make sure to reload the filter under Status > Filter Reload after applying the patch.

fathead

@marcosm
Patch is working for this.

fathead

@marcosm
So how does this work?
64:ff9b::/96 automatic, is the wan address always used for this?

igb1 ipv6-icmp 100.92.34.122:1 (fd22::8[1]) -> 192.168.1.100:8 (64:ff9b::c0a8:164[1])       NO_TRAFFIC:NO_TRAFFIC
   age 00:00:10, expires in 00:00:10, 3:3 pkts, 240:180 bytes, rule 115
@115 pass in quick on igb1 inet6 from fd22::/64 to 64:ff9b::/96 flags S/SA keep state (if-bound) label "USER_RULE: Default Allow IPv6 to IPv4 via NAT64" label "id:1748759054" ridentifier 1748759054 af-to inet from (pppoe0)
  [ Evaluations: 8186      Packets: 2840190   Bytes: 2861343252  States: 9     ]
  [ Inserted: uid 0 pid 0 State Creations: 17    ]
  [ Last Active Time: Tue Jun 10 03:09:22 2025 ]

stephenw10

Yes by default it uses the first address of whatever interface that traffic is leaving from. Hence the rule for LAN tried to the use the VIP there before the source was set correctly.

fathead

@stephenw10
Always uses the default gateway?