NPt not working on WAN interface when IPv6 is provided via a 6RD tunnel. Bug?

risola316

@Alitai Yes, I did update the filter.inc file and then rebooted and then reboot but still had the same issue once I reverted the changes for my workaround. I did mention this in my initial post but I should clarify the file was updated in "/etc/inc/filter.inc" I pasted the wrong file location (included /src/) in my original post.

Both of the links you posted appear to be for fix referenced in bug #7142 was there another change you also meant to link to?

risola316

I'm wondering if this is due to a similar bug to what was corrected in #7142. In /etc/inc/filter.inc/filter_nat_rules_generate() I see where the NPt filtering rules get applied but it only appears to occur for the main interface. For 6rd and 6to4 interfaces is there additional logic needed to correctly apply the NPt filters rules to the virtual interfaces?

viktor_g

Fixed in https://redmine.pfsense.org/issues/7142
You need to apply Patch ID bae04c3730d58f59a71ec3801e0e144f6452b7a6

risola316

@viktor_g Thanks for your reply. I did stumble across bug #7142 in my initial attempt to resolve this problem and I have replaced my filter.inc file with the one linked to in the patch ID you provided. However, after applying this update I'm still not seeing the NPt rules being appropriately applied to the _stf interface. I did verify that the changes were present in the file and tried both a filter reload and a reboot of the system to ensure they were in effect. I did notice there is another place in filter.inc where the NPt rules are generated (/etc/inc/filter.inc/filter_nat_rules_generate()). Does something similar to the fix in #7142 need to be done in that function as well? Is it possible there are also other patches I need ( I did see there were some other 6rd bug fixes targeted for the 2.5 release)?
I'm happy to provide any additional information/logs if that would be helpful. Any additional thoughts/suggestions are appreciated.

viktor_g

@risola316 Did you apply Patch bae04c3730d58f59a71ec3801e0e144f6452b7a6 ?
I have successfully tested the firewall rules, NPt and 1:1 NAT after applying this patch.
Please show the /tmp/rules.debug and pfctl -s rules after applying this patch and system reboot.

risola316

@viktor_g I do believe that I applied the appropriate patch. Please see below for the relevant section of the patched file.

I wasn't able to paste the info into my post so I've included it in the following links.

/tmp/rules.debug

https://pastebin.com/wptxDd4d

pfctl -s rules

https://pastebin.com/9rJa52QP

Thank you again for your help.

viktor_g

@risola316 That seems correct:

WAN0_CNT = "{ pppoe0 opt1_stf }"
...
binat on $WAN0_CNT inet6 from fd83:8359:2e69:54d5::/64 to any -> 2602:d8:a1aa:bd00::/64
binat on $WAN0_CNT inet6 from any to 2602:d8:a1aa:bd00::/64 -> fd83:8359:2e69:54d5::/64

But it seems that you forget fw pass rules 'from outside to 2602:d8:a1aa:bd00::/64'

Also please show the output of pfctl -s nat

risola316

@viktor_g said in NPt not working on WAN interface when IPv6 is provided via a 6RD tunnel. Bug?:

pfctl -s nat

I'm not sure what might cause the fw pass rules for outside not to be set. I'm using the default ruleset that was configured for the 6rd gateway/firewall. These are also the same I have configured for my other WAN (DHCPv6) which I am able to get connectivity to from the LAN.

Here is the output of the requested command
#anchor(pfctl -s nat)

[2.4.5-RELEASE][admin@pfSense.localdomain]/root: pfctl -s nat
no nat proto carp all
nat-anchor "natearly/*" all
nat-anchor "natrules/*" all
nat on vmx2 inet proto tcp from 192.168.13.0/24 to 192.168.100.0/24 -> (vmx2) port 1024:65535 round-robin
nat on vmx1 inet proto tcp from 192.168.13.0/24 to 192.168.0.0/24 -> 192.168.0.13 port 1024:65535
nat on vmx2 inet from <tonatsubnets> to any port = isakmp -> (vmx2) round-robin static-port
nat on vmx2 inet6 from <tonatsubnets> to any port = isakmp -> (vmx2) round-robin static-port
nat on vmx2 inet from <tonatsubnets> to any -> (vmx2) port 1024:65535 round-robin
nat on vmx2 inet6 from <tonatsubnets> to any -> (vmx2) port 1024:65535 round-robin
nat on pppoe0 inet from <tonatsubnets> to any port = isakmp -> 216.161.170.189 static-port
nat on opt1_stf inet from <tonatsubnets> to any port = isakmp -> 216.161.170.189 static-port
nat on pppoe0 inet6 from <tonatsubnets> to any port = isakmp -> 2602:d8:a1aa:bd00:: static-port
nat on opt1_stf inet6 from <tonatsubnets> to any port = isakmp -> 2602:d8:a1aa:bd00:: static-port
nat on pppoe0 inet from <tonatsubnets> to any -> 216.161.170.189 port 1024:65535
nat on opt1_stf inet from <tonatsubnets> to any -> 216.161.170.189 port 1024:65535
nat on pppoe0 inet6 from <tonatsubnets> to any -> 2602:d8:a1aa:bd00:: port 1024:65535
nat on opt1_stf inet6 from <tonatsubnets> to any -> 2602:d8:a1aa:bd00:: port 1024:65535
no rdr proto carp all
rdr-anchor "tftp-proxy/*" all
rdr-anchor "miniupnpd" all
binat on vmx2 inet6 from fd83:8359:2e69:54d5::/64 to any -> 2600:8800:4700:3820::/64
binat on vmx2 inet6 from any to 2600:8800:4700:3820::/64 -> fd83:8359:2e69:54d5::/64
binat on pppoe0 inet6 from fd83:8359:2e69:54d5::/64 to any -> 2602:d8:a1aa:bd00::/64
binat on pppoe0 inet6 from any to 2602:d8:a1aa:bd00::/64 -> fd83:8359:2e69:54d5::/64
[2.4.5-RELEASE][admin@pfSense.localdomain]/root:

One other thing I am surprised to see, is "opt1_stf" showing up in Diagnostics/States as I thought that should reflect WAN0_CNT

This opt1_stf disappears when I do the workaround described in my original post.

risola316

@viktor_g I'm not sure if this will help but I figured I would post the items your requested for the case where the workaround as described above is applied. After the workaround is applied I do have IPv6 connectivity from my LAN, but this workaround poses issues as I should not be able to create an interface assignment for the _stf interface.

Post Workaround: pfctl -s rules

https://pastebin.com/0smufdep

Post Workaround: pfctl -s rules

https://pastebin.com/iq6zyBqS

Post Workaround: pfctl -s nat

Maybe this issue is related to the combination of IPV6 via 6rd over a pppoe connection. Looking at the binat rules in the case that doesn't work the only difference I can see is that the NPt rules are being applied to the pppoe interface instead of to the opt1_stf interface. In the binat instruction that works the GRP_WAN_CNT is used which consists of (WAN0_CNT, opt1_stf).

[2.4.5-RELEASE][admin@pfSense.localdomain]/root: pfctl -s nat
no nat proto carp all
nat-anchor "natearly/*" all
nat-anchor "natrules/*" all
nat on vmx2 inet proto tcp from 192.168.13.0/24 to 192.168.100.0/24 -> (vmx2) port 1024:65535 round-robin
nat on vmx1 inet proto tcp from 192.168.13.0/24 to 192.168.0.0/24 -> 192.168.0.13 port 1024:65535
nat on vmx2 inet from <tonatsubnets> to any port = isakmp -> (vmx2) round-robin static-port
nat on vmx2 inet6 from <tonatsubnets> to any port = isakmp -> (vmx2) round-robin static-port
nat on vmx2 inet from <tonatsubnets> to any -> (vmx2) port 1024:65535 round-robin
nat on vmx2 inet6 from <tonatsubnets> to any -> (vmx2) port 1024:65535 round-robin
nat on pppoe0 inet from <tonatsubnets> to any port = isakmp -> 216.161.170.189 static-port
nat on opt1_stf inet from <tonatsubnets> to any port = isakmp -> 216.161.170.189 static-port
nat on pppoe0 inet6 from <tonatsubnets> to any port = isakmp -> 2602:d8:a1aa:bd00:: static-port
nat on opt1_stf inet6 from <tonatsubnets> to any port = isakmp -> 2602:d8:a1aa:bd00:: static-port
nat on pppoe0 inet from <tonatsubnets> to any -> 216.161.170.189 port 1024:65535
nat on opt1_stf inet from <tonatsubnets> to any -> 216.161.170.189 port 1024:65535
nat on pppoe0 inet6 from <tonatsubnets> to any -> 2602:d8:a1aa:bd00:: port 1024:65535
nat on opt1_stf inet6 from <tonatsubnets> to any -> 2602:d8:a1aa:bd00:: port 1024:65535
no rdr proto carp all
rdr-anchor "tftp-proxy/*" all
rdr-anchor "miniupnpd" all
binat on vmx2 inet6 from fd83:8359:2e69:54d5::/64 to any -> 2600:8800:4700:3820::/64
binat on vmx2 inet6 from any to 2600:8800:4700:3820::/64 -> fd83:8359:2e69:54d5::/64
binat on GRP_WAN_CNT inet6 from fd83:8359:2e69:54d5::/64 to any -> 2602:d8:a1aa:bd00::/64
binat on GRP_WAN_CNT inet6 from any to 2602:d8:a1aa:bd00::/64 -> fd83:8359:2e69:54d5::/64
[2.4.5-RELEASE][admin@pfSense.localdomain]/root:

States showing NPt rules applied on _stf interface (OPT6)

Thanks

risola316

@viktor_g I added some additional log messages to the filter Reload Status and it appears that floating rules are only generated for the WAN interface. They are not generated for the OPT1 interface even thought it is configured as a WAN.

Filter Reload Status

Initializing
Creating aliases
Creating gateway group item...
Generating Limiter rules
Generating NAT rules - Test
Creating 1:1 rules...
1) The interface: wan was found for NPt rules.
2) The interface: wan is being setup for NPt rules.
3) The interface: WAN1_COX is having it's NPt rules saved.
1) The interface: opt1 was found for NPt rules.
2) The interface: opt1 is being setup for NPt rules.
3) The interface: WAN0_CNT is having it's NPt rules saved.
Creating advanced outbound rule Cox Modem Management Access (.100.1)
Creating advanced outbound rule CenturyLink Modem Management Access (.0.1)
Creating outbound NAT rules
Creating automatic outbound rules
Setting up TFTP helper
Generating filter rules
Creating default rules
Pre-caching ICMPv6 Allow Rules...
Creating filter rule ICMPv6 Allow Rules ...
Creating filter rules ICMPv6 Allow Rules ...
1) Creating floating filter rules for interface wan ...
2) array_key_exists for interface wan ...
3a) isset(Array['if']) for interface wan ...
Setting up pass/block rules
Setting up pass/block rules ICMPv6 Allow Rules
Creating rule ICMPv6 Allow Rules
Pre-caching Default allow LAN to any rule...
Creating filter rule Default allow LAN to any rule ...
Creating filter rules Default allow LAN to any rule ...
Setting up pass/block rules
Setting up pass/block rules Default allow LAN to any rule
Creating rule Default allow LAN to any rule
Pre-caching Default allow LAN IPv6 to any rule...
Creating filter rule Default allow LAN IPv6 to any rule ...
Creating filter rules Default allow LAN IPv6 to any rule ...
Setting up pass/block rules
Setting up pass/block rules Default allow LAN IPv6 to any rule
Creating rule Default allow LAN IPv6 to any rule
Creating IPsec rules...
Creating uPNP rules...
Generating ALTQ queues
Loading filter rules
Setting up logging information
Setting up SCRUB information
Processing down interface states
Running plugins
Done

Additional Logging: filter.inc

filter.inc_filter_nat_rules_generate

/* Add binat rules for Network Prefix translation */
	if (is_array($config['nat']['npt'])) {
		foreach ($config['nat']['npt'] as $rule) {
			if (isset($rule['disabled'])) {
				continue;
			}

			if (!$rule['interface']) {
				$natif = "wan";
			} else {
				$natif = $rule['interface'];
			}
			update_filter_reload_status(sprintf(gettext("1) The interface: %s was found for NPt rules."), $natif));
			if (!isset($FilterIflist[$natif])) {
				continue;
			}
			update_filter_reload_status(sprintf(gettext("2) The interface: %s is being setup for NPt rules."), $natif));

			$srcaddr = filter_generate_address($rule, 'source');
			$dstaddr = filter_generate_address($rule, 'destination');

			$srcaddr = trim($srcaddr);
			$dstaddr = trim($dstaddr);

			$natif = $FilterIflist[$natif]['descr'];

			/* Do not form an invalid NPt rule.
			 * See https://redmine.pfsense.org/issues/8575 */
			if (!(is_subnetv6($srcaddr) || is_ipaddrv6($srcaddr)) ||
			    !(is_subnetv6($dstaddr) || is_ipaddrv6($dstaddr))) {
				continue;
			}

			update_filter_reload_status(sprintf(gettext("3) The interface: %s is having it's NPt rules saved."), $natif));
			$natrules .= "binat on \${$natif} inet6 from {$srcaddr} to any -> {$dstaddr}\n";
			$natrules .= "binat on \${$natif} inet6 from any to {$dstaddr} -> {$srcaddr}\n";

		}
	}

filter.inc_filter_generate_user_rule

/* Check to see if the interface is in our list */
	if (isset($rule['floating'])) {
		if (isset($rule['interface']) && $rule['interface'] <> "") {
			$interfaces = explode(",", $rule['interface']);
			$ifliste = "";
			foreach ($interfaces as $iface) {
				update_filter_reload_status(sprintf(gettext("1) Creating floating filter rules for interface %s ..."), $iface));
				if (array_key_exists($iface, $FilterIflist)) {
					update_filter_reload_status(sprintf(gettext("2) array_key_exists for interface %s ..."), $iface));
					if (isset($FilterIflist[$iface]['if'])) {
						update_filter_reload_status(sprintf(gettext("3a) isset($FilterIflist[$iface]['if']) for interface %s ..."), $iface));
						$ifliste .= " " . $FilterIflist[$iface]['if'] . " ";
						if (($FilterIflist[$iface]['type6'] == '6rd') ||
						    ($FilterIflist[$iface]['type6'] == '6to4')) {
							update_filter_reload_status(sprintf(gettext("3) 6rd/6to4 for interface %s ..."), $iface));
							$ifliste .= " " . $FilterIflist[$iface]['ifv6'] . " ";
						}
					} elseif (isset($FilterIflist[$iface][0]['if'])) {
						update_filter_reload_status(sprintf(gettext("3b) isset($FilterIflist[$iface][0]['if']) for interface %s ..."), $iface));
						$ifliste .= " " . $FilterIflist[$iface][0]['if'] . " ";
					}
				}
			}

===

viktor_g

@risola316 Thanks for info
Please check https://redmine.pfsense.org/issues/10757#note-3
(apply it after #7142)

risola316

@viktor_g Thanks so much for your help. After applying the second patch things are working as expected. Thanks again.