Is IPV6 NAT broken in 2.3 and 2.4?



  • Hi,

    I have more information about the OpenVPN IPV6 problem I reported earlier:

    My VPN provider supports IPV6, but I've not been able to get it to work with pfSense. I get native IPV6 from my ISP, which is handled properly in pfSense. But the addresses need to be translated to the range provided by the VPN provider, but it appears that IPV6 NAT doesn't work in 2.3 or 2.4. The provider was kind enough to setup a copy of pfSense in their lab, and this is what they said in reply to my query:

    I have assumed that the
    clients would use the native IPV6 address based on the ISP, but
    your service would mask that by supplying a shared IPV6 address to
    IPV6 websites – like it does with IPV4. Can you explain?

    Indeed, that's how it should work. The clients could use the ISP address range and pfSense then should NAT those addresses to the single one provided by the OpenVPN server. Unfortunately, the IPv6 NAT function does not seem to work correctly and rewrites the packets with the wrong IPv6 address. pfSense 2.3 always translates packets to the "link-local" address of the tunnel interface, while 2.4 gets it right only half the time.

    That behaviour has been reported a few years ago but is still not fixed,
    apparently:

    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199489
    https://lists.freebsd.org/pipermail/freebsd-pf/2014-September/007441.html

    So, for now there's probably no working solution to route IPv6 over OpenVPN with pfSense.

    I've looked in the bug report database and don't see the exact symptoms reported by the VPN provider, but there are other rejected reports about IPV6 NAT that may be similar. I'm not familiar enough with IPV6 and NAT to be sure. Can someone tell me if this is a bug or a misunderstanding (and, more important, how to get the IPV6 rerouting to work!)

    If this is a bug, how do I go about reporting it?


  • LAYER 8 Global Moderator

    "The clients could use the ISP address range and pfSense then should NAT those addresses to the single one provided by the OpenVPN server."

    It supports NPt, but that is not what the ISP is saying here - they are saying it should nat to the 1 IP provided.. This clearly NOT what NPt does..

    https://doc.pfsense.org/index.php/NPt

    You can use it to map 1 prefix to another prefix via a routed network.. See https://doc.pfsense.org/index.php/Multi-WAN_for_IPv6

    "for now there's probably no working solution to route IPv6 over OpenVPN with pfSense."

    Nonsense.. Route IPv6 over ipv6 all day every day.. This is not what your vpn is trying to do.. They are trying to hand you 1 IPv6, and have you nat all your IPv6 addresses to that 1 single address..  That model is BORKED..  And not how ipv6 is designed to be used.. If they want to provide you ipv6 then they should provide you a routed IPv6 prefix to use on your network..



  • @johnpoz:

    "The clients could use the ISP address range and pfSense then should NAT those addresses to the single one provided by the OpenVPN server."

    It supports NPt, but that is not what the ISP is saying here - they are saying it should nat to the 1 IP provided.. This clearly NOT what NPt does..

    https://doc.pfsense.org/index.php/NPt

    You can use it to map 1 prefix to another prefix via a routed network.. See https://doc.pfsense.org/index.php/Multi-WAN_for_IPv6

    "for now there's probably no working solution to route IPv6 over OpenVPN with pfSense."

    Nonsense.. Route IPv6 over ipv6 all day every day.. This is not what your vpn is trying to do.. They are trying to hand you 1 IPv6, and have you nat all your IPv6 addresses to that 1 single address..  That model is BORKED..  And not how ipv6 is designed to be used.. If they want to provide you ipv6 then they should provide you a routed IPv6 prefix to use on your network..

    I think the ISP may have misspoken about requiring that pfSense map an address range into a single IPV6 address. When I connect to one of their servers from two IPV6-capable iOS devices using IKEv2, their IPV6 addresses are mapped into different IPV6 addresses on the VPN. For example:

    iOS device# 1
        Local IPV6 address (native IPV6 from ISP): 2603:3005:2003:da00:c182:a243:c53e:ef3c
        VPN IPV6 address: 2602:ffc8:1:1::6

    iOS device #2
        Local IPV6 address (native IPV6 from ISP): 2603:3005:2003:da00:d8a6:bc49:99d5:5ed8
        VPN IPV6 address: 2602:ffc8:1:1::8

    Doesn't this mean the VPN is sending the prefix 2602:ffc8:1:1 for mapping?

    I'm not sure the VPN is sending the same prefix for OpenVPN. Their server is sending these PUSH commands related to IPV6:

    redirect-gateway ipv6
    route-ipv6 2000::/3
    ifconfig-ipv6 fdbf:1d37:bbe0:0:10::1250/112 fdbf:1d37:bbe0:0:10::1

    The redirect-gateway IPV6 command isn't recognized by 2.3.4. The VPN provider says it should be ignored and shouldn't matter. 2.4.0RC accepts the command.

    Here's the log from 2.3.4:

    Sep 19 14:08:43 	openvpn 	97800 	VERIFY OK: depth=0, C=CH, ST=Zug, O=Perfect Privacy, CN=Server_chicago.perfect-privacy.com, emailAddress=admin@perfect-privacy.com
    Sep 19 14:08:44 	openvpn 	97800 	Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
    Sep 19 14:08:44 	openvpn 	97800 	Data Channel Encrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
    Sep 19 14:08:44 	openvpn 	97800 	Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
    Sep 19 14:08:44 	openvpn 	97800 	Data Channel Decrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
    Sep 19 14:08:44 	openvpn 	97800 	Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 4096 bit RSA
    Sep 19 14:08:44 	openvpn 	97800 	[Server_chicago.perfect-privacy.com] Peer Connection Initiated with [AF_INET]104.237.193.26:1149
    Sep 19 14:08:46 	openvpn 	97800 	SENT CONTROL [Server_chicago.perfect-privacy.com]: 'PUSH_REQUEST' (status=1)
    Sep 19 14:08:46 	openvpn 	97800 	PUSH: Received control message: 'PUSH_REPLY,topology subnet,redirect-gateway def1,sndbuf 131072,rcvbuf 131072,comp-lzo adaptive,route-gateway 10.0.160.1,redirect-gateway ipv6,route-ipv6 2000::/3,ping 10,ping-restart 60,dhcp-option DNS 167.88.7.164,dhcp-option DNS 217.114.218.30,ifconfig-ipv6 fdbf:1d37:bbe0:0:10::1250/112 fdbf:1d37:bbe0:0:10::1,ifconfig 10.0.160.250 255.255.255.0,peer-id 10'
    Sep 19 14:08:46 	openvpn 	97800 	Options error: unknown --redirect-gateway flag: ipv6
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: timers and/or timeouts modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: LZO parms modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: --sndbuf/--rcvbuf options modified
    Sep 19 14:08:46 	openvpn 	97800 	Socket Buffers: R=[42080->131072] S=[57344->131072]
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: --ifconfig/up options modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: route options modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: route-related options modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: peer-id set
    Sep 19 14:08:46 	openvpn 	97800 	OPTIONS IMPORT: adjusting link_mtu to 1609
    Sep 19 14:08:46 	openvpn 	97800 	ROUTE_GATEWAY 66.31.140.1
    Sep 19 14:08:46 	openvpn 	97800 	ROUTE6: default_gateway=UNDEF
    Sep 19 14:08:46 	openvpn 	97800 	TUN/TAP device ovpnc2 exists previously, keep at program end
    Sep 19 14:08:46 	openvpn 	97800 	TUN/TAP device /dev/tun2 opened
    Sep 19 14:08:46 	openvpn 	97800 	ioctl(TUNSIFMODE): Device busy: Device busy (errno=16)
    Sep 19 14:08:46 	openvpn 	97800 	do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=1
    Sep 19 14:08:46 	openvpn 	97800 	/sbin/ifconfig ovpnc2 10.0.160.250 10.0.160.1 mtu 1500 netmask 255.255.255.0 up
    Sep 19 14:08:46 	openvpn 	97800 	/sbin/route add -net 10.0.160.0 10.0.160.1 255.255.255.0
    Sep 19 14:08:46 	openvpn 	97800 	/sbin/ifconfig ovpnc2 inet6 fdbf:1d37:bbe0:0:10::1250/112
    Sep 19 14:08:46 	openvpn 	97800 	/usr/local/sbin/ovpn-linkup ovpnc2 1500 1609 10.0.160.250 255.255.255.0 init
    Sep 19 14:08:48 	openvpn 	97800 	/sbin/route add -net 104.237.193.26 66.31.140.1 255.255.255.255
    Sep 19 14:08:48 	openvpn 	97800 	/sbin/route add -net 0.0.0.0 10.0.160.1 128.0.0.0
    Sep 19 14:08:48 	openvpn 	97800 	ERROR: FreeBSD route add command failed: external program exited with error status: 1
    Sep 19 14:08:48 	openvpn 	97800 	/sbin/route add -net 128.0.0.0 10.0.160.1 128.0.0.0
    Sep 19 14:08:48 	openvpn 	97800 	ERROR: FreeBSD route add command failed: external program exited with error status: 1
    Sep 19 14:08:48 	openvpn 	97800 	add_route_ipv6(2000::/3 -> fdbf:1d37:bbe0:0:10::1 metric -1) dev ovpnc2
    Sep 19 14:08:48 	openvpn 	97800 	/sbin/route add -inet6 2000::/3 -iface ovpnc2
    Sep 19 14:08:48 	openvpn 	97800 	ERROR: *BSD route add -inet6 command failed: external program exited with error status: 1
    Sep 19 14:08:48 	openvpn 	97800 	add_route_ipv6(2000::/3 -> fdbf:1d37:bbe0:0:10::1 metric -1) dev ovpnc2
    Sep 19 14:08:48 	openvpn 	97800 	/sbin/route add -inet6 2000::/3 -iface ovpnc2
    Sep 19 14:08:48 	openvpn 	97800 	ERROR: *BSD route add -inet6 command failed: external program exited with error status: 1
    Sep 19 14:08:48 	openvpn 	97800 	Initialization Sequence Completed
    Sep 19 14:09:24 	openvpn 	67430 	MANAGEMENT: Client connected from /var/etc/openvpn/client1.sock
    Sep 19 14:09:24 	openvpn 	67430 	MANAGEMENT: CMD 'state 1'
    Sep 19 14:09:24 	openvpn 	67430 	MANAGEMENT: CMD 'status 2'
    Sep 19 14:09:24 	openvpn 	67430 	MANAGEMENT: Client disconnected
    Sep 19 14:09:24 	openvpn 	97800 	MANAGEMENT: Client connected from /var/etc/openvpn/client2.sock
    Sep 19 14:09:24 	openvpn 	97800 	MANAGEMENT: CMD 'state 1'
    Sep 19 14:09:24 	openvpn 	97800 	MANAGEMENT: CMD 'status 2'
    Sep 19 14:09:24 	openvpn 	97800 	MANAGEMENT: Client disconnected 
    

    I'm guessing that the VPN wants addresses mapped to the prefix 2000::/3. Is that right? If so, it's certainly different than the prefix I'm seeing from the iOS IKEv2 clients – different beginning and much shorter.

    As for the route add command failures, I don't always get those. For example, they don't show up in the OpenVPN log after trying to connect to another of the VPN's servers:

    Sep 19 14:18:40 openvpn 31576 Local Options String: 'V4,dev-type tun,link-mtu 1606,tun-mtu 1500,proto UDPv4,comp-lzo,mtu-dynamic,keydir 1,cipher AES-256-CBC,auth SHA512,keysize 256,tls-auth,key-method 2,tls-client'
    Sep 19 14:18:40 openvpn 31576 Expected Remote Options String: 'V4,dev-type tun,link-mtu 1606,tun-mtu 1500,proto UDPv4,comp-lzo,mtu-dynamic,keydir 0,cipher AES-256-CBC,auth SHA512,keysize 256,tls-auth,key-method 2,tls-server'
    Sep 19 14:18:40 openvpn 31576 Local Options hash (VER=V4): '73c06b87'
    Sep 19 14:18:40 openvpn 31576 Expected Remote Options hash (VER=V4): 'ad1c1209'
    Sep 19 14:18:40 openvpn 31576 UDPv4 link local (bound): [AF_INET]66.31.140.75
    Sep 19 14:18:40 openvpn 31576 UDPv4 link remote: [AF_INET]96.9.246.194:149
    Sep 19 14:18:40 openvpn 31576 TLS: Initial packet from [AF_INET]96.9.246.194:149, sid=1464c3d9 ca5e79dc
    Sep 19 14:18:41 openvpn 31576 PID_ERR replay-window backtrack occurred [3] [TLS_AUTH-0] [00__1] 1505845120:5 1505845120:2 t=1505845121[0] r=[-1,64,15,3,1] sl=[59,5,64,528]
    Sep 19 14:18:41 openvpn 31576 VERIFY OK: depth=1, C=CH, ST=Zug, L=Zug, O=Perfect Privacy, CN=Perfect Privacy, emailAddress=admin@perfect-privacy.com
    Sep 19 14:18:41 openvpn 31576 VERIFY OK: nsCertType=SERVER
    Sep 19 14:18:41 openvpn 31576 VERIFY OK: depth=0, C=CH, ST=Zug, O=Perfect Privacy, CN=Server_newyork.perfect-privacy.com, emailAddress=admin@perfect-privacy.com
    Sep 19 14:18:41 openvpn 31576 Data Channel Encrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
    Sep 19 14:18:41 openvpn 31576 Data Channel Encrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
    Sep 19 14:18:41 openvpn 31576 Data Channel Decrypt: Cipher 'AES-256-CBC' initialized with 256 bit key
    Sep 19 14:18:41 openvpn 31576 Data Channel Decrypt: Using 512 bit message hash 'SHA512' for HMAC authentication
    Sep 19 14:18:41 openvpn 31576 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 4096 bit RSA
    Sep 19 14:18:41 openvpn 31576 [Server_newyork.perfect-privacy.com] Peer Connection Initiated with [AF_INET]96.9.246.194:149
    Sep 19 14:18:44 openvpn 31576 SENT CONTROL [Server_newyork.perfect-privacy.com]: 'PUSH_REQUEST' (status=1)
    Sep 19 14:18:44 openvpn 31576 PUSH: Received control message: 'PUSH_REPLY,topology subnet,redirect-gateway def1,sndbuf 131072,rcvbuf 131072,comp-lzo adaptive,route-gateway 10.1.67.1,redirect-gateway ipv6,route-ipv6 2000::/3,ping 10,ping-restart 60,dhcp-option DNS 96.9.249.46,dhcp-option DNS 149.56.153.190,ifconfig-ipv6 fdbf:1d37:bbe0:0:20:3:0:1243/112 fdbf:1d37:bbe0:0:20:3:0:1,ifconfig 10.1.67.243 255.255.255.0,peer-id 3'
    Sep 19 14:18:44 openvpn 31576 Options error: unknown –redirect-gateway flag: ipv6
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: timers and/or timeouts modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: LZO parms modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: --sndbuf/--rcvbuf options modified
    Sep 19 14:18:44 openvpn 31576 Socket Buffers: R=[42080->131072] S=[57344->131072]
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: –ifconfig/up options modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: route options modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: route-related options modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: peer-id set
    Sep 19 14:18:44 openvpn 31576 OPTIONS IMPORT: adjusting link_mtu to 1609
    Sep 19 14:18:44 openvpn 31576 ROUTE_GATEWAY 66.31.140.1
    Sep 19 14:18:44 openvpn 31576 ROUTE6: default_gateway=UNDEF
    Sep 19 14:18:44 openvpn 31576 TUN/TAP device ovpnc1 exists previously, keep at program end
    Sep 19 14:18:44 openvpn 31576 TUN/TAP device /dev/tun1 opened
    Sep 19 14:18:44 openvpn 31576 do_ifconfig, tt->ipv6=1, tt->did_ifconfig_ipv6_setup=1
    Sep 19 14:18:44 openvpn 31576 /sbin/ifconfig ovpnc1 10.1.67.243 10.1.67.1 mtu 1500 netmask 255.255.255.0 up
    Sep 19 14:18:44 openvpn 31576 /sbin/route add -net 10.1.67.0 10.1.67.1 255.255.255.0
    Sep 19 14:18:44 openvpn 31576 /sbin/ifconfig ovpnc1 inet6 fdbf:1d37:bbe0:0:20:3:0:1243/112
    Sep 19 14:18:44 openvpn 31576 /usr/local/sbin/ovpn-linkup ovpnc1 1500 1609 10.1.67.243 255.255.255.0 init
    Sep 19 14:18:45 openvpn 31576 MANAGEMENT: Client connected from /var/etc/openvpn/client1.sock
    Sep 19 14:18:45 openvpn 31576 MANAGEMENT: CMD 'state 1'
    Sep 19 14:18:45 openvpn 31576 MANAGEMENT: Client disconnected
    Sep 19 14:18:46 openvpn 31576 /sbin/route add -net 96.9.246.194 66.31.140.1 255.255.255.255
    Sep 19 14:18:46 openvpn 31576 /sbin/route add -net 0.0.0.0 10.1.67.1 128.0.0.0
    Sep 19 14:18:46 openvpn 31576 /sbin/route add -net 128.0.0.0 10.1.67.1 128.0.0.0
    Sep 19 14:18:46 openvpn 31576 add_route_ipv6(2000::/3 -> fdbf:1d37:bbe0:0:20:3:0:1 metric -1) dev ovpnc1
    Sep 19 14:18:46 openvpn 31576 /sbin/route add -inet6 2000::/3 -iface ovpnc1
    Sep 19 14:18:46 openvpn 31576 add_route_ipv6(2000::/3 -> fdbf:1d37:bbe0:0:20:3:0:1 metric -1) dev ovpnc1
    Sep 19 14:18:46 openvpn 31576 /sbin/route add -inet6 2000::/3 -iface ovpnc1
    Sep 19 14:18:46 openvpn 31576 Initialization Sequence Completed

    Can you provide any insight?


  • LAYER 8 Global Moderator

    "/sbin/ifconfig ovpnc2 inet6 fdbf:1d37:bbe0:0:10::1250/112"

    Borked!!! ULA with a /112 prefix.. WTF??

    There is huge difference in doing ipv6 vpn to a single client running the vpn software, vs doing it where the router is the vpn client and you have ipv6 devices you want to route through that tunnel.. Which is what your trying to do..

    So your ios device working is apples and oranges compared to running the pfsense as the client and having a ipv6 network behind to use that ipv6 vpn tunnel.



  • @johnpoz:

    There is huge difference in doing ipv6 vpn to a single client running the vpn software, vs doing it where the router is the vpn client and you have ipv6 devices you want to route through that tunnel.. Which is what your trying to do..

    So your ios device working is apples and oranges compared to running the pfsense as the client and having a ipv6 network behind to use that ipv6 vpn tunnel.

    You're probably right. I sent a query to the VPN asking what they're thinking with full address vs prefix, but haven't heard back. The VPN has been implemented on various routers that support OpenVPN, and it's available on some routers pre-programmed by Flashrouters. But it's not clear that any of them are working with IPV6. Here are forum tutorials on how to configure the VPN on some routers. Looks like the guy who implemented it on a dd-WRT router set up for IPV6, but one of the commenters said his was working with IPV4 and not IPV6. The OP didn't comment on that, so maybe he doesn't have IPV6 capability. None of the other tutorials mention IPV6.



  • One other point that I didn't mention before: In an email exchange with one of the VPN developers, I asked if they were doing IPV6 over IPV4. He said yes, that's how it works.

    Now, please remember that I'm completely out of my depth here, but does this make a difference? Is pfSense assuming that there's a separate IPV6 connection to the VPN?

    I see that in System–>Advanced-->Networking you can enable IPV6 over IPV4, but it wants an IPV4 address for the tunnel. I guess that could be the VPN's IPV4 address, but I'm also  guessing that this feature is  intended for use with an IPV6 tunnel broker (you mentioned something about using services like HE for that.) Given what's happening at their end, should the VPN be setup as a tunnel broker? Seems like that would limit me to using only one of their servers. I have separate OpenVPN configs for all of their servers, either of which I can implement in two interfaces I defined for the VPN service (so clients can use one of two available servers.)


  • Banned

    @peppersass:

    I see that in System–>Advanced-->Networking you can enable IPV6 over IPV4

    Leave that thing alone. I've never seen anyone to use it for anything. It's certainly not used for VPN nor for GIF tunnels such as HE.



  • @doktornotor:

    @peppersass:

    I see that in System–>Advanced-->Networking you can enable IPV6 over IPV4

    Leave that thing alone. I've never seen anyone to use it for anything. It's certainly not used for VPN nor for GIF tunnels such as HE.

    I believe that's for tunnels, such as he.net.  Others have used that to get IPv6 over an IPv4 connection.  I used to do similar, with a Linux firewall, before my ISP started providing IPv6.  However, it shouldn't be considered a VPN, as there's no encryption etc..


  • Banned

    Erm, no… https://doc.pfsense.org/index.php/Using_IPv6_with_a_Tunnel_Broker

    And looking at that option, the only thing that checkbox does on the entire pfSense is this:

    
            /* DIAG: add ipv6 NAT, if requested */
            if ((isset($config['diag']['ipv6nat']['enable'])) &&
                (is_ipaddr($config['diag']['ipv6nat']['ipaddr'])) &&
                (is_array($FilterIflist['wan']))) {
                    /* XXX: FIX ME!  IPV6 */
                    $natrules .= "rdr on \${$FilterIflist['wan']['descr']} proto ipv6 from any to any -> {$config['diag']['ipv6nat']['ipaddr']}\n";
            }
    
    

    Seeing that code snippet, I'd hazard to say if that config box vanished from the GUI, noone would notice in next 50 years. Not keen to even try, I'd say it breaks the ruleset altogether. The GUI code has been in fact completely broken until about a year ago without anyone noticing.



  • @doktornotor:

    Seeing that code snippet, I'd hazard to say if that config box vanished from the GUI, noone would notice in next 50 years.

    Yeah quite a few cobwebs have been spun over the last 13 years. A fun thing I like to do is run the following command in the /src directory

    find . \( -name "*.inc" -o -name "*.php" \) | xargs grep -En "(XXX|TODO|FIXME)"
    

    Some real gems in there…  :P


Log in to reply