TCP-ACK blocked when using IPV6 over GIF over IPSec

mix_room · Nov 26, 2019, 10:33 AM

I have 2 sites: Site A and Site B.
Both are connected via IPv4 to the internet. Only Site A has native IPv6. Both sites are connected via IPv4 IPSec.
In order to provide IPv6 to Site B I have created a GIF-tunnel to route IPv6 through the IPSec tunnel.

IPv4: Internet <- Site B -- IPSec -- Site A -> Internet
IPv6: Site B -- GIF over IPSec -- Site A -> Internet

Site B has the GIF-tunnel as its default IPv6 gateway. Site A has a static route to the Site B-subnet.

Computers from Site B can ping6 the internet.
However when I try to browse the v6-internet I do not get a connection.

Looking at the firewall logs of Site A I see that the replies are blocked as new incoming messages.
The request gets passed out on the WAN interface, and then the reply get blocked by the GIF-interface.

The GIF-interface has a firewall rule that allows all traffic.

If I run a packet-capture I can see all the packets on both the WAN and the GIF interfaces at Site A. But they do not get passed to Site B.

Does anyone have a suggestion to what might be causing this issue? I have seen similar things with asymmetrical routing, but everything is going and coming through Site A so I do not understand how that could be the problem.

My initial guess would be that for some reason the packets do not arrive on the GIF interface, but then I do not understand how they can arrive at all.

mix_room · Dec 6, 2019, 10:21 AM

Here is a packettrace on the GIF interface at Site A. Packets arrive from Site B.
Packettrace on WAN interface shows them going out and being returned. But the return packets never pass onto GIF on Site A.

11:03:49.404416 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x5165 (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249594115 ecr 0], length 0
11:03:50.404338 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x4d7d (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249595115 ecr 0], length 0
11:03:52.632239 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x44c8 (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249597344 ecr 0], length 0
11:03:56.839370 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x3459 (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249601551 ecr 0], length 0
11:04:05.040516 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x1451 (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249609751 ecr 0], length 0
11:04:21.275910 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0xd4e5 (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249625986 ecr 0], length 0
11:04:53.476716 AF IPv6 (28), length 84: (flowlabel 0xcb16b, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.61213 > 2a02:2e0:3fe:1001:302::.80: Flags [S], cksum 0x571d (correct), seq 3637599348, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 4249658186 ecr 0], length 0
11:10:23.901590 AF IPv6 (28), length 84: (flowlabel 0x7d059, hlim 63, next-header TCP (6) payload length: 40) LOCALNET:LOCALHOST.64956 > 2a02:26f0:18::5c7b:9bda.80: Flags [S], cksum 0x5590 (correct), seq 1794736820, win 65535, options [mss 1440,nop,wscale 6,sackOK,TS val 2891983168 ecr 0], length 0

Why does the GIF interface see these messages as incoming to it? Should they not be passing out on the GIF interface, in which case the FW rules would not apply.

JKnott · Dec 6, 2019, 11:43 AM

@mix_room

Why are you even using that GIF tunnel? Why don't you run IPv6 directly over IPSec? That's what I do with my notebook computer & OpenVPN. With that, my computer gets IPv6 through the VPN, when I'm at an IPv4 only location.

mix_room · Dec 9, 2019, 10:21 AM

@JKnott said in TCP-ACK blocked when using IPV6 over GIF over IPSec:

@mix_room

Why are you even using that GIF tunnel? Why don't you run IPv6 directly over IPSec?

Because the documentation states that you can not do this with IPsec. That was the first thing to try.
https://docs.netgate.com/pfsense/en/latest/book/ipsec/ipsec-and-ipv6.html

I tried setting up a VTI-tunnel on the link, but that doesn't work. Can not even ping the two ends of the link.

NogBadTheBad · Dec 9, 2019, 10:24 AM

Have you tried IKEv2 ?

mix_room · Dec 9, 2019, 11:36 AM

@NogBadTheBad said in TCP-ACK blocked when using IPV6 over GIF over IPSec:

Have you tried IKEv2 ?

Not yet.

Note to self and others regarding VTI: https://redmine.pfsense.org/issues/9801

Edit: after manually applying the fix from issue 9801 the tunnel passes IPv6 ping. Doesn't seem like one needs to use the GIF tunnel. Makes everything a bit easier.

JKnott · Dec 9, 2019, 3:28 PM

@mix_room said in TCP-ACK blocked when using IPV6 over GIF over IPSec:

Because the documentation states that you can not do this with IPsec.

It says you can't with IKEv1. Select IKEv2 or Auto.

mix_room · Dec 9, 2019, 3:28 PM

@JKnott said in TCP-ACK blocked when using IPV6 over GIF over IPSec:

It says you can't with IKEv1. Select IKEv2 or Auto.

Sure does. It helps if one reads properly.

Added new VTI, set up OSPFv6 and got the sites to speak to each other using ULA addresses. Next round would be to get it working with proper ones.

However - I still believe the GIF solution should have worked, and am confused as to why it seems the traffic going to wrong way.

mix_room · Dec 12, 2019, 11:32 AM

Problem "work-arounded" by using VTI-interfaces with IPv6 over IPSec tunnel.

JKnott · Dec 12, 2019, 11:51 AM

@mix_room said in TCP-ACK blocked when using IPV6 over GIF over IPSec:

Why are you even using that GIF tunnel? Why don't you run IPv6 directly over IPSec?

Because the documentation states that you can not do this with IPsec.

One other thing, IPSec was originally designed for IPv6 and back ported to IPv4. So, it's highly unlikely it would not support IPv6.