IPV6 setup with Hyperoptic (UK ISP)

yellowbrick · Sep 25, 2018, 4:41 PM

Hello,

After finally getting Hyperoptic fibre Internet in my building, I switched out their ZTE router with my Netgate SG-3100. While IPv4 worked immediately, at first, no luck with getting an IPV6 address via WAN DHCP6 (Hyperoptic offers dual stack, native IPv6, not tunnelled). I did try the Track Interface ‘trick’ people have been mentioning, but after a couple of emails to Hyperoptic support, and perhaps some change from their end and I finally have a /56 PD from them.

Also, for info:
-I cloned the Hyperoptic MAC on my SG-3100, and typed in RAW DUID from the Hyperoptic Router to see if would help get WAN DHCP6 to work. Since it started working, I have not reset them back to defaults to see if I will still get a WAN DHCP6 address.
-I have LAN RA set to ‘Assisted’. Everything else is blank/default.
-Firewall Rules WAN: Allow IPv6 ICMP any from any any TO any any
-Firewall Rules LAN: Allow IPv6 * from 'LAN Net' any TO any any (* as Gateway)

-Clients on the network (LAN) now work with IPv6 and https://test-ipv6.com/ shows 10/10 result.
-Default IPv6 Gateway WAN_DHCP6 is an fe80:: address (see below)

A couple of questions:

On the WAN interface, I do not get a ‘real’ IPV6 Gateway. The WAN gets a Link Local IPv6 Gateway only (fe80:: …) . (This was true even with the Hyperoptic ZTE router). Is this ‘normal’?
However I cannot ping6 from the firewall to the internet anywhere. I can ping6 from firewall to IPv6 address on the LAN. I do not see any blocked packets in the Firewall Logs. What could be happening here?

Many TIA!

yellowbrick · Sep 30, 2018, 2:02 PM

On 1. above, Hyperoptic support have responded the IPv6 gateway will be assigned using SLAAC only, so they cannot tell me a non-link-local address to use.

On 2.: still cannot ping using IPv6 from the pfSense router...clients on the LAN are working fine (i.e. can ping6 from a client on LAN to anywhere).

Point to note, IPv4 out from pfSense works fine...

Any ideas?

TIA

msf2000 · Sep 30, 2018, 2:46 PM

@yellowbrick said in IPV6 setup with Hyperoptic (UK ISP):

/56 PD

Did the ISP assign you an IPv6 subnet of size /56? That seems confusing since the comment you made later that the ISP is using SLAAC. If true, that would explain why DHCPv6 on the WAN is not working.

Also, have you tried setting the WAN interface "IPv6 configuration type" to "SLAAC"?

yellowbrick · Sep 30, 2018, 2:59 PM

Yes, pfSense is definitely getting a /56 PD, from the logs (manually blanked out):

Sep 30 07:22:22 dhcp6c 98391 IA_PD prefix: 2a01:xxxx:xxxx:xxxx::/56 pltime=72000 vltime=86400
Sep 30 07:22:22 dhcp6c 98391 get DHCP option IA_PD prefix, len 25
Sep 30 07:22:22 dhcp6c 98391 IA_PD: ID=0, T1=43200, T2=64800
Sep 30 07:22:22 dhcp6c 98391 get DHCP option IA_PD, len 41
Sep 30 07:22:22 dhcp6c 98391 set IA_PD
Sep 30 07:22:22 dhcp6c 98391 set IA_PD prefix

Also, both LAN and OPT1 interfaces are set to track WAN with prefix ID 0 and 1 respectively. They both are working with clients getting correct IPv6 addresses in the correct /64 subnet.

I agree the 'SLAAC' comment is confusing, but my ISP actually says the upstream gateway for the WAN interface (which gets a valid WAN IPv6 address not in in /56 PD) is only assigned via SLAAC. However, I suspect they mean link-local discovery (?) as the IPv6 gateway is an fe80:: address.

Also, as I mentioned, clients can exit using IPv6 without any problems...just the pfSense box itself cannot!

I have not tried setting WAN to SLAAC...will try it now.

Thanks.

yellowbrick · Sep 30, 2018, 3:12 PM

Setting the WAN interface to SLAAC results IPv6 not working at all...no WAN IPv6, no LAN/OPT IPv6 addresses, cannot route out, etc.

Changing WAN back to DHCPv6 means WAN gets a /128, /56 PD is received, and LAN/OPT clients start working.

strange...

msf2000 · Sep 30, 2018, 3:29 PM

Ah, I missed the part about LAN being set to track the WAN interface.... Yeah, let's leave that as is.

Sounds like the ISP's DHCPv6 server is giving out ranges (ra) but not assignments (managed), or possibly routing is not correct...
What version of pfSense are you running? People (here) have been posting DHCPv6 problems with the latest version 2.4.4.

yellowbrick · Sep 30, 2018, 3:34 PM

I am running 2.4.4.

Just to point out again, all the clients on LAN and OPT do get addresses in the correct /64 and are able to ping6 out to the net.

It's just the pfSense box (SG-3100) itself that cannot exit out to WAN using IPv6 (ssh'd into SG-3100).

msf2000 · Sep 30, 2018, 3:38 PM

OK, now I suspect the firewall.
If you change the FW rule on the LAN from 'LAN Net' to 'any', does that make it work? If not, enable logging on all WAN/LAN rules (and the default rules), and see if the FW logs turn up any clues.

JKnott · Sep 30, 2018, 6:15 PM

@yellowbrick said in IPV6 setup with Hyperoptic (UK ISP):

Setting the WAN interface to SLAAC results IPv6 not working at all...no WAN IPv6, no LAN/OPT IPv6 addresses, cannot route out, etc.
Changing WAN back to DHCPv6 means WAN gets a /128, /56 PD is received, and LAN/OPT clients start working.
strange...

That's the way mine works. DHCPv6-PD assigns an address to the WAN interface and supplies the prefix for the LANs.

yellowbrick · Oct 1, 2018, 6:40 AM

I don't see anything in the logs indicating blocked packets on WAN or LAN.

Not sure if this has anything to do with it, but I do have my WAN using a MAC clone from the ISP's router. Without this, I am not able to get a DHCPv6 address on WAN at all.

adhodgson · Oct 13, 2019, 9:07 PM

Hi,

Did you ever get this sorted? I am getting the same issue (though I didn't clone any MAC or anything else). Clients on the LAN are getting IPV6 ok, but the pfSense box itself cannot go out via it's WAN address to the Internet over IPV6. I suspect some type of routing on the Hyperoptic side is broken.

Andrew.

Derelict · Oct 14, 2019, 12:43 AM

So you're on this same ISP?

JKnott · Oct 14, 2019, 1:25 AM

@yellowbrick said in IPV6 setup with Hyperoptic (UK ISP):

On the WAN interface, I do not get a ‘real’ IPV6 Gateway. The WAN gets a Link Local IPv6 Gateway only (fe80:: …) . (This was true even with the Hyperoptic ZTE router). Is this ‘normal’?

I just noticed this. If pfSense doesn't have a WAN address, other than link local, it can't communicate with anything. On my system, I have a /128 WAN address. Devices on the LAN will still work fine though, as that /128 address is not used for routing.

Derelict · Oct 14, 2019, 1:27 AM

Poster is not saying there is no IPv6 GUA address on WAN, just that the gateway is link-local, which is normal.

yellowbrick · Oct 14, 2019, 5:45 AM

@adhodgson
I am getting pretty much the same as you:
-WAN gets a /128 GUA
-WAN gateway is a LLA
-LAN clients can ping6 OK
-Firewall itself cannot ping6 (other than to WAN gateway)

I gave up on trying to figure it out...pfSense updates were slow, as IPv6 is preferred so it has to fail before going to IPv4. I fixed this by preferring IPv4 in System->Advanced->Networking

Will try to find time to dig in deeper...

adhodgson · Oct 14, 2019, 10:38 AM

Hi,

Yes I am using the same ISP as the original poster. Just for reference my WAN output is:

WAN Interface (wan, igb0)
Status: up
DHCP: up
MAC Address: 00:1a:8c:4b:36:6c
IPv4 Address: 88.98.222.211
Subnet mask IPv4: 255.255.255.248
Gateway IPv4: 88.98.222.209
IPv6 Link Local: fe80::21a:8cff:fe4b:366c%igb0
IPv6 Address: 2a01:4b00:367b:5801:641a:32ef:9a9f:817a
Subnet mask IPv6: 128
Gateway IPv6: fe80::2ab4:48ff:fe87:c9fb
DNS servers: 127.0.0.1, 188.172.144.120, 141.0.144.64
MTU: 1500
Media: 1000baseT <full-duplex>

I believe this is normal, the problem is that for some reason the IP address 2a01:4b00:367b:5801:641a:32ef:9a9f:817a is not being routed correctly. I can see traffic going to the default gateway on a packet capture, but no return traffic, and if I try and ping that host from an external interface the traffic doesn't even seem to be visible at the pfSense box. My fix is to do what the original poster has done, prefer IPV4 to IPV6 connectivity.

I am still probably going to raise a case with the ISP in the first instance but am not hugely hopeful of a fix until I do more work at my end. The main question I want to get an answer to is whether they expect traffic on this WAN address to be routable or not, because in the situation where you have the customer provided router, you probably wouldn't even see this scenario in day-to-day operation, we are only seeing it because we are trying to access sites on the firewall box itself.

Thanks.
Andrew.

JKnott · Oct 14, 2019, 10:44 AM

@Derelict said in IPV6 setup with Hyperoptic (UK ISP):

Poster is not saying there is no IPv6 GUA address on WAN, just that the gateway is link-local, which is normal.

He also said "On the WAN interface, I do not get a ‘real’ IPV6 Gateway." and he can't ping from pfSense, but can from the LAN. He won't be able to ping very far using only a link local address.

JKnott · Oct 14, 2019, 10:47 AM

@adhodgson said in IPV6 setup with Hyperoptic (UK ISP):

I believe this is normal, the problem is that for some reason the IP address 2a01:4b00:367b:5801:641a:32ef:9a9f:817a is not being routed correctly. I can see traffic going to the default gateway on a packet capture, but no return traffic, and if I try and ping that host from an external interface the traffic doesn't even seem to be visible at the pfSense box. My fix is to do what the original poster has done, prefer IPV4 to IPV6 connectivity.

What does traceroute show? If you can, also try a traceroute to your WAN address from elsewhere. I tethered to my cell phone for that.

adhodgson · Oct 14, 2019, 1:00 PM

I can ping and trace route to the address from inside the LAN but that should be expected behaviour as pfSense knows about that address. Trace route from outside stops well before the Hyperoptic routers:

andrew@samwise:~$ traceroute 2a01:4b00:367b:5801:641a:32ef:9a9f:817a
traceroute to 2a01:4b00:367b:5801:641a:32ef:9a9f:817a (2a01:4b00:367b:5801:641a:32ef:9a9f:817a), 30 hops max, 80 byte packets
1 2001-41c8-0051-0500-0000-0000-0000-0003.no-reverse-dns-set.uk0.bigv.io (2001:41c8:51:500::3) 1.892 ms 1.917 ms 1.834 ms
2 4008.be1.cr4.man.bytemark.co.uk (2001:41c8:2000:4::1) 1.528 ms 1.524 ms 1.589 ms
3 2001:1b40:f900:8a61::1:1 (2001:1b40:f900:8a61::1:1) 1.100 ms 1.293 ms 1.259 ms
4 be16.asr01.ld5.as20860.net (2001:1b40:f000:10a:202::1) 12.704 ms 12.436 ms 12.555 ms
5 * * *
[...]
30 * * *

Andrew.

JKnott · Oct 14, 2019, 1:00 PM

@adhodgson said in IPV6 setup with Hyperoptic (UK ISP):

I can ping and trace route to the address from inside the LAN but that should be expected behaviour as pfSense knows about that address. Trace route from outside stops well before the Hyperoptic routers:

Then it's a problem with the ISP. They're supposed to advertise the prefix via a routing protocol such as OSPF. If they don't do that, then the rest of the world can't reach it. You can use ping6 -S <source address> to force the ping from the LAN interface, which does work.

This indicates one of the differences between IPv4 & IPv6. With IPv4, you need a routeable address on the WAN interface. With IPv6, link local is often used. To reach pfSense from elsewhere, you can use any routeable address on the box. In my case, I have a /128 address on the WAN interface. In your case, you have to use the LAN address.