Working around AT&T's terrible native IPv6 implementation

mwp821

Hi folks,

I have pfSense routing an AT&T Fiber uplink (directly, using one of the bypass methods) and naturally IPv6 is a bit buggy. My /60 prefix delegation is working and everything is fine on the LAN side: clients get IPv6 addresses with a 2600:1700: prefix, and I get a 10/10 on test-ipv6.com.

The trouble is on the WAN side. AT&T hands out an address in the 2001:506: prefix, which is not routable outside of their network. Effectively this means that IPv6 traffic originating from the router itself gets lost (or the return traffic gets lost?) because it tries to use that address as the source address. This causes lots of weird glitches and hangs in the webConfigurator, Unbound, etc. I've been able to mostly work around it by telling pfSense to prefer IPv4 over IPv6 and configuring Unbound to issue queries via IPv6 transport from a LAN interface.

However, I'm hoping there's a better solution out there. I still want it to assign the 2001:506: address in case AT&T needs it for some reason. I'd like it to also delegate one of the /60 prefixes to the WAN interface (if that makes sense) so I can give it an address with a 2600:1700: prefix, and pfSense should prefer it over the 2001:506: address.

I tried to just assign a VIP to the WAN interface, but when the rules reloaded, I got the following notice:

Filter Reload
There were error(s) loading the rules: /tmp/rules.debug:211: syntax error - The line in question reads [211]: pass out route-to ( igb0 162.198.reda.cted ) from to !/ tracker 1000006962 keep state allow-opts label "let out anything from firewall host itself"

So I'm not quite sure what to make of that. Do I need to do something first before I can assign VIPs from the /60 PD to the WAN interface? Am I missing any other pieces here?

Thanks in advance for any clues you can offer. I'm running 2.4.3-RELEASE-p1 (amd64) on a SG-2440.

Mike

JKnott

@mwp821

First off any address starting with 2:: or 3:: is a global unique address, that is, it's routeable. Are you saying AT&T is blocking it?

johnpoz

@jknott said in Working around AT&T's terrible native IPv6 implementation:

First off any address starting with 2:: or 3:: is a global unique address, that is, it’s routeable

Not actually true ;)

Off the top of my head there is

2001:2::/48
2001:db8::/32

Both of which will not global route ;)

mwp821

@jknott said in Working around AT&T's terrible native IPv6 implementation:

First off any address starting with 2:: or 3:: is a global unique address, that is, it’s routeable. Are you saying AT&T is blocking it?

Sorry if I used the wrong term.Yeah, AT&T is blocking it. For example, an outbound traceroute6 gets about five hops into their network and just dies. And the router just isn't reachable from outside their network (and I've verified that pfSense itself isn't blocking the communication). It's a known issue.

johnpoz

@mwp821 said in Working around AT&T's terrible native IPv6 implementation:

he trouble is on the WAN side. AT&T hands out an address in the 2001:506: prefix

I don't show that being owned by them
https://whois.arin.net/rest/net/NET6-2001-506-1

Organization MCI Communications Services, Inc. d/b/a Verizon Business (MCICS)

And I don't see an ASN for it - so yeah I don't see how it could route anywhere.. So seems like to me ATT is using space that is not theirs? But if you look verizon took over MCI many many years ago, and then didn't ATT buy part of Verizon? So they prob got this address space with that. But they have not updated Arin with this info and they currently have no ASN assigned to it... So while it is in the global space and should route.. Doesn't mean it has to go anywhere outside of the network its being used in unless they assign it to an ASN and setup the routing to be global.

If your problem is with pfsense using this IP to talk outbound for itself, I would think you should be able to assign a VIP inside one of your /60 /64 prefixes on the wan and use that for pfsense to talk outbound on..

Its not just att having horrible ipv6 setups - its pretty much all of them ;) I would just use a HE tunnel for your ipv6 connectivity ;) Until such time that your ISP gets their head out of their ASS ;)

But what exactly is pfsense needing to talk IPv6 for? Why not not just have it not give ipv6 on wan, you can select that in your delegation request.. It for sure can use ipv4 to check for pfsense updates and grab packages.

mwp821

@johnpoz said in Working around AT&T's terrible native IPv6 implementation:

If your problem is with pfsense using this IP to talk outbound for itself, I would think you should be able to assign a VIP inside one of your /60 /64 prefixes on the wan and use that for pfsense to talk outbound on..

I tried that, but I got the weird Filter Reload error I showed above.

Its not just att having horrible ipv6 setups - its pretty much all of them ;) I would just use a HE tunnel for your ipv6 connectivity ;) Until such time that your ISP gets their head out of their ASS ;)

I actually had that set up before I got this working and found it to be a little laggy. I have gigabit symmetrical and it seemed like HE couldn't keep up with it. AT&T's IPv6 network is definitely faster... when it works.

But what exactly is pfsense needing to talk IPv6 for? Why not not just have it not give ipv6 on wan, you can select that in your delegation request.. It for sure can use ipv4 to check for pfsense updates and grab packages.

I have definitely considered that. I'm concerned that AT&T needs the 2001:506: address assigned on my end for some reason or another.

johnpoz

To do what exactly? I would try not having it assign and see if everything works.

JKnott

@johnpoz said in Working around AT&T's terrible native IPv6 implementation:

@jknott said in Working around AT&T's terrible native IPv6 implementation:

First off any address starting with 2:: or 3:: is a global unique address, that is, it’s routeable

Not actually true ;)

Off the top of my head there is

2001:2::/48
2001:db8::/32

Both of which will not global route ;)

Why not? Both are within the GUA range. Is someone blocking them?

gsmornot

@mwp821
I have played with this before. The error you’re seeing is something that came with the p1 release. I used a VIP on the WAN for 2600 address in the past without issue. Since p1 I get the same error.

To be fair, all of this works fine if you use the provided gateway but that’s less interesting. I go back and forth with the bypass. If I do it agin I think I will do IPv4 only. Easier.

johnpoz

@jknott said in Working around AT&T's terrible native IPv6 implementation:

Why not? Both are within the GUA range. Is someone blocking them?

Because they are special assignment prefixes.. 2001:db8::/32 is designed for documentation purpose use... Just like 192.0.2/24 in ipv4.. There are others in ipv4 as well that do not route other than rfc1918..

2001:2::/48 is for benchmarking, and again not designed to route globally. There are others that might not route, they have caveats.. Here..
https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml

mwp821

@gsmornot said in Working around AT&T's terrible native IPv6 implementation:

I have played with this before. The error you’re seeing is something that came with the p1 release. I used a VIP on the WAN for 2600 address in the past without issue. Since p1 I get the same error.

Sure enough. Looks like it will be fixed in 2.4.4. Thank you!

JKnott

@johnpoz said in Working around AT&T's terrible native IPv6 implementation:

Because they are special assignment prefixes… 2001:db8::/32 is designed for documentation purpose use… Just like 192.0.2/24 in ipv4… There are others in ipv4 as well that do not route other than rfc1918…
2001:2::/48 is for benchmarking, and again not designed to route globally. There are others that might not route, they have caveats… Here…
https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml

Oh, that sort of thing. I wonder why they didn't use a ULA for that, instead of messing things up.