Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5

ttmcmurry

Reserved for future use. :)

State limits

AT&T Residential gateways have a state table that is far smaller than pfSense's defaults, which can result in problems once the RG begins tracking more states than available. pfSense should be set to never go above that limit. pfSense will adjust how states are managed based on its default adaptive algorithm from "Firewall Adaptive Timeouts." There is no need to adjust pfSense default Adaptive Timeout behavior, only the maximum number of states pfSesnse can use.

The values below are from known hardware & firmware capabilities. Depending on the # of devices directly plugged into the RG, like U-Verse set-top-boxes and devices NOT behind pfSense, you may need to adjust pfSense's maximum states downward. This information can be found on the RG under Settings -> Diagnostics -> NAT.

Pace 5268AC Firmware v11.5.1.532678-att - 15460 states max - Set pfSense to 15000 states
Arris NVG599 - Firmware v9.2.2h0d79 - 4096 states max - Set pfSense to 3500 states
Arris BGW210-700 - Firmware 1.9.16 - 8000 states max - Set pfSense to 7500 states
Motorola NVG589 - Firmware ? - 8192 states max - Set pfSense to 7600 states

Set the pfSense state limit in Advanced -> Firewall & NAT -> Firewall Maximum States

Note: If anyone has more up-to-date information about RG firmware and state capabilities, let me know and I'll update this table.

JKnott

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

is assigned a /60 and can only hand out eight /64 prefix delegations

Shouldn't that be 16 /64s?

ttmcmurry

@JKnott - No. The RG reserves 8 /64s for itself. Only 8 are able to be claimed by a router.

JKnott

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

No. The RG reserves 8 /64s for itself.

WOW!!! That device must have a lot of stuff to require 8 /64s. I wonder why they don't use Unique Local Addresses. Then they'd have an entire /7 to play with, without taking anything from the customer space. ULAs, like RFC 1918 addresses, on IPv4, are routeable within a private network.

Also, there's no reason for them to be so stingy with addresses. There are enough public addresses to give every single person on earth over 4000 /48s.

ttmcmurry

Can't say I fully understand AT&T's design choices. The reality is the RG is a multipurpose device.

Integrated DSL/ADSL modem
Runs at full line speed with GigaFiber service
Voice gateway for POTS service over IP
Native inside and isolated guest networks over WiFi on 2 frequencies (up to 4 SSIDs)
U-Verse TV service requires the RG
- If the U-Verse STB is not directly connected to the RG, it doesn't work right, if at all
- Said another way, U-Verse STBs must use the RG's DNS service
MoCA adapter is built-in on some models (inside network only)
Communicates in real time to AT&T so the AT&T U-Verse app can manage the RG
It totally logs where you go on the interwebs .. this is completely true

The /60 is enough for the device to fulfill its own purposes; it doesn't use all 8 /64 PDs reserved for itself yet it leaves 8 leftover PDs for power users like us. While I agree with you in principle, the average user is oblivious to these things and it just needs to work. Within that structure the RG must work a specific way for their support staff to keep costs down.

I plan on using the reserved space (1st post) for pfSense configuration tips. There are some quirks that should be accounted for in pfSense so undesired operation doesn't occur.

jkfritcher

Thanks for writing up these instructions, I've been wondering how to do this for a long time.

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

Step five: Edit the LAN/OPT interface(s), one at a time

Set the IPv6 Configuration Type to "Track Interface"

Set the Track IPv6 Interface -> IPv6 Interface to the WAN's interface name ("WAN" is the default name)

Leave the IPv6 Prefix ID at default "0" (zero)

Click on Save and Apply the changes

This step is being problematic for me.

I have two VLANs on the same physical interface that I want to assign PDs to. I can see through Status -> Interfaces that each VLAN is getting a different PD assigned to it like I configured in the custom config. I was also able to set the first VLAN's IPv6 configuration type to 'Track interface' and set it to WAN and Prefix ID 0. But when I try to configure the second VLAN to 'Track interface', with WAN and Prefix ID 0, I get the following error.

This track6 prefix ID is already being used in LAN

How were you able to successfully configure multiple interfaces with the same settings for the instructions above? Below is my custom config for dhcp6c. I did intentionally leave off the ia-na request, as on my WAN interface, I let the system auto config the address from RA.

Thanks!

interface igb0 {
	send ia-pd 0;
	send ia-pd 1;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc pd 0 {
	prefix-interface igb1.10 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb1.20 {
		sla-id 0;
		sla-len 0;
	};
};

ttmcmurry

Good observation. I have been meaning to build out my second internal interface for my work-at-home network so the company I work for can't sniff my home network. Could you try setting your 2nd interface to 1 and verify it is working proper? The configuration file on the WAN would be responsible for assigning the PDs. I can update the article once there's confirmation.

Glad you found this helpful. :)

jkfritcher

@ttmcmurry,

I've tried using 1 for the Prefix ID and I get an out of range error when saving.

I was looking through the code to see what its doing, but to be honest, PHP isn't even close to one of my favorite languages, so didn't make it far into figuring out what was going on.

SimpsomRJ

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";

Using this script I could not set it up properly. Dhcp6c was restarting all the time and IPs were not holding. I had to go back to the previous script (/var/etc/dhcp6c_wan_script.sh) to make it work.

@jkfritcher said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

@ttmcmurry,

I've tried using 1 for the Prefix ID and I get an out of range error when saving.

I was looking through the code to see what its doing, but to be honest, PHP isn't even close to one of my favorite languages, so didn't make it far into figuring out what was going on.

I was able to set it up properly. To be able to change the prefix ID you need to change the DHCPv6 Prefix Delegation size to 60 BEFORE you check the Configuration Override checkbox.

SimpsomRJ

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

Step four: Edit the WAN interface

Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
Under DHCP6 client configuration, select Configuration Override
Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
Click on Save and Apply the changes

This step should actually be:

Step four: Edit the WAN interface

Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
Under DHCP6 client configuration, change DHCPv6 Prefix Delegation size from 64 to 60
Select Configuration Override
Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
Click on Save and Apply the changes

My final conf file with 2 networks looks like this:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	send ia-pd 4;
	send ia-pd 5;
        send ia-pd 6;
	send ia-pd 7;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface vtnet0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface vtnet1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

ttmcmurry

Hello @SimpsomRJ - I reviewed what you found and how things are working on mine. I've made some changes, but it was minor. This also addresses what @jkfritcher said.

SimpsomRJ is correct, on the WAN interface, before selecting configuration override, the PD size needs to be set to 60. The irony is the option to do so is hidden once configuration override is selected. Nonetheless, pfSense accepts the value. Once this is set, each lan-side interface needs to be set to IPv6 type Track Interface. The IA-PD as specified in the .conf file should be assigned to the interface's Track IPv6 Interface statement. I.e. LAN -> IPv6 Configuration Type -> Track Interface; IPv6 Prefix ID -> 0, 1, 2, etc.

I'll update the main article. I have this working with all other configuration unchanged. Thank you for your insight!

stevebion

Hello
I have two WAN interfaces: WAN1 (Comcast), WAN2 (ATT). I get ipv6 addresses on the LAN side by tracking WAN1.
WAN2 gets ipv6 however "IPv6 delegated Prefix Subnet" is empty in ATT gateway Arris BGW210 , regardless of what I do.

I have tried the tutorial above but no luck. Any suggestions hugely appreciated
Config :

interface mvneta0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	send ia-pd 4;
	send ia-pd 5;
        send ia-pd 6;
	send ia-pd 7;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface mvneta1 {
		sla-id 0;
		sla-len 0;
	};
};

id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

ttmcmurry

@stevebion - I would pick one interface or the other for IPv6 unless you have static IPv6 addresses from both providers. The problem you'll run into is since both IPv6 networks are dynamically assigned, you won't be able to use NPT to translate between the interfaces if one or the other goes down. My article wasn't written to account for dynamically assigned IPv6 translation failover.

If you had static networks, then this article doesn't begin to address how to properly set that up. :)

stevebion

Sigh..Comcast Ipv6 will have to do.
ATT ipv6 was actually working after getting the line. Was able to do "Gateway Groups" and failover seemed to behave normally.
Currently, "IPv6 delegated Prefix Subnet" is empty, which makes me think, ATT not delegating Ipv6 at all, but that's an ATT issue, not pfsense :/

ttmcmurry

Seems netgate changed forum rules and it's no possible to edit a post if it's older than an hour...

There's an error in the DHCPv6 config. It needs to be Managed, not assisted. If it is set to assisted, one of two things happens - AT&T gateway may reply with an IPv6 address outside of your defined scope (SLAAC), or any device after the first will not get an IPv6 address, or both.

I believe I may need to look into a firewall rule to block slaac on the WAN interface.. but need more time to research and test.

JKnott

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

I would pick one interface or the other for IPv6 unless you have static IPv6 addresses from both providers. The problem you'll run into is since both IPv6 networks are dynamically assigned, you won't be able to use NPT to translate between the interfaces if one or the other goes down.

Actually, IPv6 is designed to support multiple connections and both prefixes will appear on the clients. I have done that with ULA and GUA addresses, but not 2 GUA. If one fails, then the other is still there. However, you'd need some mechanism, perhaps the metric, to favour one over the other when both are up. IPv6 also supports a priority, but I haven't done anything with that.

ttmcmurry

Overall, the end-user experience is what I would rely upon as the measurement for success. Keep in mind this article's context is a home environment where AT&T is the primary ISP and likely the only ISP.

If there's a working example of how to handle 2 GUAs which are dynamically allocated from both ISPs, on pfSense - and the failover mechanism is no worse than / as reliable as IPv4 gateway groups - then I'd love to see how that works.

In the IPv4 world, there are a few good reasons why multiple gateway IPs with assigned metrics are ultimately an unused feature in virtually all corporate & home networks. pfSense doesn't even directly support this in its DHCP server - though it could be manually overridden with Options. My guess is similar concerns would exist for IPv6. In the end, it always boils down to user experience. Still, I'd enjoy seeing a working theory in action. :)

JKnott

@ttmcmurry

There's a router priority setting on the Router Advertisement page and the help says:

If multiple IPv6 routers exist on the same network segment, they can indicate to clients in which order they should be used. If a high priority router becomes unavailable, clients will try a normal priority router, and finally a low priority router. Select either Low, Normal, or High from the list. If there is only one router on the network, use Normal.

So, you'd use 2 or 3 routers, configured for different ISPs. You'd normally use the highest priority one. However, if you are getting different address ranges, then you will have different addresses used when one fails. This isn't an issue for outgoing connections but would mess up anything for which there's a DNS entry. I use the ULA for that reason. My local DNS points to the ULA address.

A more appropriate would be in a business setting, where you "own" your address block, instead of being assigned one by an ISP. In this case, you'd have your own autonomous system and carriers, not ISPs, would route to your address, as advertised by a routing protocol such as OSPF.
In this case, you'd have 2 connections to 1 or more carriers and you'd select one to be your main connection. Of course, home and small business users are unlikely to do this.

There are several improvements in IPv6 v IPv4, beyond addresses. The concept of multiple addresses on a network is one of them. As I mentioned, I have both ULA and GUA prefixes on mine.

ttmcmurry

If that design were implemented in a LAN, it would potentially bypass pfSense. The firewall needs to be the only routing interface in a subnet with end-user devices to guarantee security as configured. Theoretically, sure, I get it, doesn't seem practical in the real world.

Indulging the idea - I'd attempt multi-metric routing on one or more routers ahead of pfSense's WAN interface to set proper expectations of pfSense's routing behavior, and to simplify pfSense's configuration while maintaining security integrity on the LAN nets.

Diving deeper in the rabbit hole - if pfSense were in transparent mode and was between each layer-1 connection, then sure, I can dig this. But that's way off this article's main topic.

I wholly agree with you on static addressing, that's the most ideal scenario and perfect for failover configurations, not to mention ease of configuration & maintenance over time. AT&T still doesn't allow static IPv6 address allocation for residential customers.

chriv

Hi!

I'm not new to pfSense, but out of practice.

I'm struggling trying to get IPv6 working correctly with pfSense on AT&T Fiber with a BGW200-710. While all I really need is Stateless config with working IPv6 addresses through RAs on both the WAN interface and all devices on my LAN, I do have pfSense running on a device with four Ethernet ports and I recognize that using prefix delegation for each port (LAN, OPT1, OPT2) is good design.

I'd really like to throw the BGW200-710 RG in the trash (I only have gigabit Internet; no other services from AT&T), but I've been unable so far to make my devices work with a direct connection to the ONT, so I'm stuck with it (for now).

My WAN adapter correctly gets an IPv6 address. My LAN adapter does not. Devices on my LAN do not. I decided to try your solution to see if that improved things. It did not (for me). I have nothing on the OPT1 or OPT2 ports because I haven't decided how I'm going to use them yet.

Here is the version of your script I customized:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb2 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface igb3 {
		sla-id 0;
		sla-len 0;
	};
};

I think I followed your instructions to the letter. I was a little confused with why I didn't have to set Prefix Delegation options on the DHCPv6 Server used to assist the RAs, but I didn't mess with those settings as your instructions did not include that.

I don't know how to troubleshoot this because I don't know where the IPv6 and DHCPv6 debugging is saved/logged.

Any help is appreciated.

EDIT: Never Mind. Your script worked perfectly once I fixed a configuration error on my side (I sloppily applied it to the DHCP IPv4 configuration instead of the DHCPv6 client). Once I fixed the issue, both my LAN interface and my LAN devices correctly configured IPv6 addresses in the properly delegated subnet.

Thanks!