Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5

[ttmcmurry]

I've been working on this one for a while. This is the result of others posting their work across various forums, reading BSD docs, and plenty of testing as a result of needing something to do while being stuck at home. :)

The purpose of this is to make it easier for AT&T customers who wish to assign more than one IPv6 prefix delegation inside their pfSense firewall to more than one internal network interface. I am providing an example dhcp.conf script and explaining what's needed step-by-step. AT&T customers must have been furnished a Residential Gateway (Pace 5268AC / Arris BGW210-700, possibly others) and have configured the RG in DMZ+/IP Passthrough mode. This has been written with pfSense 2.4.5 in mind.

Why do this? In short, AT&T U-Verse & Fiber customer equipment is assigned a /60 and can only hand out eight /64 prefix delegations. It is not possible to request a larger PD, however it is possible to request multiple /64 PDs from pfSense's WAN interface. Since the pfSense UI does not expose this functionality directly, it is possible to take advantage of it by supplying a dhcp.conf to override pfSense DHCP6 behavior available from the UI.

I welcome improvements and feedback and will be happy to update this to help make other's lives easier and work within pfSense native functionality as much as possible.

Once this script is in place, if you need to reassign interfaces & prefix delegations, the script has to be updated. You will need to edit the IPv6 Track Interface Prefix ID on the LAN/OPT interfaces with the IA-PD you specify in the .conf file.

Assumptions:

• The WAN interface IPv6 Configuration type is configured for "none" or,
• The WAN interface IPv6 Configuration type is configured for DHCP6 and IPv6 Prefix Delegation is set /60
• The WAN interface IPv6 DHCP6 Client Option "Do not allow PD/Address release" is UNCHECKED
• The LAN/OPT interfaces' DHCP6 option is set to "none"
• DHCPv6 Server & RA -> DHCP6 Server -> Disabled
• DHCPv6 Server & RA -> Router Advertisements -> Defaults (Router Mode: Assisted)

Step one: Update the "interface" stanza

• Make a local copy of the code below
• Look at Interfaces -> Assignments -> Network Port for the adapter associated to the WAN interface
• Replace the adapter in the interface stanza below with the WAN adapter network port name below, e.g. hn0, igb0, vmx0, eth0, etc
• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10
• IA-NA Note: The IA-NA is an arbitrary number. A unique number must be chosen for each device connected to the AT&T residential gateway (RG) which will request a prefix delegation from the RG. If only one device will be requesting PDs from the RG (i.e. this pfSense firewall), then "ia-na 0" is fine.

Step two: Update the "ia-pd" stanzas

• Look at Interfaces -> Assignments -> Network Port for the adapter associated to each LAN/OPT interface(s)
• Replace the prefix-interface "hn1" with LAN/OPT adapter network port name
• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10
• You can arbitrarily assign adapters to PDs, staring with PD 0 and working down the list
  • SLA-ID is always zero (0)
  • SLA-LEN is always zero (0)
• Formatting is specific. Each new PD stanza needs to be formatted exactly as PD0; only update the adapter name
• If you are not assigning an adapter to a PD, leave it blank as shown below
• Note: You do not need to request all 8 PDs if you don't need them. You may remove any "send ia-pd" & "id-assoc pd x" statements where you aren't assigning them to interfaces.
• Note: Assigned PDs will result in numerically different networks, depending on the RG
  • Pace 5268AC first assigns F then decrements to 8 to PD 0-7, i.e. PD0 = ::xxxF::/64
  • Arris BGW210-700 first assigns 8 then increments to F to PD 0-7, i.e. PD0 = ::xxx8::/64

Step three: Add the script to pfSense

• Create this file on pfSense under Diagnostics -> Edit File
• Copy and paste your edited script into the text window
• In the grey filename box, enter /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save

Step four: Edit the WAN interface

• Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
• Under DHCP6 client configuration, select Configuration Override
• Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save and Apply the changes

Step five: Edit the LAN/OPT interface(s), one at a time

• Set the IPv6 Configuration Type to "Track Interface"
• Set the Track IPv6 Interface -> IPv6 Interface to the WAN's interface name ("WAN" is the default name)
• Set the IPv6 Prefix ID to the PD number configured in the .conf file
• Click on Save and Apply the changes

Step six: Enable pfSense DHCPv6 Server & Test

• For each configured interface..
• DHCPv6 Server & RA -> DHCPv6 Server -> Enable -> Save
• DHCPv6 Server & RA -> Router Advertisements -> Router Mode -> Assisted (Default)
• Test a client in each configured, connected network

---ONLY COPY/PASTE THE CODE YOU EDIT BELOW HERE INTO PFSENSE---

interface hn0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	send ia-pd 4;
	send ia-pd 5;
        send ia-pd 6;
	send ia-pd 7;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface hn1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 { };
id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

[ttmcmurry]

Reserved for future use. :)

State limits

AT&T Residential gateways have a state table that is far smaller than pfSense's defaults, which can result in problems once the RG begins tracking more states than available. pfSense should be set to never go above that limit. pfSense will adjust how states are managed based on its default adaptive algorithm from "Firewall Adaptive Timeouts." There is no need to adjust pfSense default Adaptive Timeout behavior, only the maximum number of states pfSesnse can use.

The values below are from known hardware & firmware capabilities. Depending on the # of devices directly plugged into the RG, like U-Verse set-top-boxes and devices NOT behind pfSense, you may need to adjust pfSense's maximum states downward. This information can be found on the RG under Settings -> Diagnostics -> NAT.

Pace 5268AC Firmware v11.5.1.532678-att - 15460 states max - Set pfSense to 15000 states
Arris NVG599 - Firmware v9.2.2h0d79 - 4096 states max - Set pfSense to 3500 states
Arris BGW210-700 - Firmware 1.9.16 - 8000 states max - Set pfSense to 7500 states
Motorola NVG589 - Firmware ? - 8192 states max - Set pfSense to 7600 states

Set the pfSense state limit in Advanced -> Firewall & NAT -> Firewall Maximum States

Note: If anyone has more up-to-date information about RG firmware and state capabilities, let me know and I'll update this table.

[JKnott]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

is assigned a /60 and can only hand out eight /64 prefix delegations

Shouldn't that be 16 /64s?

[ttmcmurry]

@JKnott - No. The RG reserves 8 /64s for itself. Only 8 are able to be claimed by a router.

[JKnott]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

No. The RG reserves 8 /64s for itself.

WOW!!! That device must have a lot of stuff to require 8 /64s. I wonder why they don't use Unique Local Addresses. Then they'd have an entire /7 to play with, without taking anything from the customer space. ULAs, like RFC 1918 addresses, on IPv4, are routeable within a private network.

Also, there's no reason for them to be so stingy with addresses. There are enough public addresses to give every single person on earth over 4000 /48s.

[ttmcmurry]

Can't say I fully understand AT&T's design choices. The reality is the RG is a multipurpose device.

• Integrated DSL/ADSL modem
• Runs at full line speed with GigaFiber service
• Voice gateway for POTS service over IP
• Native inside and isolated guest networks over WiFi on 2 frequencies (up to 4 SSIDs)
• U-Verse TV service requires the RG
  • If the U-Verse STB is not directly connected to the RG, it doesn't work right, if at all
  • Said another way, U-Verse STBs must use the RG's DNS service
• MoCA adapter is built-in on some models (inside network only)
• Communicates in real time to AT&T so the AT&T U-Verse app can manage the RG
• It totally logs where you go on the interwebs .. this is completely true

The /60 is enough for the device to fulfill its own purposes; it doesn't use all 8 /64 PDs reserved for itself yet it leaves 8 leftover PDs for power users like us. While I agree with you in principle, the average user is oblivious to these things and it just needs to work. Within that structure the RG must work a specific way for their support staff to keep costs down.

I plan on using the reserved space (1st post) for pfSense configuration tips. There are some quirks that should be accounted for in pfSense so undesired operation doesn't occur.

[jkfritcher]

Thanks for writing up these instructions, I've been wondering how to do this for a long time.

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

Step five: Edit the LAN/OPT interface(s), one at a time

• Set the IPv6 Configuration Type to "Track Interface"
• Set the Track IPv6 Interface -> IPv6 Interface to the WAN's interface name ("WAN" is the default name)
• Leave the IPv6 Prefix ID at default "0" (zero)
• Click on Save and Apply the changes

This step is being problematic for me.

I have two VLANs on the same physical interface that I want to assign PDs to. I can see through Status -> Interfaces that each VLAN is getting a different PD assigned to it like I configured in the custom config. I was also able to set the first VLAN's IPv6 configuration type to 'Track interface' and set it to WAN and Prefix ID 0. But when I try to configure the second VLAN to 'Track interface', with WAN and Prefix ID 0, I get the following error.

This track6 prefix ID is already being used in LAN

How were you able to successfully configure multiple interfaces with the same settings for the instructions above? Below is my custom config for dhcp6c. I did intentionally leave off the ia-na request, as on my WAN interface, I let the system auto config the address from RA.

Thanks!

interface igb0 {
	send ia-pd 0;
	send ia-pd 1;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc pd 0 {
	prefix-interface igb1.10 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb1.20 {
		sla-id 0;
		sla-len 0;
	};
};

[ttmcmurry]

Good observation. I have been meaning to build out my second internal interface for my work-at-home network so the company I work for can't sniff my home network. Could you try setting your 2nd interface to 1 and verify it is working proper? The configuration file on the WAN would be responsible for assigning the PDs. I can update the article once there's confirmation.

Glad you found this helpful. :)

[jkfritcher]

@ttmcmurry,

I've tried using 1 for the Prefix ID and I get an out of range error when saving.

I was looking through the code to see what its doing, but to be honest, PHP isn't even close to one of my favorite languages, so didn't make it far into figuring out what was going on.

[SimpsomRJ]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";

Using this script I could not set it up properly. Dhcp6c was restarting all the time and IPs were not holding. I had to go back to the previous script (/var/etc/dhcp6c_wan_script.sh) to make it work.

@jkfritcher said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

@ttmcmurry,

I've tried using 1 for the Prefix ID and I get an out of range error when saving.

I was looking through the code to see what its doing, but to be honest, PHP isn't even close to one of my favorite languages, so didn't make it far into figuring out what was going on.

I was able to set it up properly. To be able to change the prefix ID you need to change the DHCPv6 Prefix Delegation size to 60 BEFORE you check the Configuration Override checkbox.

[SimpsomRJ]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

Step four: Edit the WAN interface

Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
Under DHCP6 client configuration, select Configuration Override
Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
Click on Save and Apply the changes

This step should actually be:

Step four: Edit the WAN interface

• Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
• Under DHCP6 client configuration, change DHCPv6 Prefix Delegation size from 64 to 60
• Select Configuration Override
• Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save and Apply the changes

My final conf file with 2 networks looks like this:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	send ia-pd 4;
	send ia-pd 5;
        send ia-pd 6;
	send ia-pd 7;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface vtnet0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface vtnet1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

[ttmcmurry]

Hello @SimpsomRJ - I reviewed what you found and how things are working on mine. I've made some changes, but it was minor. This also addresses what @jkfritcher said.

SimpsomRJ is correct, on the WAN interface, before selecting configuration override, the PD size needs to be set to 60. The irony is the option to do so is hidden once configuration override is selected. Nonetheless, pfSense accepts the value. Once this is set, each lan-side interface needs to be set to IPv6 type Track Interface. The IA-PD as specified in the .conf file should be assigned to the interface's Track IPv6 Interface statement. I.e. LAN -> IPv6 Configuration Type -> Track Interface; IPv6 Prefix ID -> 0, 1, 2, etc.

I'll update the main article. I have this working with all other configuration unchanged. Thank you for your insight!

[stevebion]

Hello
I have two WAN interfaces: WAN1 (Comcast), WAN2 (ATT). I get ipv6 addresses on the LAN side by tracking WAN1.
WAN2 gets ipv6 however "IPv6 delegated Prefix Subnet" is empty in ATT gateway Arris BGW210 , regardless of what I do.

I have tried the tutorial above but no luck. Any suggestions hugely appreciated
Config :

interface mvneta0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	send ia-pd 4;
	send ia-pd 5;
        send ia-pd 6;
	send ia-pd 7;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface mvneta1 {
		sla-id 0;
		sla-len 0;
	};
};

id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

[ttmcmurry]

@stevebion - I would pick one interface or the other for IPv6 unless you have static IPv6 addresses from both providers. The problem you'll run into is since both IPv6 networks are dynamically assigned, you won't be able to use NPT to translate between the interfaces if one or the other goes down. My article wasn't written to account for dynamically assigned IPv6 translation failover.

If you had static networks, then this article doesn't begin to address how to properly set that up. :)

[stevebion]

Sigh..Comcast Ipv6 will have to do.
ATT ipv6 was actually working after getting the line. Was able to do "Gateway Groups" and failover seemed to behave normally.
Currently, "IPv6 delegated Prefix Subnet" is empty, which makes me think, ATT not delegating Ipv6 at all, but that's an ATT issue, not pfsense :/

[ttmcmurry]

Seems netgate changed forum rules and it's no possible to edit a post if it's older than an hour...

There's an error in the DHCPv6 config. It needs to be Managed, not assisted. If it is set to assisted, one of two things happens - AT&T gateway may reply with an IPv6 address outside of your defined scope (SLAAC), or any device after the first will not get an IPv6 address, or both.

I believe I may need to look into a firewall rule to block slaac on the WAN interface.. but need more time to research and test.

[JKnott]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

I would pick one interface or the other for IPv6 unless you have static IPv6 addresses from both providers. The problem you'll run into is since both IPv6 networks are dynamically assigned, you won't be able to use NPT to translate between the interfaces if one or the other goes down.

Actually, IPv6 is designed to support multiple connections and both prefixes will appear on the clients. I have done that with ULA and GUA addresses, but not 2 GUA. If one fails, then the other is still there. However, you'd need some mechanism, perhaps the metric, to favour one over the other when both are up. IPv6 also supports a priority, but I haven't done anything with that.

[ttmcmurry]

Overall, the end-user experience is what I would rely upon as the measurement for success. Keep in mind this article's context is a home environment where AT&T is the primary ISP and likely the only ISP.

If there's a working example of how to handle 2 GUAs which are dynamically allocated from both ISPs, on pfSense - and the failover mechanism is no worse than / as reliable as IPv4 gateway groups - then I'd love to see how that works.

In the IPv4 world, there are a few good reasons why multiple gateway IPs with assigned metrics are ultimately an unused feature in virtually all corporate & home networks. pfSense doesn't even directly support this in its DHCP server - though it could be manually overridden with Options. My guess is similar concerns would exist for IPv6. In the end, it always boils down to user experience. Still, I'd enjoy seeing a working theory in action. :)

[JKnott]

@ttmcmurry

There's a router priority setting on the Router Advertisement page and the help says:

If multiple IPv6 routers exist on the same network segment, they can indicate to clients in which order they should be used. If a high priority router becomes unavailable, clients will try a normal priority router, and finally a low priority router. Select either Low, Normal, or High from the list. If there is only one router on the network, use Normal.

So, you'd use 2 or 3 routers, configured for different ISPs. You'd normally use the highest priority one. However, if you are getting different address ranges, then you will have different addresses used when one fails. This isn't an issue for outgoing connections but would mess up anything for which there's a DNS entry. I use the ULA for that reason. My local DNS points to the ULA address.

A more appropriate would be in a business setting, where you "own" your address block, instead of being assigned one by an ISP. In this case, you'd have your own autonomous system and carriers, not ISPs, would route to your address, as advertised by a routing protocol such as OSPF.
In this case, you'd have 2 connections to 1 or more carriers and you'd select one to be your main connection. Of course, home and small business users are unlikely to do this.

There are several improvements in IPv6 v IPv4, beyond addresses. The concept of multiple addresses on a network is one of them. As I mentioned, I have both ULA and GUA prefixes on mine.

[ttmcmurry]

If that design were implemented in a LAN, it would potentially bypass pfSense. The firewall needs to be the only routing interface in a subnet with end-user devices to guarantee security as configured. Theoretically, sure, I get it, doesn't seem practical in the real world.

Indulging the idea - I'd attempt multi-metric routing on one or more routers ahead of pfSense's WAN interface to set proper expectations of pfSense's routing behavior, and to simplify pfSense's configuration while maintaining security integrity on the LAN nets.

Diving deeper in the rabbit hole - if pfSense were in transparent mode and was between each layer-1 connection, then sure, I can dig this. But that's way off this article's main topic.

I wholly agree with you on static addressing, that's the most ideal scenario and perfect for failover configurations, not to mention ease of configuration & maintenance over time. AT&T still doesn't allow static IPv6 address allocation for residential customers.

[chriv]

Hi!

I'm not new to pfSense, but out of practice.

I'm struggling trying to get IPv6 working correctly with pfSense on AT&T Fiber with a BGW200-710. While all I really need is Stateless config with working IPv6 addresses through RAs on both the WAN interface and all devices on my LAN, I do have pfSense running on a device with four Ethernet ports and I recognize that using prefix delegation for each port (LAN, OPT1, OPT2) is good design.

I'd really like to throw the BGW200-710 RG in the trash (I only have gigabit Internet; no other services from AT&T), but I've been unable so far to make my devices work with a direct connection to the ONT, so I'm stuck with it (for now).

My WAN adapter correctly gets an IPv6 address. My LAN adapter does not. Devices on my LAN do not. I decided to try your solution to see if that improved things. It did not (for me). I have nothing on the OPT1 or OPT2 ports because I haven't decided how I'm going to use them yet.

Here is the version of your script I customized:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb2 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface igb3 {
		sla-id 0;
		sla-len 0;
	};
};

I think I followed your instructions to the letter. I was a little confused with why I didn't have to set Prefix Delegation options on the DHCPv6 Server used to assist the RAs, but I didn't mess with those settings as your instructions did not include that.

I don't know how to troubleshoot this because I don't know where the IPv6 and DHCPv6 debugging is saved/logged.

Any help is appreciated.

EDIT: Never Mind. Your script worked perfectly once I fixed a configuration error on my side (I sloppily applied it to the DHCP IPv4 configuration instead of the DHCPv6 client). Once I fixed the issue, both my LAN interface and my LAN devices correctly configured IPv6 addresses in the properly delegated subnet.

Thanks!

[thomas.munn]

I seem to be getting somewhere. I have almost the exact configuration as above. I am getting the following in my logs:

Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD: ID=1, T1=1800, T2=2880
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD prefix, len 25
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD prefix: 2600:1700:1852:88de::/64 pltime=3600 vltime=3600
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD, len 41
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD: ID=2, T1=1800, T2=2880
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD prefix, len 25
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD prefix: 2600:1700:1852:88dd::/64 pltime=3600 vltime=3600
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option client ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:25:f0:a5:cc:00:25:90:b9:e6:6c
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option server ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:26:f3:80:30:e8:b2:fe:47:f8:b1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option preference, len 1
Oct 24 01:35:15 firewall dhcp6c[20706]: preference: 255
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option DNS, len 16
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option domain search list, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: XID mismatch
Oct 24 01:35:15 firewall dhcp6c[20706]: receive reply from fe80::eab2:feff:fe47:f8b1%igb1 on igb1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option client ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:25:f0:a5:cc:00:25:90:b9:e6:6c
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option server ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:26:f3:80:30:e8:b2:fe:47:f8:b1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option status code, len 19

I don't seem to have any ipv6 on any addresses(except on wan!) . I used the same as above, but with slightly different interface assignments. The system really doesn't tell me why it isn't working even in advanced debug mode.

If I go to 'defaults' I do get working ipv6 on lan only. I did notice another script dhcp6c_wan_script.sh should I try that?

[thomas.munn]

Never Mind, like the other guy, I had to use the right script. Seems to be working now. Why does it take a bit for an ipv6 address to get assigned? I see the same behaviour on some of my vm's in a different isp.

[thomas.munn]

How to make netgate work with ipv6 on multiple interfaces (redo with table)

I've been working on this one for a while. This is the result of others posting their work across various forums, reading BSD docs, and plenty of testing as a result of needing something to do while being stuck at home. :)

The purpose of this is to make it easier for AT&T customers who wish to assign more than one IPv6 prefix delegation inside their pfSense firewall to more than one internal network interface. I am providing an example dhcp.conf script and explaining what's needed step-by-step. AT&T customers must have been furnished a Residential Gateway (Pace 5268AC / Arris BGW210-700, possibly others) and have configured the RG in DMZ+/IP Passthrough mode. This has been written with pfSense 2.4.5 in mind.

Why do this? In short, AT&T U-Verse & Fiber customer equipment is assigned a /60 and can only hand out eight /64 prefix delegations. It is not possible to request a larger PD, however it is possible to request multiple /64 PDs from pfSense's WAN interface. Since the pfSense UI does not expose this functionality directly, it is possible to take advantage of it by supplying a dhcp.conf to override pfSense DHCP6 behavior available from the UI.

I welcome improvements and feedback and will be happy to update this to help make other's lives easier and work within pfSense native functionality as much as possible.

Once this script is in place, if you need to reassign interfaces & prefix delegations, the script has to be updated. You will need to edit the IPv6 Track Interface Prefix ID on the LAN/OPT interfaces with the IA-PD you specify in the .conf file.

Assumptions:

• The WAN interface IPv6 Configuration type is configured for "none" or,
• The WAN interface IPv6 Configuration type is configured for DHCP6 and IPv6 Prefix Delegation is set /60
• The WAN interface IPv6 DHCP6 Client Option "Do not allow PD/Address release" is UNCHECKED
• The LAN/OPT interfaces' DHCP6 option is set to "none"
• DHCPv6 Server & RA -> DHCP6 Server -> Disabled
• DHCPv6 Server & RA -> Router Advertisements -> Defaults (Router Mode: Assisted)

Step one: Update the "interface" stanza

• Make a local copy of the code below
• Look at Interfaces -> Assignments -> Network Port for the adapter associated to the WAN interface
• Replace the adapter in the interface stanza below with the WAN adapter network port name below, e.g. hn0, igb0, vmx0, eth0, etc
• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10
• IA-NA Note: The IA-NA is an arbitrary number. A unique number must be chosen for each device connected to the AT&T residential gateway (RG) which will request a prefix delegation from the RG. If only one device will be requesting PDs from the RG (i.e. this pfSense firewall), then "ia-na 0" is fine.

Step two: Update the "ia-pd" stanzas

• Look at Interfaces -> Assignments -> Network Port for the adapter associated to each LAN/OPT interface(s)

• Replace the prefix-interface "hn1" with LAN/OPT adapter network port name

• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10

• You can arbitrarily assign adapters to PDs, staring with PD 0 and working down the list

  • SLA-ID is always zero (0)
  • SLA-LEN is always zero (0)

• Formatting is specific. Each new PD stanza needs to be formatted exactly as PD0; only update the adapter name

• If you are not assigning an adapter to a PD, leave it blank as shown below

• Note: You do not need to request all 8 PDs if you don't need them. You may remove any "send ia-pd" & "id-assoc pd x" statements where you aren't assigning them to interfaces.

• Note

  : Assigned PDs will result in numerically different networks, depending on the RG

  • Pace 5268AC first assigns F then decrements to 8 to PD 0-7, i.e. PD0 = ::xxxF::/64
  • Arris BGW210-700 first assigns 8 then increments to F to PD 0-7, i.e. PD0 = ::xxx8::/64

Step three: Add the script to pfSense

• Create this file on pfSense under Diagnostics -> Edit File
• Copy and paste your edited script into the text window
• In the grey filename box, enter /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save

Step four: Edit the WAN interface

• Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
• Under DHCP6 client configuration, select Configuration Override
• Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save and Apply the changes

Step five: Edit the LAN/OPT interface(s), one at a time

• Set the IPv6 Configuration Type to "Track Interface"
• Set the Track IPv6 Interface -> IPv6 Interface to the WAN's interface name ("WAN" is the default name)
• Set the IPv6 Prefix ID to the PD number configured in the .conf file
• Click on Save and Apply the changes

Step six: Enable pfSense DHCPv6 Server & Test

• For each configured interface..
• DHCPv6 Server & RA -> DHCPv6 Server -> Enable -> Save
• DHCPv6 Server & RA -> Router Advertisements -> Router Mode -> Assisted (Default)
• Test a client in each configured, connected network

Step Seven: FOLLOW THE DIRECTIONS! Easy to foobar the typing. Be patient, slow, and it will eventually work. I did add a completed configuration, and a table which helps a little in doing the work.

interface igb1 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb3 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface igb2 {
		sla-id 0;
		sla-len 0;
	};
};

Interface delegations

Interface	Name	Association
igb1	WAN
igb3	LAN	0
igb0	IOTVIDEO	1
DMZ	igb2	2

I highly recommend that you backup your configuration and save. At&t can be very annoying since they don't hand out the whole /60 Oh well.

[ttmcmurry]

Glad to see that y'all got it working. :)

Regarding why does it take so long to get an address .. My best guess is more at the AT&T RG's IPv6 stack is simply slow. The RG already knows what /60 is assigned. Delegating that out should be in the tenths of a second to a second to work, end-to-end.

The reality is IPv6 is treated like a second class citizen by AT&T and if I'm being truthful, they have zero reason to prioritize developing a better, faster stack. Like chriv, I would be happy to interface with the ONT and be done with it.

One of these days I wanted to do a repost of everything for a 'newer' version of pfSense .. since the forum moderators started locking thread edits.

[thomas.munn]

@ttmcmurry I added the table just because it makes the whole thing a little simpler. It’s just crazy that AT&T does all this nonsense trying to prevent me from doing multiple segments with IPv6. Honestly I think SLAAC is a lot easier than stupid DHCPv6

[ttmcmurry]

It's important to remember SLAAC doesn't register with DNS or DHCP. This is one of the reasons why it isn't the best solution for all situations. It might be fine for home use if the apps you use don't need DNS to find other devices. SLAAC also doesn't support features like DHCP options, which are commonly used in businesses and especially with VOIP systems.

Between lack of configurability, no ability to perform DNS registration, thus making life very difficult for anyone running an Active Directory domain, SLAAC has a place, but it doesn't work great for corporate environments.

Also, be careful when using SLAAC if you're using pfSense for DNS filtering. When I had it enabled, devices were being configured by the AT&T router as the DNS source and was bypassing pfSense.

[JKnott]

@ttmcmurry

With SLAAC, you point your DNS at the consistent address, which is often based on the MAC address. On the other hand, if you use DHCPv6, you won't be able to use Android devices, as for some stupid reason it's not supported. I have never been able to determine any reason for it not to be supported, other than the developer doesn't like it. His arguments against it could also apply to SLAAC.

[ttmcmurry]

@JKnott
I suppose I have 2 observations:

1. Android devices will still work on the IPv4 stack. If Managed DHCPv6 mode is enabled, you're correct, those will not get an IPv6 address.

2. With respect to using pfSense to perform DNS filtering while using stateless autoconfig/Assisted mode/SLAAC.
   When using DHCPv6 in this manner, when pfSense's IPv6 WAN IP is dynamic, problems become apparent quickly. Since a static IPv6 WAN address is not available to configure in pfSense's DHCPv6 RA DNS Server list, what happens is the RA is passed on from the AT&T RG. Therefore in this configuration, DNS will bypass pfSense..

If you're not using pfSense to filter/route DNS toward a filtering provider, then this isn't a concern and go stateless and you're probably fine with allowing AT&T to monitor where you go on the internet.

Yes, a configuration paradox exists because Google won't support DHCPv6 on Android. The addressing problems are resolved when using DHCPv6 in Managed mode. However, Android devices cannot be allocated IPv6 addresses as a result. In the end the same underlying goal is achievable, albeit without direct IPv6 support.

It's about compromising & determining which compromise covers the majority of your needs.

Of course I would love to be proven wrong. The moment I set DHCPv6 for Stateless or Assisted, my subnets get addresses from the AT&T RG. If that's a configuration error on my side (firewall rule on wan interface??) I'd love to fix that and update this doc!

[lilchancep]

To anyone that comes to this thread I have made a GitHub to have all the changes that have been made throughout this thread. Also for easier reading. [all input is welcome]

Link

@ttmcmurry I hope this is okay, I made sure to give you 190% credit as this is your hard work.

[deet]

A couple of notes:

• On the WAN interface settings page, setting the prefix size to /60 has no effect on dhcp6c or what the dhcp6c client requests from the ISP. The reason for doing that is to tell the web UI what range of prefix IDs to allow on the LAN configuration pages. If you leave it at /64, it will only allow '0', and it won't let you set that on more than one interface. You're setting it to /60 to trick the web UI validator, not because you want a /60 from your ISP.
• With this in mind, although you can use arbitrary values for ia_pd in your config file, you will want to stick to 0–8 for prefix IDs so that they will validate in the UI.

[ttmcmurry]

@deet have you tested this effect?

in 2.4.5, the UI doesn't allow other interfaces to track a wan interface to use a metric higher than 0 when the wan is set to /64.

It was necessary to set both the script and the corresponding interface in the UI to use the same prefix id.

I believe there's a DHCP Service dependency on the interface in the UI to use the tracked interface prefix specified there. Said another way, I do not believe the DHCP v6 Service evaluates a dhcp.conf file if the WAN's DHCP Override contains a conf; it has to use the value specified in the interface's UI.

If this behavior has changed in "p1" - could you provide confirmation and the updated steps for others to follow?

[ttmcmurry]

@lilchancep - thank you. Ever since the netgate forum rules change, it became difficult to keep this up to date. I have been waiting for 2.5 to come out to make any significant overhaul, but would end up in the same place, unable to update it after a few hours.

I've starred the page you created in github. :)

[deet]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

@deet have you tested this effect?

I am describing my own findings, so yes.

I was initially confused about the need to set the prefix to /60 in the WAN settings before setting the override file path. "Why would it matter what the prefix length is set to, if we're just going to override it in the .conf file?" I wondered. So I tried setting the WAN prefix length to different values to see what would happen.

I found that the .conf override file is what governs the behavior of dhcp6c, not the value in the field, "overriding" indeed.

Setting the WAN prefix length to /60 does not, therefore, have any bearing on what prefix dhcp6c requests.

But the pfsense web GUI does use that value when validating the prefix IDs on the LAN pages. If you leave it at /64, the only valid prefix ID is 0. The validator looks at /64 and calculates only one available prefix ID. By setting the WAN prefix length to /60, however, the validator calculates a range of up to 16 IDs. (Of which 8 are actually available, but that's a gateway limitation, not an IPv6 thing.)

So like I say, setting it to /60 matters not because it tells dhcp6c to do anything relevant, but because without it, the pfsense web GUI validator won't let you set the values you need to set in the LAN pages.

in 2.4.5, the UI doesn't allow other interfaces to track a wan interface to use a metric higher than 0 when the wan is set to /64.

Right.

It was necessary to set both the script and the corresponding interface in the UI to use the same prefix id.

With this method, yes. But ordinarily, no; "ia_pd" in the .conf file doesn't have to be anything in particular; it is purely an index for configuration purposes. And, ordinarily, the prefix ID in the GUI doesn't have to be anything particular, either, except that it needs to be within a range valid for the delegated prefix, because ordinarily it would end up appended to the delegated prefix being tracked. These are, ordinarily, separate values unrelated to each other.

For example, if this were Comcast, and we'd been properly delegated a /60, we could put 'f' as the ID in one interface and '9' as the ID in another, and each network would get a prefix ending in 'f' and '9' respectively. Thus can this field be set to anything we want as long as it results in a valid prefix.

But because of the way we're fitting square pegs into round holes, we need the ia_pd in the .conf file to match the "prefix ID" in the GUI. Not because it will be appended to the delegated prefix as usual — because the delegated prefix we get will go straight to the tracking interface. Rather, we are using that field to tell pfSense which /64 prefix (referring to the ia_pd set in the conf file) goes to which interface (which is also indicated in the conf file).

In practice, the prefix we get will not be the usual <delegated /60 prefix> + <"prefix suffix" to form a complete /64 prefix>, but rather <whatever /64 prefix is assigned to us by the gateway>.

Apologies if I am trying to dig myself out of a hole with this explanation. It makes sense the way I'm reading it but I get that it's confusing and I am probably not being clear. It's just, after wondering why setting /60 would matter in the GUI when we were using a conf override file (and the answer is, for GUI-related reasons only), my next question was why the prefix ID set in the GUI would have any bearing, since the /64 being delegated has no more bits available for any suffixes to be appended. The answer is, the ID is not actually being appended as usual. Instead, through this method, we are setting not a "prefix suffix" but rather telling pfsense which ia_pd dhpc6c should associate with the interface.

I believe there's a DHCP Service dependency on the interface in the UI to use the tracked interface prefix specified there.

The dependency is in how the pfsense GUI works, not in the DHCP service.

Said another way, I do not believe the DHCP v6 Service evaluates a dhcp.conf file if the WAN's DHCP Override contains a conf; it has to use the value specified in the interface's UI.

Sort of. The .conf file does indeed take full control of what actually happens to all interfaces. But we have to do this GUI stuff to make sure the GUI matches what dhcp6c is doing.

If this behavior has changed in "p1" - could you provide confirmation and the updated steps for others to follow?

The steps are the same! We're doing all the right things. My comments are as much for my own future reference as for anything. "Wait why does it matter what we do in the GUI if we're overriding everything in the conf file? Oh, it's because we need the GUI to match what the conf file is doing, and we have to fudge some things in order for that to happen." That's all.

Clear as mud?

[ttmcmurry]

@deet
Clear as mud. I just installed 2.5 and will get around verifying all of this when I have some spare time. :)

At least we understand each other!

[cbennett2010]

So, I updated to 2.5.0 and attempted to implement this hoping I could get ipv6 running on my individual internal vlans. That being said, I setup the script as above and it looks something like this:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb1.100 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb1.110 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 { 
	prefix-interface igb1.200 {
		sla-id 0;
		sla-len 0;
	};
};

However, I WAN is the only interface to pull a ipv6 address, and I am seeing this in the logs:

dhcp6c[64417]: unexpected interface (1)

been having some trouble with ipv6 after updating to 2.5.0 but you're script has given me a little bit of progress. Any help would be appreciated, and if you need more details please let me know and I'll update as soon as possible.

[deet]

@cbennett2010 if you go to Status > interfaces, do the interface names all exist and match?

[cbennett2010]

@deet Yes they match. here's whats shown under status below:

[ttmcmurry]

@cbennett2010

In your example, WAN interface igb0 is calling
script "/var/etc/dhcp6c_wan_script.sh";

Have you tried using the code from the first post instead?
script "var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";

I recently moved to 2.5 and I opted to not have my WAN interface request domain-name and domain-name servers in my ipv6 script. That said, I specify these values elsewhere.

My current config is listed below. Maybe that will help. Also, dhcpd is sensitive to syntax, tabs, etc. I'll also post a screenshot from notepad++ with all characters displayed, perhaps that will help.

interface hn0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface hn1.50 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface hn1.40 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface hn2 {
		sla-id 0;
		sla-len 0;
	};
};

[cbennett2010]

@ttmcmurry Thank you for the reply while I try to figure this out. So I made the modifications to change the script that is running, and verified the Text Syntax:

but alas still seeing the same error:

Feb 28 16:41:46 dhcp6c[23870]: Sending Solicit
Feb 28 16:41:46 dhcp6c[64417]: unexpected interface (1)

I'm beginning to wonder if AT&T really only did allocate a /64 to me. Because looking at my RG device and seeing what is allocated to it, I see this:

Global Unicast IPv6 Address	****:****:****:7a90::1
Link-local IPv6 Address	fe80::8a96:4eff:fe89:6d70
IPv6 Addressing Subnet (including length)	****:****:****:7a90::/64
IPv6 Delegated Prefix Subnet (including length)

And this is the address that I see on my WAN:

****:****:****:7a90:92e2:baff:fe80:c6f8

So not sure why i'm not seeing a Delegated Prefix Subnet on my AT&T RG.

Thanks for all the help so far!