Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5

[chriv]

Hi!

I'm not new to pfSense, but out of practice.

I'm struggling trying to get IPv6 working correctly with pfSense on AT&T Fiber with a BGW200-710. While all I really need is Stateless config with working IPv6 addresses through RAs on both the WAN interface and all devices on my LAN, I do have pfSense running on a device with four Ethernet ports and I recognize that using prefix delegation for each port (LAN, OPT1, OPT2) is good design.

I'd really like to throw the BGW200-710 RG in the trash (I only have gigabit Internet; no other services from AT&T), but I've been unable so far to make my devices work with a direct connection to the ONT, so I'm stuck with it (for now).

My WAN adapter correctly gets an IPv6 address. My LAN adapter does not. Devices on my LAN do not. I decided to try your solution to see if that improved things. It did not (for me). I have nothing on the OPT1 or OPT2 ports because I haven't decided how I'm going to use them yet.

Here is the version of your script I customized:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb1 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb2 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface igb3 {
		sla-id 0;
		sla-len 0;
	};
};

I think I followed your instructions to the letter. I was a little confused with why I didn't have to set Prefix Delegation options on the DHCPv6 Server used to assist the RAs, but I didn't mess with those settings as your instructions did not include that.

I don't know how to troubleshoot this because I don't know where the IPv6 and DHCPv6 debugging is saved/logged.

Any help is appreciated.

EDIT: Never Mind. Your script worked perfectly once I fixed a configuration error on my side (I sloppily applied it to the DHCP IPv4 configuration instead of the DHCPv6 client). Once I fixed the issue, both my LAN interface and my LAN devices correctly configured IPv6 addresses in the properly delegated subnet.

Thanks!

[thomas.munn]

I seem to be getting somewhere. I have almost the exact configuration as above. I am getting the following in my logs:

Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD: ID=1, T1=1800, T2=2880
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD prefix, len 25
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD prefix: 2600:1700:1852:88de::/64 pltime=3600 vltime=3600
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD, len 41
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD: ID=2, T1=1800, T2=2880
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option IA_PD prefix, len 25
Oct 24 01:35:15 firewall dhcp6c[20706]: IA_PD prefix: 2600:1700:1852:88dd::/64 pltime=3600 vltime=3600
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option client ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:25:f0:a5:cc:00:25:90:b9:e6:6c
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option server ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:26:f3:80:30:e8:b2:fe:47:f8:b1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option preference, len 1
Oct 24 01:35:15 firewall dhcp6c[20706]: preference: 255
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option DNS, len 16
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option domain search list, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: XID mismatch
Oct 24 01:35:15 firewall dhcp6c[20706]: receive reply from fe80::eab2:feff:fe47:f8b1%igb1 on igb1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option client ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:25:f0:a5:cc:00:25:90:b9:e6:6c
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option server ID, len 14
Oct 24 01:35:15 firewall dhcp6c[20706]: DUID: 00:01:00:01:26:f3:80:30:e8:b2:fe:47:f8:b1
Oct 24 01:35:15 firewall dhcp6c[20706]: get DHCP option status code, len 19

I don't seem to have any ipv6 on any addresses(except on wan!) . I used the same as above, but with slightly different interface assignments. The system really doesn't tell me why it isn't working even in advanced debug mode.

If I go to 'defaults' I do get working ipv6 on lan only. I did notice another script dhcp6c_wan_script.sh should I try that?

[thomas.munn]

Never Mind, like the other guy, I had to use the right script. Seems to be working now. Why does it take a bit for an ipv6 address to get assigned? I see the same behaviour on some of my vm's in a different isp.

[thomas.munn]

How to make netgate work with ipv6 on multiple interfaces (redo with table)

I've been working on this one for a while. This is the result of others posting their work across various forums, reading BSD docs, and plenty of testing as a result of needing something to do while being stuck at home. :)

The purpose of this is to make it easier for AT&T customers who wish to assign more than one IPv6 prefix delegation inside their pfSense firewall to more than one internal network interface. I am providing an example dhcp.conf script and explaining what's needed step-by-step. AT&T customers must have been furnished a Residential Gateway (Pace 5268AC / Arris BGW210-700, possibly others) and have configured the RG in DMZ+/IP Passthrough mode. This has been written with pfSense 2.4.5 in mind.

Why do this? In short, AT&T U-Verse & Fiber customer equipment is assigned a /60 and can only hand out eight /64 prefix delegations. It is not possible to request a larger PD, however it is possible to request multiple /64 PDs from pfSense's WAN interface. Since the pfSense UI does not expose this functionality directly, it is possible to take advantage of it by supplying a dhcp.conf to override pfSense DHCP6 behavior available from the UI.

I welcome improvements and feedback and will be happy to update this to help make other's lives easier and work within pfSense native functionality as much as possible.

Once this script is in place, if you need to reassign interfaces & prefix delegations, the script has to be updated. You will need to edit the IPv6 Track Interface Prefix ID on the LAN/OPT interfaces with the IA-PD you specify in the .conf file.

Assumptions:

• The WAN interface IPv6 Configuration type is configured for "none" or,
• The WAN interface IPv6 Configuration type is configured for DHCP6 and IPv6 Prefix Delegation is set /60
• The WAN interface IPv6 DHCP6 Client Option "Do not allow PD/Address release" is UNCHECKED
• The LAN/OPT interfaces' DHCP6 option is set to "none"
• DHCPv6 Server & RA -> DHCP6 Server -> Disabled
• DHCPv6 Server & RA -> Router Advertisements -> Defaults (Router Mode: Assisted)

Step one: Update the "interface" stanza

• Make a local copy of the code below
• Look at Interfaces -> Assignments -> Network Port for the adapter associated to the WAN interface
• Replace the adapter in the interface stanza below with the WAN adapter network port name below, e.g. hn0, igb0, vmx0, eth0, etc
• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10
• IA-NA Note: The IA-NA is an arbitrary number. A unique number must be chosen for each device connected to the AT&T residential gateway (RG) which will request a prefix delegation from the RG. If only one device will be requesting PDs from the RG (i.e. this pfSense firewall), then "ia-na 0" is fine.

Step two: Update the "ia-pd" stanzas

• Look at Interfaces -> Assignments -> Network Port for the adapter associated to each LAN/OPT interface(s)

• Replace the prefix-interface "hn1" with LAN/OPT adapter network port name

• If using VLANs, remember to use numerical subinterface number e.g. hn0.10 for VLAN 10

• You can arbitrarily assign adapters to PDs, staring with PD 0 and working down the list

  • SLA-ID is always zero (0)
  • SLA-LEN is always zero (0)

• Formatting is specific. Each new PD stanza needs to be formatted exactly as PD0; only update the adapter name

• If you are not assigning an adapter to a PD, leave it blank as shown below

• Note: You do not need to request all 8 PDs if you don't need them. You may remove any "send ia-pd" & "id-assoc pd x" statements where you aren't assigning them to interfaces.

• Note

  : Assigned PDs will result in numerically different networks, depending on the RG

  • Pace 5268AC first assigns F then decrements to 8 to PD 0-7, i.e. PD0 = ::xxxF::/64
  • Arris BGW210-700 first assigns 8 then increments to F to PD 0-7, i.e. PD0 = ::xxx8::/64

Step three: Add the script to pfSense

• Create this file on pfSense under Diagnostics -> Edit File
• Copy and paste your edited script into the text window
• In the grey filename box, enter /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save

Step four: Edit the WAN interface

• Set IPv6 Configuration Type to "DHCP6" (it may already be set, see "assumptions" above)
• Under DHCP6 client configuration, select Configuration Override
• Enter the following in Configuration File Override: /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf
• Click on Save and Apply the changes

Step five: Edit the LAN/OPT interface(s), one at a time

• Set the IPv6 Configuration Type to "Track Interface"
• Set the Track IPv6 Interface -> IPv6 Interface to the WAN's interface name ("WAN" is the default name)
• Set the IPv6 Prefix ID to the PD number configured in the .conf file
• Click on Save and Apply the changes

Step six: Enable pfSense DHCPv6 Server & Test

• For each configured interface..
• DHCPv6 Server & RA -> DHCPv6 Server -> Enable -> Save
• DHCPv6 Server & RA -> Router Advertisements -> Router Mode -> Assisted (Default)
• Test a client in each configured, connected network

Step Seven: FOLLOW THE DIRECTIONS! Easy to foobar the typing. Be patient, slow, and it will eventually work. I did add a completed configuration, and a table which helps a little in doing the work.

interface igb1 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb3 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface igb2 {
		sla-id 0;
		sla-len 0;
	};
};

Interface delegations

Interface	Name	Association
igb1	WAN
igb3	LAN	0
igb0	IOTVIDEO	1
DMZ	igb2	2

I highly recommend that you backup your configuration and save. At&t can be very annoying since they don't hand out the whole /60 Oh well.

[ttmcmurry]

Glad to see that y'all got it working. :)

Regarding why does it take so long to get an address .. My best guess is more at the AT&T RG's IPv6 stack is simply slow. The RG already knows what /60 is assigned. Delegating that out should be in the tenths of a second to a second to work, end-to-end.

The reality is IPv6 is treated like a second class citizen by AT&T and if I'm being truthful, they have zero reason to prioritize developing a better, faster stack. Like chriv, I would be happy to interface with the ONT and be done with it.

One of these days I wanted to do a repost of everything for a 'newer' version of pfSense .. since the forum moderators started locking thread edits.

[thomas.munn]

@ttmcmurry I added the table just because it makes the whole thing a little simpler. It’s just crazy that AT&T does all this nonsense trying to prevent me from doing multiple segments with IPv6. Honestly I think SLAAC is a lot easier than stupid DHCPv6

[ttmcmurry]

It's important to remember SLAAC doesn't register with DNS or DHCP. This is one of the reasons why it isn't the best solution for all situations. It might be fine for home use if the apps you use don't need DNS to find other devices. SLAAC also doesn't support features like DHCP options, which are commonly used in businesses and especially with VOIP systems.

Between lack of configurability, no ability to perform DNS registration, thus making life very difficult for anyone running an Active Directory domain, SLAAC has a place, but it doesn't work great for corporate environments.

Also, be careful when using SLAAC if you're using pfSense for DNS filtering. When I had it enabled, devices were being configured by the AT&T router as the DNS source and was bypassing pfSense.

[JKnott]

@ttmcmurry

With SLAAC, you point your DNS at the consistent address, which is often based on the MAC address. On the other hand, if you use DHCPv6, you won't be able to use Android devices, as for some stupid reason it's not supported. I have never been able to determine any reason for it not to be supported, other than the developer doesn't like it. His arguments against it could also apply to SLAAC.

[ttmcmurry]

@JKnott
I suppose I have 2 observations:

1. Android devices will still work on the IPv4 stack. If Managed DHCPv6 mode is enabled, you're correct, those will not get an IPv6 address.

2. With respect to using pfSense to perform DNS filtering while using stateless autoconfig/Assisted mode/SLAAC.
   When using DHCPv6 in this manner, when pfSense's IPv6 WAN IP is dynamic, problems become apparent quickly. Since a static IPv6 WAN address is not available to configure in pfSense's DHCPv6 RA DNS Server list, what happens is the RA is passed on from the AT&T RG. Therefore in this configuration, DNS will bypass pfSense..

If you're not using pfSense to filter/route DNS toward a filtering provider, then this isn't a concern and go stateless and you're probably fine with allowing AT&T to monitor where you go on the internet.

Yes, a configuration paradox exists because Google won't support DHCPv6 on Android. The addressing problems are resolved when using DHCPv6 in Managed mode. However, Android devices cannot be allocated IPv6 addresses as a result. In the end the same underlying goal is achievable, albeit without direct IPv6 support.

It's about compromising & determining which compromise covers the majority of your needs.

Of course I would love to be proven wrong. The moment I set DHCPv6 for Stateless or Assisted, my subnets get addresses from the AT&T RG. If that's a configuration error on my side (firewall rule on wan interface??) I'd love to fix that and update this doc!

[lilchancep]

To anyone that comes to this thread I have made a GitHub to have all the changes that have been made throughout this thread. Also for easier reading. [all input is welcome]

Link

@ttmcmurry I hope this is okay, I made sure to give you 190% credit as this is your hard work.

[deet]

A couple of notes:

• On the WAN interface settings page, setting the prefix size to /60 has no effect on dhcp6c or what the dhcp6c client requests from the ISP. The reason for doing that is to tell the web UI what range of prefix IDs to allow on the LAN configuration pages. If you leave it at /64, it will only allow '0', and it won't let you set that on more than one interface. You're setting it to /60 to trick the web UI validator, not because you want a /60 from your ISP.
• With this in mind, although you can use arbitrary values for ia_pd in your config file, you will want to stick to 0–8 for prefix IDs so that they will validate in the UI.

[ttmcmurry]

@deet have you tested this effect?

in 2.4.5, the UI doesn't allow other interfaces to track a wan interface to use a metric higher than 0 when the wan is set to /64.

It was necessary to set both the script and the corresponding interface in the UI to use the same prefix id.

I believe there's a DHCP Service dependency on the interface in the UI to use the tracked interface prefix specified there. Said another way, I do not believe the DHCP v6 Service evaluates a dhcp.conf file if the WAN's DHCP Override contains a conf; it has to use the value specified in the interface's UI.

If this behavior has changed in "p1" - could you provide confirmation and the updated steps for others to follow?

[ttmcmurry]

@lilchancep - thank you. Ever since the netgate forum rules change, it became difficult to keep this up to date. I have been waiting for 2.5 to come out to make any significant overhaul, but would end up in the same place, unable to update it after a few hours.

I've starred the page you created in github. :)

[deet]

@ttmcmurry said in Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5:

@deet have you tested this effect?

I am describing my own findings, so yes.

I was initially confused about the need to set the prefix to /60 in the WAN settings before setting the override file path. "Why would it matter what the prefix length is set to, if we're just going to override it in the .conf file?" I wondered. So I tried setting the WAN prefix length to different values to see what would happen.

I found that the .conf override file is what governs the behavior of dhcp6c, not the value in the field, "overriding" indeed.

Setting the WAN prefix length to /60 does not, therefore, have any bearing on what prefix dhcp6c requests.

But the pfsense web GUI does use that value when validating the prefix IDs on the LAN pages. If you leave it at /64, the only valid prefix ID is 0. The validator looks at /64 and calculates only one available prefix ID. By setting the WAN prefix length to /60, however, the validator calculates a range of up to 16 IDs. (Of which 8 are actually available, but that's a gateway limitation, not an IPv6 thing.)

So like I say, setting it to /60 matters not because it tells dhcp6c to do anything relevant, but because without it, the pfsense web GUI validator won't let you set the values you need to set in the LAN pages.

in 2.4.5, the UI doesn't allow other interfaces to track a wan interface to use a metric higher than 0 when the wan is set to /64.

Right.

It was necessary to set both the script and the corresponding interface in the UI to use the same prefix id.

With this method, yes. But ordinarily, no; "ia_pd" in the .conf file doesn't have to be anything in particular; it is purely an index for configuration purposes. And, ordinarily, the prefix ID in the GUI doesn't have to be anything particular, either, except that it needs to be within a range valid for the delegated prefix, because ordinarily it would end up appended to the delegated prefix being tracked. These are, ordinarily, separate values unrelated to each other.

For example, if this were Comcast, and we'd been properly delegated a /60, we could put 'f' as the ID in one interface and '9' as the ID in another, and each network would get a prefix ending in 'f' and '9' respectively. Thus can this field be set to anything we want as long as it results in a valid prefix.

But because of the way we're fitting square pegs into round holes, we need the ia_pd in the .conf file to match the "prefix ID" in the GUI. Not because it will be appended to the delegated prefix as usual — because the delegated prefix we get will go straight to the tracking interface. Rather, we are using that field to tell pfSense which /64 prefix (referring to the ia_pd set in the conf file) goes to which interface (which is also indicated in the conf file).

In practice, the prefix we get will not be the usual <delegated /60 prefix> + <"prefix suffix" to form a complete /64 prefix>, but rather <whatever /64 prefix is assigned to us by the gateway>.

Apologies if I am trying to dig myself out of a hole with this explanation. It makes sense the way I'm reading it but I get that it's confusing and I am probably not being clear. It's just, after wondering why setting /60 would matter in the GUI when we were using a conf override file (and the answer is, for GUI-related reasons only), my next question was why the prefix ID set in the GUI would have any bearing, since the /64 being delegated has no more bits available for any suffixes to be appended. The answer is, the ID is not actually being appended as usual. Instead, through this method, we are setting not a "prefix suffix" but rather telling pfsense which ia_pd dhpc6c should associate with the interface.

I believe there's a DHCP Service dependency on the interface in the UI to use the tracked interface prefix specified there.

The dependency is in how the pfsense GUI works, not in the DHCP service.

Said another way, I do not believe the DHCP v6 Service evaluates a dhcp.conf file if the WAN's DHCP Override contains a conf; it has to use the value specified in the interface's UI.

Sort of. The .conf file does indeed take full control of what actually happens to all interfaces. But we have to do this GUI stuff to make sure the GUI matches what dhcp6c is doing.

If this behavior has changed in "p1" - could you provide confirmation and the updated steps for others to follow?

The steps are the same! We're doing all the right things. My comments are as much for my own future reference as for anything. "Wait why does it matter what we do in the GUI if we're overriding everything in the conf file? Oh, it's because we need the GUI to match what the conf file is doing, and we have to fudge some things in order for that to happen." That's all.

Clear as mud?

[ttmcmurry]

@deet
Clear as mud. I just installed 2.5 and will get around verifying all of this when I have some spare time. :)

At least we understand each other!

[cbennett2010]

So, I updated to 2.5.0 and attempted to implement this hoping I could get ipv6 running on my individual internal vlans. That being said, I setup the script as above and it looks something like this:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb1.100 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface igb1.110 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 { 
	prefix-interface igb1.200 {
		sla-id 0;
		sla-len 0;
	};
};

However, I WAN is the only interface to pull a ipv6 address, and I am seeing this in the logs:

dhcp6c[64417]: unexpected interface (1)

been having some trouble with ipv6 after updating to 2.5.0 but you're script has given me a little bit of progress. Any help would be appreciated, and if you need more details please let me know and I'll update as soon as possible.

[deet]

@cbennett2010 if you go to Status > interfaces, do the interface names all exist and match?

[cbennett2010]

@deet Yes they match. here's whats shown under status below:

[ttmcmurry]

@cbennett2010

In your example, WAN interface igb0 is calling
script "/var/etc/dhcp6c_wan_script.sh";

Have you tried using the code from the first post instead?
script "var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";

I recently moved to 2.5 and I opted to not have my WAN interface request domain-name and domain-name servers in my ipv6 script. That said, I specify these values elsewhere.

My current config is listed below. Maybe that will help. Also, dhcpd is sensitive to syntax, tabs, etc. I'll also post a screenshot from notepad++ with all characters displayed, perhaps that will help.

interface hn0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	script "/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface hn1.50 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 {
	prefix-interface hn1.40 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 {
	prefix-interface hn2 {
		sla-id 0;
		sla-len 0;
	};
};

[cbennett2010]

@ttmcmurry Thank you for the reply while I try to figure this out. So I made the modifications to change the script that is running, and verified the Text Syntax:

but alas still seeing the same error:

Feb 28 16:41:46 dhcp6c[23870]: Sending Solicit
Feb 28 16:41:46 dhcp6c[64417]: unexpected interface (1)

I'm beginning to wonder if AT&T really only did allocate a /64 to me. Because looking at my RG device and seeing what is allocated to it, I see this:

Global Unicast IPv6 Address	****:****:****:7a90::1
Link-local IPv6 Address	fe80::8a96:4eff:fe89:6d70
IPv6 Addressing Subnet (including length)	****:****:****:7a90::/64
IPv6 Delegated Prefix Subnet (including length)

And this is the address that I see on my WAN:

****:****:****:7a90:92e2:baff:fe80:c6f8

So not sure why i'm not seeing a Delegated Prefix Subnet on my AT&T RG.

Thanks for all the help so far!