Multiple IPv6 Prefix Delegation over AT&T Residential Gateway for pfSense 2.4.5

cbennett2010

@ttmcmurry Ok so, I made a little bit of progress, there is something definitely different with pfsense. So I disabled everything IPv6 within pfsense, and ssh'd into the box and killed all dhcp6c processes, I then proceeded to manually run the dhcp6c client on my WAN interface with the following command:

/usr/local/sbin/dhcp6c -D -c /usr/local/etc/rc.d/att-rg-dhcpv6-pd.conf  igb0

when I did that, amazingly I was able to pull ipv6 addresses on all my interfaces as was originally expected. However, now that I have everything turned off, the DHCP6 servers and RA's aren't on, but still, a little bit of progress.

Any thoughts of where to look next, i'm just poking around in the dark at this point.

Thanks Again!

cbennett2010

@ttmcmurry Is it possible that through the chains of scripts being ran since we call this script within yours:

/var/etc/dhcp6c_wan_dhcp6withoutra_script.sh

then that script calls:

/var/etc/rtsold_igb0_script.sh

which should be starting dhcp6c client. along with setting a few other things, but after tinkering around, i've fond that I start to see this in the logs:

XID mismatch

Which makes me wonder, is it somehow calling dhcp6c client to run multiple times? Because if I comment out the dhcp6c_wan_dhcp6withoutra_script.sh from your script dhcp6c still starts and assigns addresses.

lilchancep

So idk how much this will help you @ttmcmurry but on my pfsense 2.5 Installation this script worked without any changes at all.

mitsurugi78

@lilchancep Which script is that you used for 2.5? I've seen a few different posts of scripts and am a bit unsure of which to use now. Thanks

lilchancep

@mitsurugi78 Here is all the steps taken from this thread and cleaned up.

https://github.com/lilchancep/att-pfsense-ipv6

mitsurugi78

@lilchancep thanks greatly appreciated!

ttmcmurry

@mitsurugi78

I think we're also highlighting some bugs - see redmine #11187 and #11454

@lilchancep

It might be time to start a 2.5 thread. :)

slab_bulkhead

@ttmcmurry thank you so much for your work on this! One of my biggest irritations with AT&T was the inability to pull more than one /64, while on Spectrum I can get a /56 PD with no issues at all. I have this working on 2.5 -- I had some issues at first and then discovered it was because things do not behave well with IPv6 enabled on multiple WAN interfaces at the same time (I still have the Spectrum modem connected until service cancels out at the end of the month).

I am on VDSL and therefore am unable to attempt bypassing the gateway.

djdawson

@ttmcmurry I've got a Humax BGW320-500 with my symmetric 1G service from AT&T and its NAT State Table size is 8192. Here are my RG details, copied from the device status page:

Manufacturer: HUMAX
Model Number: BGW320-500
Software Version: 2.14.4
Hardware Version: 02001F0046005

styxl

I have attempted to do this on "21.05.2-RELEASE" to no avail; the script simply doesnt seem to work anymore.

this is the error i get:

Dec  2 20:40:00 Scimitar dhcp6c[58269]: /var/etc/dhcp6c_wan.conf 20: syntax error
Dec  2 20:40:00 Scimitar dhcp6c[58269]: /var/etc/dhcp6c_wan.conf 20: fatal parse failure: exiting (1 errors)
Dec  2 20:40:00 Scimitar dhcp6c[58269]: failed to parse configuration file
Dec  2 20:40:00 Scimitar rtsold[58345]: Starting dhcp6 client for interface wan(igb0)
Dec  2 20:40:01 Scimitar reboot[98400]: rebooted by root
Dec  2 20:40:01 Scimitar syslogd: exiting on signal 15

styxl

my config is as such:

interface igb0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface ix0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 { 
	prefix-interface ix1 {
		sla-id 0;
		sla-len 0;
	};
};

dmac1418

I wanted to chime in and thank you for all your info on this thread.

I followed your guide via the GitHub adaptation and it worked no problem on 2.5.2-RELEASE (amd64) and the AT&T BGW320 modem/gateway.

interface em0 {
	send ia-na 0;
	send ia-pd 0;
	send ia-pd 1;
	send ia-pd 2;
	send ia-pd 3;
	request domain-name-servers;
	request domain-name;
	script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
	prefix-interface igb0 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 1 { 
	prefix-interface igb0.11 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 2 { 
	prefix-interface igb0.12 {
		sla-id 0;
		sla-len 0;
	};
};
id-assoc pd 3 { };

Thanks & Happy New Year!

styxl

@dmac1418 lucky you; i gave up on making it work on the SG-5100

thekorn

Hello!

I'm running pfsense 2.6.0-release, and am trying to get ipv6 to work with multiple VLANs.

I've been able to implement all steps except step 7, enabling the DHCPv6 server and testing. When I go to dhcpv6 server & ra, dhcpv6 server, enable, save it kicks back "A valid range must be specified for any mode except Stateless DHCP."

I put in a range of :: to ::ffff:ffff:ffff:ffff, and that made it happy. Might want to update the... github.

Note that I was able to retireve an ipv6 address without this. I assume this is because SLAAC is being passed through to the residential gateway, and it's assigning the IPv6 address instead?

Thanks for all the hard work documenting this! I certainly wasn't going to figure this all out on my own!

styxl

@styxl i finally got mine to work, my config was wrong;

interface igb0 {
        send ia-na 0;
        send ia-pd 0;
        send ia-pd 1;
        send ia-pd 2;
        send ia-pd 3;
        send ia-pd 4;
        send ia-pd 5;
        send ia-pd 6;
        request domain-name-servers;
        request domain-name;
        script "/var/etc/dhcp6c_wan_script.sh";
};
id-assoc na 0 { };
id-assoc pd 0 {
        prefix-interface ix0 {
                sla-id 0;
                sla-len 0;
        };
};
id-assoc pd 1 {
        prefix-interface ix1.101 {
                sla-id 0;
                sla-len 0;
        };
};
id-assoc pd 2 { };
id-assoc pd 3 { };
id-assoc pd 4 { };
id-assoc pd 5 { };
id-assoc pd 6 { };
id-assoc pd 7 { };

lilchancep

@thekorn Updated the repo, let me know if you have anything else you think I should add.

thekorn

@lilchancep Now that you mention it, I suppose now is a good a time as any to add that it works with a Nokia BGW320-505 (fw 3.15.7).

(States are same as before, 8k, set pf to 7.5k)

lolipoplo

I see discussions on ATT's state table limit.

I tested it using nmap and it doesn't seem to be a concern really

I have once blown the states to almost 100 k on pfsense and the network still seemed working fine browsing the web, while the entries on BGW320 stayed at 8k.

since the RG is set to FW/NAT passthru (maybe has only one rule that's similar to allow from any to any), any packet can effective create a state, removing an old one.

I think it's a bad idea to limit the state table on pfsense, because if you are maxing out on RG's state table and pf removes your state, your mid-flight connections all have to reset

ttmcmurry

@lolipoplo - (edit) I'm going to attempt answering this without knowing the use case for your pfSense installation or the composition of the devices & traffic in your network.

If pfSense is taking on the WAN IP of the RG (meaning you aren't using static IPs from AT&T), and if the states table in the RG has 8k entries in it, then its state table is full and has been depleted. The RG is unaware of the states pfSense is tracking and vice versa. What you'd need to do is look at pfSense's state table and examine the state flags. If the RG's state table is depleted, you should see a high number of syn_sent, no_traffic (stateful/tcp), or single:no_traffic (stateless/udp). Another way of looking at it is there could be more than 8k pfSense states in established/single/multiple:established/single/multiple state - depending on how fast the RG is terminating those connections and pfSense gives up/times out; that number would be nowhere near 100K established states. You'd need to dump both state tables and compare the two - neither table is telling the full truth on its own.

If you have AT&T static IPs, and have assigned a Static IP to pfSense's WAN MAC, that configuration bypasses the RG's state table. pfSense is then in full control of its states without the RG being the middleman.

What I'm unsure about is what the behavior of "Cascaded Router" does as I've never gotten that to work .. the fee for static IPs was more than affordable considering how much time I sank into trying to get Cascaded Router to work when the benefits of using it were also unclear.

The idea expressed in this forum thread is if pfSense shares the WAN IP with the AT&T RG, then pfSense must respect the RG's state table limitations for overall network reliability. Keep in mind the RG needs states to do other things beyond internet: TV (uVerse), Voice/VoIP, MoCA, built-in WiFi, and so on. Exhausting this limit with pfSense would likely cause adverse performance for anything connected to the RG, that's not behind pfSense, but also anything connected to pfSense where the RG is likely killing states or not accepting new connections from pfSense, due to exhaustion.

lolipoplo

@ttmcmurry I am using the IP Passthru (pfS taking the WAN IP of RG)

I was just doing nmap syn scan on a /64 prefix, which is of course likely just SYN_SENT:CLOSED so I didn't read too much into pf/RG's state tables.

I have no idea of how RG handles state table either. Would CLOSED/NO_TRAFFIC be prioritzed for deletion? then the least recently matched states? I was making this guess because it did not impact my ESTABLISHED. (downloading huge file or running speedtest.net)

I have no devices other than pfS connected to RG.

Maybe I should try again with with more than 8k ESTABLISHED and then see the behavior