Switched to AT&T fiber, IPv6 tunnel broken
-
@johnpoz said in Switched to AT&T fiber, IPv6 tunnel broken:
I don't think those devices do a true bridge, more like a nat with with dmz host sort of thing.
It's similar, but different. In Passthrough mode, pfSense gets the public v4 IP of the BGW. For v6, the pfSense gets a routeable IP via DHCP for its WAN IP, and any delegated prefixes that it requests. pfSense thinks that it's directly on the WAN with routeable addresses and prefixes.
The BGW then 1:1 NATs every flow to/from pfSense, keeping the same source/destination address/port on both sides of the NAT. There's a snippet from the BGW's NAT table below. The x's are to obscure my routeable addresses and prefixes.
Guessing one reason they don't do a true bridge is to enable the BGW to NAT+route in parallel for its non-Passthrough LAN-side clients (none in my case).
-
@marcg well that is good, then it should work for the OP.
-
Ran a traceroute yesterday,
[25.07.1-RELEASE][root@firewall]/root: traceroute -P 41 184.1.1.1 traceroute to 184.1.1.1 (184.1.1.1), 64 hops max, 40 byte packets 1 dsldevice.attlocal.net (192.168.1.254) 0.574 ms 0.472 ms 0.469 ms 2 * * * 3 * * * 4 * * * 5 * * * 6 192.205.32.138 (192.205.32.138) 14.657 ms 15.537 ms * 7 * * *A separate run on that showed a different path but at a similar point I got a response from something and then nothing else.
One solution that I keep seeing during my searching is to switch to a business account. Just got a basic price list for that and it's flat out ridiculous. Over a thousand dollars a month to be provisioned at 1Gbps and no idea if I'd still see this problem or not.
-
@BiloxiGeek said in Switched to AT&T fiber, IPv6 tunnel broken:
traceroute -P 41 184.1.1.1
Protocol 41 is not a port.. Traceroute support tcp and udp I don't think that is going to work.
-
@johnpoz I was given the impression that the traceroute on a pfSense box can use the -P 41 to test protocols. But I will dig into it deeper once I get home from work to doublecheck.
-
@BiloxiGeek yeah it is possible to do raw and specific protocol - but not sure 41 is an option.. But that might depend on what version of traceroute, etc. I think you can do such things with hping3 as well.. But not sure of all protocols that you can do, etc.
Just wanted to let you know that might want to verify it was doing what you expected it to be doing before you think its reporting a clear failure, etc.
-
@johnpoz I'm gonna setup a linode nanode this afternoon to do some testing from outside AT&T's sphere of influence. See if I can get a better idea of where the 6in4 packets are being dropped.
I did find this a bit ago:
nmap -6 -sO -p 41 <ip>(That's a capital letter O.)
That appears to probe for protocol 41 specifically. I have to test when I get home to see if I should be testing to the v4 address or the v6. (or both)
-
@BiloxiGeek this is what I get, which I do have he tunnel up.
Starting Nmap 7.97 ( https://nmap.org ) at 2025-09-04 13:56 -0500 Nmap scan report for tserv9.chi1.ipv6.he.net (184.105.253.14) Host is up (0.010s latency). PROTOCOL STATE SERVICE 41 open|filtered ipv6 Nmap done: 1 IP address (1 host up) scanned in 0.44 secondsYou do understand you should be checking to your HE tunnel IPv4 address for the tunnel endpoint you are using.. For example you see above I am using the chi pop for HE at that 184.105.254.14 address
-
Here's what I see with a BGW320 in Passthrough mode using ATT's native v4/v6 services. No HE tunnel running.
Testing was done from an Ubuntu machine behind pfSense to the HE POP address provided by @johnpoz . First hop is pfSense, second hop is the BGW, and the packet makes it to HE no later than hop 8.
prompt# traceroute -P 41 184.105.254.14 traceroute to 184.105.254.14 (184.105.254.14), 30 hops max, 60 byte packets 1 pfSense.home.arpa (192.168.15.1) 0.136 ms 0.080 ms 0.080 ms 2 192.168.1.254 (192.168.1.254) 0.700 ms 0.373 ms 0.398 ms 3 * * * 4 * * * 5 * * * 6 * * * 7 * * * 8 e0-35.switch1.sjc2.he.net (64.62.171.65) 4.678 ms 5.896 ms 3.869 ms 9 * * * 10 * * * 11 * * * -
@marcg yeah I don't think that traceroute will work. What does your nmap test show
edit:
You know I don't even think the nmap test is valid.. I picked another tunnel endpoint so not to mess with with current tunnel I have open.. And when I sniff on pfsense wan and do another nmap.. I see nmap send what appears like protocol 41, but get no answer and lists as invalid.. Did it actually go out the wire?? and just not get an answer.. Or did it not really go out the wire and just caught in the packet capture.
15:10:57.645055 IP 209.snipped > 184.105.253.10: IP6 [length 0 < 40] (invalid) 15:10:57.806265 IP 209.snipped> 184.105.253.10: IP6 [length 0 < 40] (invalid)184.105.253.10 is their dallas pop tserv8.dal1.ipv6.he.net
If I sniff the previous IP where I do have a tunnel open... I see responses.
So I am not sure there is a way to test if 41 is open with nmap or traceroute..
but since @marcg is using the same device and also on att and can open a tunnel - this shows that at least where he is at they are not filtering.. Why would they filter it some locations but not all?
-
@johnpoz said in Switched to AT&T fiber, IPv6 tunnel broken:
@marcg yeah I don't think that traceroute will work. What does your nmap test show
The traceroute showed that Protocol 41, IPv6, makes it through the ATT network to HE (with no HE tunnel up).
Similarly, from a LAN host, nmap shows he.net is reachable over v6.
prompt# nmap -6 -sO -p 41 tserv9.chi1.ipv6.he.net Starting Nmap 7.80 ( https://nmap.org ) at 2025-09-04 13:30 PDT Nmap scan report for tserv9.chi1.ipv6.he.net (2001:470:0:6e::2) Host is up (0.049s latency). Other addresses for tserv9.chi1.ipv6.he.net (not scanned): 184.105.253.14 rDNS record for 2001:470:0:6e::2: tserv1.chi1.he.net PROTOCOL STATE SERVICE 41 open|filtered ipv6It'd be helpful if the OP left the BGW's IP Passthrough up, re-enabled IPv6 on its LAN side, and repeated these tests from their machine. As you suggested earlier, disabling LAN side IPv6 on the BGW could be the cause of the problem. Testing that is simpler -- a few mouse clicks -- than any other testing we're doing in this thread.
-
@BiloxiGeek do you have a static ipv4? I pay extra for a static address it helps because my tunnel ipv4 address stays the same and my vpn also.
-
@marcg said in Switched to AT&T fiber, IPv6 tunnel broken:
prompt# nmap -6 -sO -p 41 tserv9.chi1.ipv6.he.net
-p is port, not protocol.
6in4 is protocol 41. It is not UDP (17) or TCP (6). You'd use the port number as part of a UDP or TCP packet, not 6in4.
-
@JKnott said in Switched to AT&T fiber, IPv6 tunnel broken:
-p is port, not protocol.
6in4 is protocol 41. It is not UDP (17) or TCP (6). You'd use the port number as part of a UDP or TCP packet, not 6in4.
For nmap with -sO, -p specifies protocol.
This surprised me at first, too.
-
I do not have a static IP. I just connected my linux system directly to one of the 1G ports on the BGW320. I get a 192.168 address. That’s to be expected I figure since I’ve set the passthrough mode to DHCPS-Fixed and given it the pfSense mac address.
I do get two IPv6 addresses on eth0, one is a prefix::49/128 and another the same /64 prefix with the last half of the address defined based on my mac address. I can ping over IPv6 to tunnelbroker.net so it looks like IPv6 native is okay.
With those two addresses on my linux system should I be able to get pfSense to request a /64 and then use that prefix to set up static IPv6 addresses on my LAN systems? I think I tried that a few days ago and it never worked, but I may have made various changes between then and now. That’s the end result I’d like to get to. I’d prefer to use the addresses I got from he.net if possible but if I have to use this native stack it wouldn’t be too bad. I’d have to keep track of the delegation they give me cause if it changes I’d have to go back to he.net’s free DNS and update everything there.
-
@BiloxiGeek said in Switched to AT&T fiber, IPv6 tunnel broken:
I do get two IPv6 addresses on eth0, one is a prefix::49/128 and another the same /64 prefix with the last half of the address defined based on my mac address. I can ping over IPv6 to tunnelbroker.net so it looks like IPv6 native is okay.
With those two addresses on my linux system should I be able to get pfSense to request a /64 and then use that prefix to set up static IPv6 addresses on my LAN systems? I think I tried that a few days ago and it never worked, but I may have made various changes between then and now. That’s the end result I’d like to get to. I’d prefer to use the addresses I got from he.net if possible but if I have to use this native stack it wouldn’t be too bad. I’d have to keep track of the delegation they give me cause if it changes I’d have to go back to he.net’s free DNS and update everything there.
Sounds like the pfSense WAN interface has both routeable DHCPv6 and SLAAC addresses on the segment connecting it to the BGW. Mine does, too, even though the interface is configured for DHCPv6. If you use DDNS, one or the other will get registered. Either one will work.
To get a delegated /64 prefix to use on one LAN-side interface, use Track Interface. If you need prefixes for multiple LAN-side interfaces -- the BGW can hand out up to eight -- this thread explains how to request multiple /64s. There's another explanation of the steps here.
To enable inbound access from the Internet with dynamic prefixes/addresses, I run parallel ULAs and GUAs on my LAN networks. I have static ULAs for key machines internally. On the WAN interface, there are port forwards/NATs that direct incoming traffic on the pfSense WAN address to the ULA of the appropriate internal host. Since the ULAs are fixed, inbound traffic for a particular service always lands on the correct internal host regardless of what the WAN address is at that time. Works well for my simple network.
I don't pay attention to the GUAs of internal hosts other than making sure firewall rules are correctly configured.
Also worth noting that my ATT prefixes haven't changed in several years -- across reboots, etc. -- even though they're dynamic.
