IPv6 PPPoE - LAN bridge0 - pfsense can ping6 - LAN clients cannot [SOLVED]

workingman

Hi pfSense forum folks.

I have seen similar issues here referring to problems when using a bridge interface - https://forum.pfsense.org/index.php?topic=64175.0 - and the redmine bug about it - https://redmine.pfsense.org/issues/4218

Unfortunately the workaround there did not resolve my issue.

This setup has worked in the past, unfortunately I have no idea why it did when it did or when it broke exactly… I know, not much help that. :-\

I can say my setup is virtually identical to the setup my brother uses. Same fw hardware (soekris boxes), same ISP, same WAN DHCP6 with Track Interface on the LAN. The only significant difference is he does not use a bridge and everything seems to work great.

From a debian server at my brother's house.

# ping6 pfsense.org
PING pfsense.org(2610:160:11:11::69) 56 data bytes
64 bytes from 2610:160:11:11::69: icmp_seq=1 ttl=55 time=64.6 ms
64 bytes from 2610:160:11:11::69: icmp_seq=2 ttl=55 time=68.3 ms

Debian server here.

# ping6 pfsense.org
PING pfsense.org(2610:160:11:11::69) 56 data bytes
–- pfsense.org ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6129ms

From my pfSense box directly it does work.

[2.3.2-RELEASE]/root: ping6 pfsense.org
PING6(56=40+8+8 bytes) 2607:f2c0:a000:15e:200:24ff:fece:3118 –> 2610:160:11:11::69
16 bytes from 2610:160:11:11::69, icmp_seq=0 hlim=56 time=76.501 ms
16 bytes from 2610:160:11:11::69, icmp_seq=1 hlim=56 time=76.908 ms

I've tried to determine if this is a routing issue or firewall block but even with a working example to look at I haven't been able to put my finger on it, so I'm asking for help here.

I have attempted the workaround from the redmine bug and while I did get a link_local address cooked up from bridge mac I still had no joy.

When comparing to the non-bridge LAN at my brother's I noticed there was no similar address. Just the ISP assigned IP via track interface and inet6 fe80::1:1%vr0 prefixlen 64.

So I have removed the block from interfaces.inc and should be back to stock state.

Any grand ideas folks? Happy to provide more info or test pretty much anything at this point.

junicast

Please paste (IPv6) routing table of the debian host that doesn't do Internet. If default gateway is link local address please also give pfsense's link local address of that very interface.
A look into your firewall rules would also be great.

Where exactly do you bridge and why?

workingman

Thanks for the reply pmisch. Really appreciate it.

Unfortunately I have just updated to 2.3.3 and when it came up I didn't get an IPv6 address on my bridge0 LAN interface at all. Just the router address inet6 fe80::1:1%bridge0

I can say that my simple test of disabling the firewall with pfctl -d did not make it work before when I was on 2.3.2 so I do suspect a routing issue now more than pfsense blocking me.

On both debian boxes I am pretty sure default was just:

::/0 fe80::1:1 UGDAe 1024 0 0 eth0

Not getting any IPv6 inside at my house right now but that's what dhcp set at my brothers and again it works just fine. (he also updated to 2.3.3 today and alls well)

The one other odd thing I noticed was how his mac address for his LAN interface shows up in the IPv6 address on both WAN and LAN sides. Which happens only on the WAN side at my house. Now the bridge0 interface does seem to generate a new mac address each time it gets created so I am starting to point my attention there.

Working brother's box LAN interface ends in cb:1e:28

WAN IP - SNIP-a000:104:200:24ff:fecb:1e28/64
LAN IP - SNIP-f00e:4d00:200:24ff:fecb:1e28/56

Really looks like link local addressing just the prefix is different. OK and his LAN has no link local (no fe80…) just the IP above and the router addy inet6 fe80::1:1%vr0... hmm.

I'll try a few things to see if I can get IPv6 on both WAN and LAN at my house and chime back in once I do.

Firewall rule - Default allow LAN IPv6 to any rule

Finally - Where exactly do you bridge and why?

Soekris box has 4 wired NICs and I have an atheros G card in there too that I use for older devices (iphone 3gs and an old laptop) that are not N capable. So 1 wired NIC for WAN and I bridge the other 3 wired plus wifi to create bridge0 which I then set as the LAN. I have a separate AP to provide N only wireless.

workingman

So I am finally back at home and was able to kick IPv6 into gear.

After noticing the mac address of the LAN seemed to be in use and finding and enabling this sysctl:

net.link.bridge.inherit_mac: 1

Things now look even more like the working setup my brother has here at home with the bridge0 mac showing up in both the WAN and LAN v6 IP.

ether 00:00:24:ce:31:18

WAN IP - SNIP-a000:15e:200:24ff:fece:3118/64
LAN IP - SNIP-f00e:1200:200:24ff:fece:3118/56

Routing table on the not working debian box here - assigned via dhcp6 - looks the same as on the one that works on my brother's network

::/0 fe80::1:1 UGDAe 1024 1 78 eth0

Unfortunately this did not make magic happen and I still seem to be in the same state as before where pfSense is the only device where IPv6 actually works.

LAN clients can ping both LAN and WAN IPs of pfSense but go no further.

Any other info I can provide or things to test just let me know.

doktornotor

https://redmine.pfsense.org/issues/4218

workingman

Hi doktornotor and thanks for the reply.

I did include that redmine bug link in my original post having already tried the workaround from post #4 https://redmine.pfsense.org/issues/4218#note-4 but saw no change in IPv6 connectivity from my LAN clients.

Noticing significant changes to that interfaces.inc file now that I'm on 2.3.3 I just tried the workaround again but still no joy.

Going to try another workaround, setting auto_linklocal on the bridge interface instead of the one suggested and will report back should any magic happen.

workingman

Hi again,

I've made what I think is a much simpler mod to the interfaces.inc file and do see link local address on my bridge interface.

inet6 fe80::1:1%bridge0 prefixlen 64 scopeid 0xc
inet6 fe80::200:24ff:fece:3118%bridge0 prefixlen 64 scopeid 0xc
nd6 options=21 <performnud,auto_linklocal>Only two lines and one is just a comment but still no change from the LAN.


[2.3.3-RELEASE]/root: diff /etc/inc/interfaces.inc /etc/inc/interfaces.inc.org 
568,570d567
<         /* Enable auto_linklocal for bridges */
<     	   mwexec("/sbin/ifconfig {$bridge['bridgeif']} inet6 auto_linklocal");
<

```</performnud,auto_linklocal>

Fabio72

I'm on 2.3.3 with an HE tunnel
My lan interface is a bridge and has a link local address and ipv6 is working as before bridging the lan side

bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
ether 02:80:65:67:b1:00
inet 192.168.1.250 netmask 0xffffff00 broadcast 192.168.1.255 
inet6 2001:470:SNIP:1::1 prefixlen 64 
inet 172.16.16.1 netmask 0xffffffff broadcast 172.16.16.1 
inet6 fe80::280:65ff:fe67:b100%bridge0 prefixlen 64 scopeid 0x12 
nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto stp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: em1_vlan10 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 10 priority 128 path cost 20000
member: em1_vlan30 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 12 priority 128 path cost 20000</learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud></up,broadcast,running,simplex,multicast>

workingman

Hi Fabio!

Glad to hear it is working somewhere with bridging. Gives me hope :)

Quick question for you though. Have you modified the interfaces.inc like https://forum.pfsense.org/index.php?topic=64175.0 or did it just work?

My mod to add auto_linklocal seems to do the same thing but I'm still struggling to get any LAN clients working.

I'm testing today by trying to enable WAN access to some IPv6 services to see what that does.

Just did some quick routing checks using - route -n get -inet6 pfsense.org - and it does know which interface to send traffic out (or in to my LAN). Makes me think the problem may be upstream and possibly out of my control.

workingman

Good news everyone!

After some tcpdump testing using my brother's house as my target I was able to see that my IPv6 traffic was getting to him and he was sending replies but they never made it back home to me.

Forwarded that info to my ISP and they have fixed it. So not a pfSense issue at all nor was it due to the use of a bridge interface. Thanks to everyone who replied. It helped steer me in the right direction.

From - http://test-ipv6.com/

Your Internet Service Provider (ISP) appears to be TEKSAVVY - TekSavvy Solutions, Inc., CA

Good news! Your current configuration will continue to work as web sites enable IPv6.

Your DNS server (possibly run by your ISP) appears to have IPv6 Internet access.
Your readiness score
10/10

Will it still work tomorrow? Who can say, but I'm happy now and we can mark this issue SOLVED (if I can find that button I'll do it myself).

Fabio72

@workingman:

Quick question for you though. Have you modified the interfaces.inc like https://forum.pfsense.org/index.php?topic=64175.0 or did it just work?

I did with no modifications. Just enabled all from the gui.