IPv6 PPPoE - LAN bridge0 - pfsense can ping6 - LAN clients cannot [SOLVED]

  • Hi pfSense forum folks.

    I have seen similar issues here referring to problems when using a bridge interface - https://forum.pfsense.org/index.php?topic=64175.0 - and the redmine bug about it - https://redmine.pfsense.org/issues/4218

    Unfortunately the workaround there did not resolve my issue.

    This setup has worked in the past, unfortunately I have no idea why it did when it did or when it broke exactly… I know, not much help that.  :-\

    I can say my setup is virtually identical to the setup my brother uses.  Same fw hardware (soekris boxes), same ISP, same WAN DHCP6 with Track Interface on the LAN.  The only significant difference is he does not use a bridge and everything seems to work great.

    From a debian server at my brother's house.

    # ping6 pfsense.org
    PING pfsense.org(2610:160:11:11::69) 56 data bytes
    64 bytes from 2610:160:11:11::69: icmp_seq=1 ttl=55 time=64.6 ms
    64 bytes from 2610:160:11:11::69: icmp_seq=2 ttl=55 time=68.3 ms

    Debian server here.

    # ping6 pfsense.org
    PING pfsense.org(2610:160:11:11::69) 56 data bytes
    –- pfsense.org ping statistics ---
    7 packets transmitted, 0 received, 100% packet loss, time 6129ms

    From my pfSense box directly it does work.

    [2.3.2-RELEASE]/root: ping6 pfsense.org
    PING6(56=40+8+8 bytes) 2607:f2c0:a000:15e:200:24ff:fece:3118 –> 2610:160:11:11::69
    16 bytes from 2610:160:11:11::69, icmp_seq=0 hlim=56 time=76.501 ms
    16 bytes from 2610:160:11:11::69, icmp_seq=1 hlim=56 time=76.908 ms

    I've tried to determine if this is a routing issue or firewall block but even with a working example to look at I haven't been able to put my finger on it, so I'm asking for help here.

    I have attempted the workaround from the redmine bug and while I did get a link_local address cooked up from bridge mac I still had no joy.

    When comparing to the non-bridge LAN at my brother's I noticed there was no similar address.  Just the ISP assigned IP via track interface and inet6 fe80::1:1%vr0 prefixlen 64.

    So I have removed the block from interfaces.inc and should be back to stock state.

    Any grand ideas folks?  Happy to provide more info or test pretty much anything at this point.

  • Please paste (IPv6) routing table of the debian host that doesn't do Internet. If default gateway is link local address please also give pfsense's link local address of that very interface.
    A look into your firewall rules would also be great.

    Where exactly do you bridge and why?

  • Thanks for the reply pmisch.  Really appreciate it.

    Unfortunately I have just updated to 2.3.3 and when it came up I didn't get an IPv6 address on my bridge0 LAN interface at all.  Just the router address inet6 fe80::1:1%bridge0

    I can say that my simple test of disabling the firewall with pfctl -d did not make it work before when I was on 2.3.2 so I do suspect a routing issue now more than pfsense blocking me.

    On both debian boxes I am pretty sure default was just:

    ::/0                          fe80::1:1                  UGDAe 1024 0    0 eth0

    Not getting any IPv6 inside at my house right now but that's what dhcp set at my brothers and again it works just fine.  (he also updated to 2.3.3 today and alls well)

    The one other odd thing I noticed was how his mac address for his LAN interface shows up in the IPv6 address on both WAN and LAN sides.  Which happens only on the WAN side at my house.  Now the bridge0 interface does seem to generate a new mac address each time it gets created so I am starting to point my attention there.

    Working brother's box LAN interface ends in cb:1e:28

    WAN IP - SNIP-a000:104:200:24ff:fecb:1e28/64
    LAN IP - SNIP-f00e:4d00:200:24ff:fecb:1e28/56

    Really looks like link local addressing just the prefix is different.  OK and his LAN has no link local (no fe80…) just the IP above and the router addy inet6 fe80::1:1%vr0... hmm.

    I'll try a few things to see if I can get IPv6 on both WAN and LAN at my house and chime back in once I do.

    Firewall rule - Default allow LAN IPv6 to any rule

    Finally - Where exactly do you bridge and why?

    Soekris box has 4 wired NICs and I have an atheros G card in there too that I use for older devices (iphone 3gs and an old laptop) that are not N capable.  So 1 wired NIC for WAN and I bridge the other 3 wired plus wifi to create bridge0 which I then set as the LAN.  I have a separate AP to provide N only wireless.

  • So I am finally back at home and was able to kick IPv6 into gear.

    After noticing the mac address of the LAN seemed to be in use and finding and enabling this sysctl:

    net.link.bridge.inherit_mac: 1

    Things now look even more like the working setup my brother has here at home with the bridge0 mac showing up in both the WAN and LAN v6 IP.

    ether 00:00:24:ce:31:18

    WAN IP - SNIP-a000:15e:200:24ff:fece:3118/64
    LAN IP - SNIP-f00e:1200:200:24ff:fece:3118/56

    Routing table on the not working debian box here - assigned via dhcp6 - looks the same as on the one that works on my brother's network

    ::/0                          fe80::1:1                  UGDAe 1024 1    78 eth0

    Unfortunately this did not make magic happen and I still seem to be in the same state as before where pfSense is the only device where IPv6 actually works.

    LAN clients can ping both LAN and WAN IPs of pfSense but go no further.

    Any other info I can provide or things to test just let me know.

  • Banned

  • Hi doktornotor and thanks for the reply.

    I did include that redmine bug link in my original post having already tried the workaround from post #4 https://redmine.pfsense.org/issues/4218#note-4 but saw no change in IPv6 connectivity from my LAN clients.

    Noticing significant changes to that interfaces.inc file now that I'm on 2.3.3 I just tried the workaround again but still no joy.

    Going to try another workaround, setting auto_linklocal on the bridge interface instead of the one suggested and will report back should any magic happen.

  • Hi again,

    I've made what I think is a much simpler mod to the interfaces.inc file and do see link local address on my bridge interface.

    inet6 fe80::1:1%bridge0 prefixlen 64 scopeid 0xc
    inet6 fe80::200:24ff:fece:3118%bridge0 prefixlen 64 scopeid 0xc
    nd6 options=21 <performnud,auto_linklocal>Only two lines and one is just a comment but still no change from the LAN.

    [2.3.3-RELEASE]/root: diff /etc/inc/interfaces.inc /etc/inc/interfaces.inc.org 
    <         /* Enable auto_linklocal for bridges */
    <     	   mwexec("/sbin/ifconfig {$bridge['bridgeif']} inet6 auto_linklocal");

  • I'm on 2.3.3 with an HE tunnel
    My lan interface is a bridge and has a link local address and ipv6 is working as before bridging the lan side

    bridge0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
    ether 02:80:65:67:b1:00
    inet netmask 0xffffff00 broadcast 
    inet6 2001:470:SNIP:1::1 prefixlen 64 
    inet netmask 0xffffffff broadcast 
    inet6 fe80::280:65ff:fe67:b100%bridge0 prefixlen 64 scopeid 0x12 
    nd6 options=1 <performnud>id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto stp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: em1_vlan10 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 10 priority 128 path cost 20000
    member: em1_vlan30 flags=143 <learning,discover,autoedge,autoptp>ifmaxaddr 0 port 12 priority 128 path cost 20000</learning,discover,autoedge,autoptp></learning,discover,autoedge,autoptp></performnud></up,broadcast,running,simplex,multicast>

  • Hi Fabio!

    Glad to hear it is working somewhere with bridging.  Gives me hope  :)

    Quick question for you though.  Have you modified the interfaces.inc like https://forum.pfsense.org/index.php?topic=64175.0 or did it just work?

    My mod to add auto_linklocal seems to do the same thing but I'm still struggling to get any LAN clients working.

    I'm testing today by trying to enable WAN access to some IPv6 services to see what that does.

    Just did some quick routing checks using - route -n get -inet6 pfsense.org - and it does know which interface to send traffic out (or in to my LAN).  Makes me think the problem may be upstream and possibly out of my control.

  • Good news everyone!

    After some tcpdump testing using my brother's house as my target I was able to see that my IPv6 traffic was getting to him and he was sending replies but they never made it back home to me.

    Forwarded that info to my ISP and they have fixed it.  So not a pfSense issue at all nor was it due to the use of a bridge interface.  Thanks to everyone who replied.  It helped steer me in the right direction.

    From - http://test-ipv6.com/

    Your Internet Service Provider (ISP) appears to be TEKSAVVY - TekSavvy Solutions, Inc., CA
    Good news! Your current configuration will continue to work as web sites enable IPv6.

    Your DNS server (possibly run by your ISP) appears to have IPv6 Internet access.
    Your readiness score

    Will it still work tomorrow?  Who can say, but I'm happy now and we can mark this issue SOLVED (if I can find that button I'll do it myself).

  • @workingman:

    Quick question for you though.  Have you modified the interfaces.inc like https://forum.pfsense.org/index.php?topic=64175.0 or did it just work?

    I did with no modifications. Just enabled all from the gui.