Cannot Ping pfSense ULA From Subnet Without ULA Assigned to Firewall



  • I'm seeing some unexpected behavior that I wanted to run by the pfSense community before filing a bug report. I have a pfSense gateway with roughly 10 VLANS. On some of these VLANs, the firewall is set to track the WAN IPv6 interface and hand out GUAs. On other VLANs, I've assigned the firewall a ULA and distribute the ULA prefix via RAs or static assignments.

    During a recent attempt to assign all of my VLANs the same DNS server IP (fd53::1), I discovered that not all of my subnets could reach the ULA address, despite simplifying my firewall rules (for testing purposes) to allow all traffic between subnets. Can anyone confirm that they're seeing the same behavior (or not) before I file a bug report? Here are some additional details:

    • I've assigned pfSense a ULA (fd53::1) on a VLAN interface (VLAN 53). My other VLANs all have their own interfaces and corresponding subnets, with various addressing schemes.

    • If the source subnet has a ULA address set on the firewall interface for that VLAN (for instance, fd00:172:28:203::1), then devices in that subnet can ping the firewall's ULA addresses from a different subnet.

    • If the firewall is set to track the WAN interface on a subnet (and thus, doesn't have a ULA assigned on its subnet-facing interface), then devices in that subnet cannot ping any of the firewall's ULA's in different subnets.

    • Devices in different subnets can ping each other's ULA's OK.

    • Devices in different subnets can ping GUA's on the firewall OK. For instance, using the same source and destination subnets, I can ping the firewall if I assign it an address of 53::1. If I change the firewall address to fd53::1, pings no longer succeed.

    • Pings run directly on the firewall itself (using the diagnostic menu) work OK between subnets regardless of ULA assignment.

    To me, this suggests an issue with the firewall's routing between ULA and non-ULA interfaces, but I wanted to get some feedback before pursuing this further. I'm aware that there is already a bug report for ULA's assigned as VIP's - this isn't related because I'm not using VIP's here.


  • LAYER 8 Global Moderator

    And what specific rules do you have on these interfaces?

    Lets call it Lan1 and Lan2

    So what are your rules if source is set as Lan1 Net for your rule, then that will be the prefix that is set on the interface. if you have a GUA on it then lan1 net would be that prefix.. If you have a ULA then that would be the lan net..

    So lets be clear on what rules you have in place on your different interfaces.

    You said you simplified the rules - so what are these exactly?

    Happy to try and duplicate.. I have some vlans that are GUA, and other that have no IPv6, so should be easy to just assign a ULA on one of these.. And then your saying devices in the gua prefix can not talk to the ula pfsense IP interface?

    edit: Trying to duplicate..

    So I have my lan, which has gua on it from HE part of my /48
    I then on my DTV vlan which had no ipv6 on it at all put

    fd53::1/64 per your example

    Now from the HE lan vlan I can ping pfsense ula vlan interface..
    C:>ping fd53::1

    Pinging fd53::1 with 32 bytes of data:
    Reply from fd53::1: time<1ms
    Reply from fd53::1: time<1ms
    Reply from fd53::1: time<1ms

    From this

    If the firewall is set to track the WAN interface on a subnet (and thus, doesn't have a ULA assigned on its subnet-facing interface), then devices in that subnet cannot ping any of the firewall's ULA's in different subnets.

    My setup seems to work fine, I have a gua vlan and it can ping ula vlan.. My isp does not support ipv6 so I can not easy test tracking a tracked interface... But I don't see how it would be any different than static assigned gua..

    Rules on my lan which is the gua prefix are just any any ipv4+ipv6 with source be set to lan net.



  • Hi @johnpoz,

    Appreciate your taking the time to test this out! I'm trying to ping from a GUA to a ULA, which is what you tested, so it sounds like it's working OK for you. It would appear that we have several differences between our setups, though. My main LAN subnet is tracking the WAN IPv6 prefix, and I also advertise a ULA prefix. As you said, not sure why either of those things would make any difference, but I'm trying to narrow down what may be causing the behavior I'm seeing. Would you mind posting screenshots of your interface configurations? I'm including the same for my interfaces below, as well as a image of my firewall config. It's set to Allow All IPv4&6 from Any to Any in both VLANs.

    0_1543668459674_FirewallRules.png 0_1543668528081_2018-12-01 (1).png 0_1543668532684_2018-12-01 (3).png


  • LAYER 8 Global Moderator

    sure here is my gua interface, that my pc is in - lan.. And here is the vlan I have running that my dtv interface with the ula address assigned.

    And then my pc info that is in the gua lan segment that I ping the ula.. Do you have any rules in floating?

    0_1543674074359_guaulainfo.png

    0_1543674109549_lanrules.png

    I noticed you have a /128 prefix - that is WRONG ;)

    That is more than likely your issue.



  • @johnpoz, are you sure that the /128 prefix is wrong? I only have one IP address in that subnet - the firewall - as incoming ping requests should be routed from a different subnet. Nevertheless, I tried changing both subnets to a /64 prefix for testing purposes, but that didn't allow me to ping the firewall.

    I only have one floating rule, and that's just a Match for QoS tagging. Here are some screenshots as a sanity check. Any other ideas as to what might be causing this?
    0_1543872819764_2018-12-03 (2).png
    0_1543872878994_2018-12-03 (1).png



  • @johnpoz said in Cannot Ping pfSense ULA From Subnet Without ULA Assigned to Firewall:

    I noticed you have a /128 prefix - that is WRONG ;)

    I have a /128 on my WAN interface. Works fine, including pinging. Packet Capture shows the pings coming from that /128 address.


  • LAYER 8 Global Moderator

    Dude see the other thread your talking in - I hear you /128 is viable address for a tunnel and loopback - but its not really a global address mask.. Talk to your isp if you have problem with it, etc..

    Min viable prefix in ipv6 is /64 --- your the ipv6 is better than sliced bread guy around here.. You should know this ;)



  • Either way, I still can't ping the firewall address regardless of the prefix length. 53::1 works fine, fd53::1 doesn't work - and the 53::1 address works whether the prefix is /64 or /128. I've tested from Ubuntu, Windows, and Android, to rule out an OS related issue, but they're all a no-go.

    Is anyone able to test this from a interface IPv6 address set to Track WAN Prefix? That's the only different variable that I can see here. If not, I'll likely spin up a clean pfSense VM to see whether this is occurring on a fresh install. Is it possible that a plugin could be causing this, and if so, is there a consistent way to disable them without complete removal?



  • In case anyone encounters the same problem, I happened upon the solution by chance months later while configuring NTP on a multi-homed Ubuntu server and wanted to post it here for posterity's sake.

    My main LAN subnet obtains an IPv6 address via WAN DHCPv6 prefix tracking and advertises it via RA's. Because most of my other VLAN's use ULA's and eschew dynamic GUA's, I have the firewall set up with static IPv6 addresses which get advertised to each subnet. For consistency's sake, I wanted devices on my main LAN to obtain a ULA as well as a GUA, and had configured the RA's to include a ULA prefix of fd00:172:28:198::/64. Unfortunately, since pfSense doesn't currently support multiple IPv6 prefixes per tracked interface ([https://redmine.pfsense.org/issues/5999]) this means that the client ping was going out through it's ULA, hitting it's destination, and then being routed back through the firewall. Since the firewall didn't itself have a ULA on the LAN interface, it had no route back to the client, and the traffic was dropped. Removing the ULA prefix from the RA's on that interface resolved this issue.

    If anyone has a solution for allowing both static and dynamic IPv6 addresses on the same interface while avoiding the issues I encountered here, I'd appreciate the suggestions.


  • LAYER 8 Netgate

    Unfortunately, since pfSense doesn't currently support multiple IPv6 prefixes per tracked interface

    I am doing exactly this.

    DHCP6 serves the delegated prefix based on track interface settings:

    0_1550862519923_Screen Shot 2019-02-22 at 11.02.50 AM.png

    RA Serves BOTH the tracked interface prefix and the configured ULA:

    0_1550862549999_Screen Shot 2019-02-22 at 11.04.13 AM.png

    ETA: Forgot about the VIPs:

    0_1550867177689_Screen Shot 2019-02-22 REDACTED at 12.23.30 PM.png

    My Mac's LAN interface looks like this:

    vlan0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
    	options=3<RXCSUM,TXCSUM>
    	ether a8:60:b6:19:15:fe 
    	inet6 fe80::14ea:9daa:af44:6b3d%vlan0 prefixlen 64 secured scopeid 0xb 
    	inet6 2600:beef:cafe:ab01:1092:8d9e:537d:5207 prefixlen 64 autoconf secured 
    	inet6 fd8c:9857:66db:1:4c9:e132:ee:396a prefixlen 64 autoconf secured 
    	inet 192.168.223.6 netmask 0xffffff00 broadcast 192.168.223.255
    	inet6 2600:beef:cafe:ab01::145a prefixlen 64 dynamic 
    	inet6 2600:beef:cafe:ab01:edf8:e8af:4a5c:da72 prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:edf8:e8af:4a5c:da72 prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:b95b:1aa2:a178:32c9 prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:b95b:1aa2:a178:32c9 prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:b0f0:e7fb:d671:c1e prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:b0f0:e7fb:d671:c1e prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:91f5:785f:2283:21fc prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:91f5:785f:2283:21fc prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:b1d6:45a3:c20:1d15 prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:b1d6:45a3:c20:1d15 prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:4c01:bf7b:6079:299b prefixlen 64 deprecated autoconf temporary 
    	inet6 fd8c:9857:66db:1:4c01:bf7b:6079:299b prefixlen 64 deprecated autoconf temporary 
    	inet6 2600:beef:cafe:ab01:347d:50ed:27e3:2de9 prefixlen 64 autoconf temporary 
    	inet6 fd8c:9857:66db:1:347d:50ed:27e3:2de9 prefixlen 64 autoconf temporary 
    	nd6 options=201<PERFORMNUD,DAD>
    	vlan: 223 parent interface: en0
    	media: autoselect (1000baseT <full-duplex>)
    	status: active
    

    The IPv6 stack in the Mac sources from a global address to global destinations and a ULA address to ULA destinations.

    $ ping6 -c1 www.google.com
    PING6(56=40+8+8 bytes) 2600:beef:cafe:ab01:347d:50ed:27e3:2de9 --> 2607:f8b0:4005:80b::2004
    16 bytes from 2607:f8b0:4005:80b::2004, icmp_seq=0 hlim=55 time=21.027 ms
    
    $ ping6 -c1 nc.mydomain.com
    PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> fd8c:9857:66db:4::c0a8:e012
    16 bytes from fd8c:9857:66db:4::c0a8:e012, icmp_seq=0 hlim=63 time=0.695 ms
    

    I use split DNS to give ULA addresses to inside queries. It's been working great. And now my inside network no longer breaks in the unlikely event Cox changes my PD.

    One thing I have not tried is pinging ULA from an interface with only global addresses and pinging global from interfaces with only ULA addresses. A lot of the burden there is on the client's stack. Considering, with a /56, I don't have any problem assigning GUA and ULA to every IPv6 interface it has never come up. Let me try something real quick:

    From LAN to the LAN GUA

    $ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab01:208:a2ff:fe0a:593f
    PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab01:208:a2ff:fe0a:593f
    16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=0 hlim=64 time=0.352 ms
    16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=1 hlim=64 time=0.391 ms
    16 bytes from 2600:beef:cafe:ab01:208:a2ff:fe0a:593f, icmp_seq=2 hlim=64 time=0.325 ms
    

    From LAN to another interface's GUA:

    $ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab02:208:a2ff:fe0a:593f
    PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab02:208:a2ff:fe0a:593f
    16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=0 hlim=64 time=0.455 ms
    16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=1 hlim=64 time=0.292 ms
    16 bytes from 2600:beef:cafe:ab02:208:a2ff:fe0a:593f, icmp_seq=2 hlim=64 time=0.298 ms
    ^C
    --- 2600:beef:cafe:ab02:208:a2ff:fe0a:593f ping6 statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/std-dev = 0.292/0.348/0.455/0.075 ms
    

    From LAN to a host on DMZ GUA:

    $ ping6 -S fd8c:9857:66db:1:347d:50ed:27e3:2de9 2600:beef:cafe:ab04::c0a8:e012
    PING6(56=40+8+8 bytes) fd8c:9857:66db:1:347d:50ed:27e3:2de9 --> 2600:beef:cafe:ab04::c0a8:e012
    16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=0 hlim=63 time=0.775 ms
    16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=1 hlim=63 time=0.612 ms
    16 bytes from 2600:beef:cafe:ab04::c0a8:e012, icmp_seq=2 hlim=63 time=0.629 ms
    ^C
    --- 2600:beef:cafe:ab04::c0a8:e012 ping6 statistics ---
    3 packets transmitted, 3 packets received, 0.0% packet loss
    round-trip min/avg/max/std-dev = 0.612/0.672/0.775/0.073 ms
    

    So it looks like it's all working as expected. There are GUA and ULA addresses on the configured interfaces.

    Rebooting based on the trouble description in that redmine:

    Yeah. it all just works here. GUA and ULA /64s on all interfaces configured as such. All the above pings still work. Seems fine.

    For completeness, this is how I did one of the linux servers (nextcloud - accessible on GUA from the outside and ULA from the inside):

    auto ens18
    iface ens18 inet static
    address 192.168.224.18
    netmask 255.255.255.0
    gateway 192.168.224.1
    dns-search example.com admin.example.com
    dns-nameservers 192.168.223.1
    
    iface ens18 inet6 auto
       privext 2
       accept-ra 2
       up ip -6 addr add 2600:beef:cafe:ab04::192.168.224.18/64 dev $IFACE label $IFACE:0
       down ip -6 addr del 2600:beef:cafe:ab04::192.168.224.18/64 dev $IFACE label $IFACE:0
       up ip -6 addr add fd8c:9857:66db:4::192.168.224.18/64 dev $IFACE label $IFACE:0
       down ip -6 addr del fd8c:9857:66db:4::192.168.224.18/64 dev $IFACE label $IFACE:0
    

    (Please do not use ULA prefix fd8c:9857:66db::/48. Please generate your own.)


Log in to reply