Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pFsense not responding to multicast arp whohas

    Scheduled Pinned Locked Moved IPv6
    16 Posts 4 Posters 930 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • O
      Ofloo
      last edited by Ofloo

      I'm using pfsense 2.7.0 latest version, today I noticed strange behavior. I have a FreeBSD bhyve VM which, static IPv6 on the pFsense interface. The IPv6 addresses are assigned through SLAAC. It gets an IPv6 other clients on the same bridge can ping the gateway/static IPv6 on the interface on pFsense.

      When I tcpdump the traffic I see that the client is asking through multicast who has the IPv6 this also arrives on the pFsense interface, however no responds as the ping timeouts keep continuing. But for some reason whohas gets no answer. It does work for all the other clients.

      When I in turn then ping from pFsense too that very same clients I also get some ping timeouts but eventually after loosing a few packets everything works. This to me doesn't make any sense. Can anyone explain. The issue is clear it's pFsense that isn't doing what it is supposed to be doing it's not responding to the ARP requests ! Once the ARP requests got through network traffic worked.

      Clearly pFsense does not respond to whohas client does and once arp table is filled on the client side traffic flows ! So the client does respond while pFsense does not.

      It is vlan traffic though. But shouldn't matter one bit especially that it does work for all the other clients on that same bridge.

      No bridge filters are disabled as well, checked tcpdump on the pFsense side and only who-has arrived nothing left the interface to respond.

      net.link.bridge.pfil_local_phys: 0 -> 0
      net.link.bridge.pfil_member: 0 -> 0
      net.link.bridge.pfil_bridge: 0 -> 0
      net.link.bridge.pfil_onlyip: 0 -> 0
      

      But why is this? Or what could the cause be !?

      PS: it wasn't being filtered, tried adding a allow any rule to the top did not matter also all my filter deny rules have log flag enabled none where triggered.

      JKnottJ 1 Reply Last reply Reply Quote 0
      • JKnottJ
        JKnott @Ofloo
        last edited by JKnott

        @Ofloo

        There's no such thing as arp on IPv6. When a device needs the MAC address, it sends a Neighbor Solicit and expects a reply. Do you see the reply? Is the device being pinged capable of responding?

        BTW, pfSense has nothing to do with communications between devices on the same LAN.

        PfSense running on Qotom mini PC
        i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
        UniFi AC-Lite access point

        I haven't lost my mind. It's around here...somewhere...

        O 1 Reply Last reply Reply Quote 0
        • O
          Ofloo @JKnott
          last edited by Ofloo

          no there's no reply

          ifconfig lagg0.100
          lagg0.100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
          	description: PUBLIC
          	options=4600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
          	ether 0c:c4:7a:bd:29:84
          	inet6 fe80::ec4:7aff:febd:2984%lagg0.100 prefixlen 64 scopeid 0xe
          	inet6 fc03:dead:1eaf:cafe::1 prefixlen 64
          	inet 10.13.17.1 netmask 0xffffff00 broadcast 10.13.17.255
          	inet 10.10.10.1 netmask 0xffffffff broadcast 10.10.10.1
          	groups: vlan
          	vlan: 100 vlanproto: 802.1q vlanpcp: 0 parent interface: lagg0
          	media: Ethernet autoselect
          	status: active
          	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
          
          tcpdump -ni lagg0.100 host fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae
          tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
          listening on lagg0.100, link-type EN10MB (Ethernet), capture size 262144 bytes
          17:40:27.837061 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:28.832493 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:29.918182 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:30.921650 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:31.912520 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:32.950058 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          17:40:33.971711 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
          
          JKnottJ johnpozJ 2 Replies Last reply Reply Quote 0
          • JKnottJ
            JKnott @Ofloo
            last edited by

            @Ofloo said in pFsense not responding to multicast arp whohas:

            no there's no reply

            Again, if the devices are on the same LAN, as they appears to be, pfSense has nothing to do with it. Things like neighbor solicitation, just like arp on IPv4 work only on the local subnet and do not pass through pfSense.

            PfSense running on Qotom mini PC
            i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
            UniFi AC-Lite access point

            I haven't lost my mind. It's around here...somewhere...

            O 1 Reply Last reply Reply Quote 0
            • O
              Ofloo @JKnott
              last edited by Ofloo

              @JKnott it's not because it doesn't pass through pfsense that pfsense can't have like a bad network driver ! Then the issue is still a pfsense issue. This might seem hard to grasp but pfsense is the package not just the pf firewall !

              And if even that would be the case then pf wouldn't really be a pfsense thing they if you talk development they just package a gui around FreeBSD with pf firewall ! But still they package the os and drivers and so forth. So pFsense isn't just a firewall because that's not even what they develop. They develop a gui, yet on their site they claim to develop a firewall. Which is my point exactly it's the package.

              JKnottJ 1 Reply Last reply Reply Quote 0
              • JKnottJ
                JKnott @Ofloo
                last edited by

                @Ofloo

                You could completely remove pfSense from the LAN and your problem would very likely still be there. For traffic entirely on the local LAN, the entire responsibility of pfSense is the DNS & DHCP servers, along with SLAAC. It will not block arp or neighbor solicit.

                PfSense running on Qotom mini PC
                i5 CPU, 4 GB memory, 32 GB SSD & 4 Intel Gb Ethernet ports.
                UniFi AC-Lite access point

                I haven't lost my mind. It's around here...somewhere...

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator @Ofloo
                  last edited by johnpoz

                  @Ofloo said in pFsense not responding to multicast arp whohas:

                  who has fc03:dead:1eaf:cafe::1

                  So this is pfsense address? And your saying its not answering?

                  Or are you saying the traffic is not flowing through a bridge on pfsense to some other client that has that address?

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  O 1 Reply Last reply Reply Quote 0
                  • O
                    Ofloo @johnpoz
                    last edited by

                    @johnpoz the client is bhyve bridged, however it reaches the interface on pFsense, which is a vlan on a lagg0 and that isn't answering. Maybe shouldn't of mentioned the bridge as it only confuses things. pFsense has no bridge it's LACP lagg0.100 with vlan. And it has the IPv6 on that interface but it's not answering. The call of who has.

                    johnpozJ 1 Reply Last reply Reply Quote 0
                    • johnpozJ
                      johnpoz LAYER 8 Global Moderator @Ofloo
                      last edited by

                      @Ofloo said in pFsense not responding to multicast arp whohas:

                      Maybe shouldn't of mentioned the bridge as it only confuses

                      No you clearly should and need to "mention" it..So pfsense has a leg in this bridge you setup on bhyve? I have no direct experience with that hypervisor - but plenty of other ones, proxmox, esxi, hyper-v, virtualbox, etc.

                      But have seen in the past things on setup in a virtual switch/port/bridge having issues talking to each other, etc.

                      A drawing exactly have setup and you sniffed on pfsense and saw the discovery? To a multicast address - its possible rules on pfsense are not letting that through?

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                      O 1 Reply Last reply Reply Quote 0
                      • O
                        Ofloo @johnpoz
                        last edited by Ofloo

                        @johnpoz

                        e1d0a7df-5a49-4a8b-a89a-530de64240d7-afbeelding.png

                        Pfsense receives the traffic but doesn't respond to who-has ofcourse it also has a pseudo vlan interface lagg0.100 but is not bridged ! That's what I thought as well but pfctl -d doesn't improve replying, the only thing that did work at some point but not always is that I ping from pfsense to lets say server 1 once the ndp table is filled all traffic flows. reboot the vm ndp table is empty again and same store all over.

                        johnpozJ 1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator @Ofloo
                          last edited by

                          @Ofloo is pfsense physical - from that drawing can't tell if physical or virtual.

                          When pfsense pings something.. And then you can ping pfsense back, what source IP is it pinging from?

                          I tired to duplicate - I don't have a lagg setup.. Set same ULA address as you, and not seeing any problems it answering neighbor solicitation

                          ipv6.jpg

                          If me I would remove the lagg from the equation.. Does it work then?

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          O 1 Reply Last reply Reply Quote 0
                          • O
                            Ofloo @johnpoz
                            last edited by

                            @johnpoz network interfaces lagg0 are real cards no virtual cards

                            1 Reply Last reply Reply Quote 0
                            • jimpJ
                              jimp Rebel Alliance Developer Netgate
                              last edited by

                              Could be this:

                              https://redmine.pfsense.org/issues/13423

                              In which case there won't be a fix until the next release since it requires a change in the kernel.

                              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                              Need help fast? Netgate Global Support!

                              Do not Chat/PM for help!

                              O 1 Reply Last reply Reply Quote 0
                              • O
                                Ofloo @jimp
                                last edited by Ofloo

                                @jimp it could be that issue not sure

                                21:03:12.076167 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:13.100224 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:14.096234 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:15.121082 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:16.116240 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:17.117124 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:18.198316 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                21:03:19.196136 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
                                

                                This is what i see so no responds from pfsense, this looks exactly the same as the bug you referring too

                                now when i ping from pfsense to the server/client

                                ping6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae
                                PING6(56=40+8+8 bytes) fc03:dead:1eaf:cafe::1 --> fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=0 hlim=64 time=0.565 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=1 hlim=64 time=0.198 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=2 hlim=64 time=0.222 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=3 hlim=64 time=0.186 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=4 hlim=64 time=0.227 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=5 hlim=64 time=0.198 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=6 hlim=64 time=0.196 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=7 hlim=64 time=0.212 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=8 hlim=64 time=0.213 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=9 hlim=64 time=0.220 ms
                                16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=10 hlim=64 time=0.222 ms
                                

                                and after that !!! it works visa versa

                                21:06:38.446559 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > fc03:dead:1eaf:cafe::1: ICMP6, echo request, seq 0, length 16
                                21:06:38.446628 IP6 fc03:dead:1eaf:cafe::1 > fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae: ICMP6, echo reply, seq 0, length 16
                                21:06:39.455884 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > fc03:dead:1eaf:cafe::1: ICMP6, echo request, seq 1, length 16
                                21:06:39.455913 IP6 fc03:dead:1eaf:cafe::1 > fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae: ICMP6, echo reply, seq 1, length 16
                                
                                1 Reply Last reply Reply Quote 0
                                • jimpJ
                                  jimp Rebel Alliance Developer Netgate
                                  last edited by

                                  Sounds like that issue, then.

                                  We'll have a fix for that in Plus 23.09. It's working well there now for us on lab systems that hit the problem before.

                                  No ETA on whenever the next CE release might be.

                                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                  Need help fast? Netgate Global Support!

                                  Do not Chat/PM for help!

                                  O 1 Reply Last reply Reply Quote 0
                                  • O
                                    Ofloo @jimp
                                    last edited by

                                    @jimp I understand, that this is the case however I think it's sad that CE is neglected in this way.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.