pFsense not responding to multicast arp whohas

johnpoz

@Ofloo said in pFsense not responding to multicast arp whohas:

who has fc03:dead:1eaf:cafe::1

So this is pfsense address? And your saying its not answering?

Or are you saying the traffic is not flowing through a bridge on pfsense to some other client that has that address?

Ofloo

@johnpoz the client is bhyve bridged, however it reaches the interface on pFsense, which is a vlan on a lagg0 and that isn't answering. Maybe shouldn't of mentioned the bridge as it only confuses things. pFsense has no bridge it's LACP lagg0.100 with vlan. And it has the IPv6 on that interface but it's not answering. The call of who has.

johnpoz

@Ofloo said in pFsense not responding to multicast arp whohas:

Maybe shouldn't of mentioned the bridge as it only confuses

No you clearly should and need to "mention" it..So pfsense has a leg in this bridge you setup on bhyve? I have no direct experience with that hypervisor - but plenty of other ones, proxmox, esxi, hyper-v, virtualbox, etc.

But have seen in the past things on setup in a virtual switch/port/bridge having issues talking to each other, etc.

A drawing exactly have setup and you sniffed on pfsense and saw the discovery? To a multicast address - its possible rules on pfsense are not letting that through?

Ofloo

@johnpoz

Pfsense receives the traffic but doesn't respond to who-has ofcourse it also has a pseudo vlan interface lagg0.100 but is not bridged ! That's what I thought as well but pfctl -d doesn't improve replying, the only thing that did work at some point but not always is that I ping from pfsense to lets say server 1 once the ndp table is filled all traffic flows. reboot the vm ndp table is empty again and same store all over.

johnpoz

@Ofloo is pfsense physical - from that drawing can't tell if physical or virtual.

When pfsense pings something.. And then you can ping pfsense back, what source IP is it pinging from?

I tired to duplicate - I don't have a lagg setup.. Set same ULA address as you, and not seeing any problems it answering neighbor solicitation

If me I would remove the lagg from the equation.. Does it work then?

Ofloo

@johnpoz network interfaces lagg0 are real cards no virtual cards

jimp

Could be this:

https://redmine.pfsense.org/issues/13423

In which case there won't be a fix until the next release since it requires a change in the kernel.

Ofloo

@jimp it could be that issue not sure

21:03:12.076167 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:13.100224 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:14.096234 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:15.121082 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:16.116240 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:17.117124 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:18.198316 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32
21:03:19.196136 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > ff02::1:ff00:1: ICMP6, neighbor solicitation, who has fc03:dead:1eaf:cafe::1, length 32

This is what i see so no responds from pfsense, this looks exactly the same as the bug you referring too

now when i ping from pfsense to the server/client

ping6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae
PING6(56=40+8+8 bytes) fc03:dead:1eaf:cafe::1 --> fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=0 hlim=64 time=0.565 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=1 hlim=64 time=0.198 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=2 hlim=64 time=0.222 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=3 hlim=64 time=0.186 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=4 hlim=64 time=0.227 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=5 hlim=64 time=0.198 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=6 hlim=64 time=0.196 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=7 hlim=64 time=0.212 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=8 hlim=64 time=0.213 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=9 hlim=64 time=0.220 ms
16 bytes from fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae, icmp_seq=10 hlim=64 time=0.222 ms

and after that !!! it works visa versa

21:06:38.446559 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > fc03:dead:1eaf:cafe::1: ICMP6, echo request, seq 0, length 16
21:06:38.446628 IP6 fc03:dead:1eaf:cafe::1 > fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae: ICMP6, echo reply, seq 0, length 16
21:06:39.455884 IP6 fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae > fc03:dead:1eaf:cafe::1: ICMP6, echo request, seq 1, length 16
21:06:39.455913 IP6 fc03:dead:1eaf:cafe::1 > fc03:dead:1eaf:cafe:5a9c:fcff:fe07:6cae: ICMP6, echo reply, seq 1, length 16

jimp

Sounds like that issue, then.

We'll have a fix for that in Plus 23.09. It's working well there now for us on lab systems that hit the problem before.

No ETA on whenever the next CE release might be.

Ofloo

@jimp I understand, that this is the case however I think it's sad that CE is neglected in this way.