Multicast not leaving PFSense VM on ESX (vCloud in promiscious mode)

bbruun

I have a VMware ESX vCloud environment with two identically speced PFSense 2.5.0 VM's running.

I've had our hosting provider enable promiscious mode on the NICs so they can use CARP.

I have several VM's with keepalived on several LAN's and the multicasts from these arrive at the two PFSense servers just fine.

The problem I have, is that even though the PFSense in its filter logs says "pass,out" for 224.0.0.18 then there are no other VM's on the network that can see these messages.

Both VM's are configured with VMXNET3 NICs and have the VMware tools package installed.

Is this something that anyone else have observed ? Is the multicast messages sent from PFSense special/different from keepalived's VRRP packages?

Here are the output from the two PFSense servers showing they say they send the 224.0.0.18 multicast package, but nothing is registered anywhere on the any VMs

From firewall 1

Apr 14 16:46:39 gct-fw-001 filterlog[20235]: 19,,,1000000202,vmx1,match,pass,out,4,0xe0,,255,0,0,none,112,carp,56,10.100.10.2,224.0.0.18,advertise,255,28,2,240,1

From firewall 2

Apr 14 14:46:39 gct-fw-002 filterlog[14928]: 53,,,1000000202,vmx1,match,pass,out,4,0xe0,,255,0,0,DF,112,carp,56,10.100.10.3,224.0.0.18,advertise,255,28,2,100,1

Neither of them have any incoming multicast packages from the other but from keepalived there are many eg

Seen on firewall 1

Apr 14 16:46:39 gct-fw-001 filterlog[20235]: 19,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1625,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1
Apr 14 16:46:40 gct-fw-001 filterlog[20235]: 19,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1626,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1

Seen on firewall 2

Apr 14 14:46:39 gct-fw-002 filterlog[14928]: 53,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1625,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1
Apr 14 14:46:39 gct-fw-002 filterlog[14928]: 53,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1625,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1
Apr 14 14:46:40 gct-fw-002 filterlog[14928]: 53,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1626,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1
Apr 14 14:46:40 gct-fw-002 filterlog[14928]: 53,,,1000000202,vmx1,match,pass,in,4,0xc0,,255,1626,0,none,112,carp,40,10.100.10.5,224.0.0.18,advertise,255,10,2,121,1

(I've tried to get log entries as close to eachother as possible on both firewalls)

bbruun

@bbruun I've had the "MAC Address Changes" security setting disabled in the vCloud environment. Multicast packets are still not leaving the PFSense network cards.

Any input besides changing to physical boxes which won't be possible, is much appreciated.

bbruun

Problem isolated and solved

Working with a hosting provider and not having access to the underlying configuration layer means things get lost in translation.

The problem is/was Forged Transmits in the ESX environment that needed to be disabled so the CARP IP on the PFSense can create multiple MAC addresses and send/receive on these.

First paragraph here says it, but not having access and poor communication with hosting provider makes it difficult to debug by one self.
https://docs.netgate.com/pfsense/en/latest/troubleshooting/high-availability-virtual.html?highlight=vmware