HA+CARP for pfSense on VMware ESXi and promiscuous mode issue

mauro.tridici

I've set-up 2 pfSense in CARP on two ESXi nodes and, following the available documentation, I've created another port-group only for pfSense with promiscuos mode enabled.

Problem is that now pfSense receives all traffic that goes to the Virtual Switch, so LAN interface it's "flooded" by unwanted traffic and, more important, from within pfSense it's possible to sniff all the traffic that goes through the virtual switch.

Is there another way to be able to have CARP working on ESXi without promiscuous mode?
Or, alternatively, can I create a firewall rule to make pfSense ignore the unwanted traffic? If yes, could you please show me an example rule?

Thank you very much,
Mauro

mauro.tridici

Dear Expert Users,

I would simplify my request as follows:

"can I create a firewall rule to make pfSense ignore the unwanted traffic caused by promiscous mode"? If yes, could you please show me an example rule?

Thank you,
Mauro

KOM

@mauro-tridici Traffic between two clients on the same network doesn't hit pfSense at all. You need to enable promiscuous mode on all hypervisors if you want to use pfSense in an HA config.

Are you seeing an actual performance problem?

mauro.tridici

HI @KOM ,

many thanks for your answer.
I would say that HA is working very well and I'm not seeing any performance problem at this moment.
Anyway, since promiscuous mode has been enabled on every virtual switch of both the hypervisors I'm using, I see that, in PFSENSE GUI -> Status -> TRAFFIC GRAPH, also the traffic of other clients on the same network is captured.

In my mind, for example, I would like to see only traffic generated by pfsense master and slave WAN IPs in "Traffic Graph -> WAN section".
In other words, I would like to see only WAN IPs and CARP VIP traffic and not all the traffic generated by the neighbours.

But I don't know if I can do that.
Sorry, but I'm a newbie.
Thank you,
Mauro

KOM

@mauro-tridici My previous post was a little confusing. With promiscuous mode enabled, pfSense will technically "see" traffic not meant for its MAC address, but it won't process it.

I run pfSense under VMware ESXi and I do not see this same effect that you see. Intra-LAN traffic doesn't show on the traffic graph.

mauro.tridici

@kom thank you for your clarification and feedback.
Can I ask you what is the version of VMware ESXi and pfSense you are using?
Did you activate only promiscous mode on VMware portgroups? Or "MAC address changes" and "Forged trasmits" are also enabled?

Thank you in advance for your patience.
Mauro

mauro.tridici

@KOM I would add that, in my scenario, the WAN dedicated vSwitch has the following options state:

WAN_vSwitch:
promiscuous mode disabled
mac address changes enabled
forged transmits enabled

I created two different portgroups:

WAN_PG_for_other_VM:
promiscuous mode disabled
mac address changes enabled
forged transmits enabled

WAN_PG_for_pfSense:
promiscuous mode enabled (override option has been checked)
mac address changes enabled
forged transmits enabled

I hope it can help to help me :)
Thank you,
Mauro

KOM

@mauro-tridici I'm running 6.7.0 and 6.5.0. All my vSwitches have all three security options enabled.

mauro.tridici

@kom I'm using VMware ESXi v.7 (latest update).
I really don't understand why I'm experiencing this strange behaviour.

May be the problem related to the pfsense version I'm using (the latest one - v.2.5.2 community edition). I think that promiscuous mode should detect neighbours traffic, maybe it's normal. But something is wrong in pfsense that shows neighbours traffic also in STATUS->TRAFFIC GRAPH-> WAN using LOCAL filter.

Anyway, thank you for the time you spent for me.