Pinging CARP - ICMP DUP reply
-
I get dupe replies when I ping the Virtual CARP IP (internal and external interfaces in my setup) while running pfSense virtualized in SmartOS without any Windows hosts present.
-
I have a similar problem and very similar setup - 3xESXi hosts, master and backup CARP pfsense firewalls
We have multiple subnets which all go through the firewall.
If I ping anything outside of the subnet which needs to go over the gateway (master pfsense firewall) then I get the DUP! issue when the master firewall is on the same ESXi host as the servers which are pinging to and from each other.I have set promiscuous on all port-groups/switches and the net.reverse… flag is set on all ESXi hosts and rebooted.
I cannot use your workaround of keeping the firewalls on a separate host as I need to distribute my VMs on all host.
Linux hosts are in one subnet and windows in another. I'm going to try this with linux in both to see if this could be a windows issue.
Any thoughts?
-
Afaik Net.ReversePathFwdCheckPromisc = 1 should only be necessary when running redundant uplinks from the vSwitch to a physical switch.
Either way, have you been able to obtain a packet capture from both cluster members, the windows server, and your client when the problem occurs?
That should help figuring out who is doing what when. -
Confirming via packet capture on each leg would be helpful. We've had a couple reports of this but no solid leads, it either clears itself up when other things (e.g. switched) are changed/upgraded or goes away before the source can be found.
One customer's packet captures suggested it was the request being duplicated, and pfSense was just responding to each request it saw.
-
Hi,
I can arrange packet captures today, I assume you need captures on the server sending the ping and on the end receiving the ping?
Thanks
-
Since I opened this thread months ago, we have gone through many iterations / escalations with pfSense support as well as VMware. For the record, we are strictly working with VMware ESX 5.1, not sure if the same symptoms / solution applies to other platforms / versions.
In short: It's a vSwitch / VDS limitation. It's a forward only switch, and the issue is around multiple uplinks. My network guys can provide a better analysis, however the solution was ensure that the switch any CARP traffic is connected to only has one uplink to the outside. Multiple uplinks (even in standby mode) resulted in packet duplication.
We now have a dedicated firewall ESXi cluster, running multiple instances of pfSesne (CARP) and respective uplinks groups only have one port defined. To provide redundancy we have > 1 ESXi node, HA / DRS enabled, with appropriate affinity groups keeping our firewall services highly available.
As a side note: CARP, VRRP, HSRP are similar in their origin, operation and implementation. We have a VRRP cluster (keepalived) running without any issues on the same vSwitch / VDS infrastructure that the problematic pfSense CARP cluster resulted in packet duplication. Perhaps there's something in CARP that can / should be tweaked?
-
Since I opened this thread months ago, we have gone through many iterations / escalations with pfSense support as well as VMware. For the record, we are strictly working with VMware ESX 5.1, not sure if the same symptoms / solution applies to other platforms / versions.
In short: It's a vSwitch / VDS limitation. It's a forward only switch, and the issue is around multiple uplinks. My network guys can provide a better analysis, however the solution was ensure that the switch any CARP traffic is connected to only has one uplink to the outside. Multiple uplinks (even in standby mode) resulted in packet duplication.
We now have a dedicated firewall ESXi cluster, running multiple instances of pfSesne (CARP) and respective uplinks groups only have one port defined. To provide redundancy we have > 1 ESXi node, HA / DRS enabled, with appropriate affinity groups keeping our firewall services highly available.
As a side note: CARP, VRRP, HSRP are similar in their origin, operation and implementation. We have a VRRP cluster (keepalived) running without any issues on the same vSwitch / VDS infrastructure that the problematic pfSense CARP cluster resulted in packet duplication. Perhaps there's something in CARP that can / should be tweaked?
This is interesting.
We have basically the same setup here, but use the multiple uplinks for bandwidth and failover purposes and hence need them. Do you have any more info from the network guys about this or perhaps someone can pinpoint the issue further so that we can solve it while keeping multiple uplinks? -
Since I opened this thread months ago, we have gone through many iterations / escalations with pfSense support as well as VMware. For the record, we are strictly working with VMware ESX 5.1, not sure if the same symptoms / solution applies to other platforms / versions.
In short: It's a vSwitch / VDS limitation. It's a forward only switch, and the issue is around multiple uplinks. My network guys can provide a better analysis, however the solution was ensure that the switch any CARP traffic is connected to only has one uplink to the outside. Multiple uplinks (even in standby mode) resulted in packet duplication.
We now have a dedicated firewall ESXi cluster, running multiple instances of pfSesne (CARP) and respective uplinks groups only have one port defined. To provide redundancy we have > 1 ESXi node, HA / DRS enabled, with appropriate affinity groups keeping our firewall services highly available.
As a side note: CARP, VRRP, HSRP are similar in their origin, operation and implementation. We have a VRRP cluster (keepalived) running without any issues on the same vSwitch / VDS infrastructure that the problematic pfSense CARP cluster resulted in packet duplication. Perhaps there's something in CARP that can / should be tweaked?
I'm having a similar issue, but can't afford to have a dedicated cluster for the pfSense instances. So I configured the dvSwitch ports used by the pfSenses so that they use only one uplink (and only on these ports, since we need uplink redundancy for the other vms), and the duplicate pings immediately stopped. So far so good !
-
Nevermind, DUPs are back…
-
I know this is an old thread. I got similar issue, would like to share how i fixed this. I just disable ipv4 and ipv6 in the host nic that causes the dup icmp.
-
On vphere edit your distributed port group then 'teaming and failover' and on failover order :
- Active Uplinks : uplink 1
- Standby uplinks : uplink 2
-
Hi,
I had the same problem, using VIP + CARP, followed all best practices for pfSense and still got DUP echo reply.
Thanks to camembert, problem for me was uplink 1 and 2 set to "active", after uplink 2 was set to standy, it worked fine. -
Worked fine to me, I changed the teaming only in Port Group used by CARP (not distributed vswitch)!
Thanks friends!
-
You can have both uplinks active if you enable this advanced host parameter: Net.ReversePathFwdCheckPromisc (see pfSense Troubleshooting guide)
By the way I discovered today that if your VM has "VM DirectPath IO" enabled it bypass this parameter and you will have duplicated packet again.
