CARP : duplicated traffic
-
Hello,
I face a strange problem using CARP (2 members) / pfSense 2.1.4 / Netgate C2758 : duplicated trafic.
I read https://forum.pfsense.org/index.php?topic=61770.0;prev_next=next#new : nothing is virtualized here, and the interfaces are assigned in correct order.
Duplicated traffic is not limited to ping. SSH for example is also duplicated. I face the same problem with IPv4 and IPv6. Not all traffic is duplated, only part of it.
For example with ping / IPv4. In this case, some replies are duplicated :
- echo request is not duplicated : "WAN IP –> pfSense (CARP + NAT1:1) --> RFC1918 LAN IP"
- pfSense duplicates the echo reply (==> represents duplicated trafic) : "RFC1918 LAN IP --> pfSense (CARP + NAT1:1) ==> external IP"
Each CARP member sends an echo reply. The strange thing is that both echo replies have the same source MAC address : the MAC address of the WAN physical interface of the primary CARP member. So one of the firewalls sends frames using as source address the MAC address of a physical interface belonging to another physical device !
Another example :
- echo request is duplicated : "RFC1918 LAN IP –> pfSense (CARP + NAT1:1) ==> external IP"
- so the target responds to both requests and two replies reach the ping source (pfsense does not duplicate again on the way back) : "WAN IP ==> pfSense (CARP + NAT1:1) ==> RFC1918 LAN IP"
It's obviously not a NAT issue, because I face the same problem using IPv6 without any NAT.
Does anyone have an idea of how to solve this issue ?
Romain
-
Hi rdr,
Are you using some form of bonding/teaming of NICs on the firewalls?
Edit: I guess lagg is the most appropriate term in this context.
-
Hi vindenesen !
No I don't use any kind of NIC Teaming / LAG / bonding etc. Each firewall has 3 utilized physical interfaces : WAN, LAN and CARP SYNC.
Romain
-
@rdr:
The strange thing is that both echo replies have the same source MAC address : the MAC address of the WAN physical interface of the primary CARP member. So one of the firewalls sends frames using as source address the MAC address of a physical interface belonging to another physical device !
If two interfaces work in CARP mode they get a virtual MAC address as well as a virtual IP.
Do you have both interfaces LAN and WAN in CARP mode? And does CARP work correctly, so that one pfSense box shows all CARPs working as master and the other as backup?
-
Hello !
Indeed both WAN and LAN interfaces are in CARP mode.
All VIPs are master on one firewall / backup on the other.I will have a look as soon as I can to the actual utilization of the virtual mac addresses.
Romain
-
I noticed another interresting thing, regarding gateway monitoring. The backup CARP member sends two echo requests to the gateway IP address : one using it's own IP and mac addresses, and one using IP and mac addresses of the physical WAN interface of the primary CARP member.
But this does not happen for every pings… only a subset of the primary CARP member gateway monitoring pings are duplicated on the secondary.About the virtual mac addresses : traffic reaching the CARP members has a virtual mac address destination. Traffic going out of a CARP member after having traversed it has a physical mal address source.
-
Hi !
problem is solved : it was a bonding issue, but not regarding pfSense.
The pfSense CARP cluster is linked to a couple of Juniper Virtual Chassis switches without any bonding.
But a cluster of Juniper SRX routers is also linked to the same Virtual Chassis using "Chassis Cluster Redundant Ethernet Interface Link Aggregation Groups".
And my mistake was I forgot to configure LAGs on the Virtual Chassis Interfaces linked to the SRX cluster reth LAG.http://www.juniper.net/documentation/en_US/junos12.1x46/topics/example/chassis-cluster-redundant-ethernet-interface-link-aggregation-group-configuring-cli.html : "For aggregation to take place, the switch used to connect the nodes in the cluster must enable IEEE 802.3ad link aggregation for the redundant Ethernet interface physical child links on each node."
Romain