Carp is flooding my network with VRRP packets (Yes, that doesn't make sense)
-
I have CARP setup on all of my interfaces, including my lan interface. I run 2 PFSense boxes in HA.
I was looking at wireshark on my desktop computer.
I noticed that my desktop was receiving 1 packet directed at 224.0.0.18 per second from the primary PFSense router
The packet is identified as VRRP, which is an entirely different, but similar protocol to CARP
How can I stop flooding my network with this traffic, and make it so the routers communicate directly?
They have a direct link to each-other for pfsync
see attached image for capture of the wireshark data
-
CARP shows up in Wireshark as VRRP.
-
CARP shows up in Wireshark as VRRP.
That's good to know.
Now the question is: how do I get it to do unicast
-
I wouldn't call one per second a "flood."
You don't. CARP/VRRP are multicast to 224.0.0.18.
If you routinely use CARP instead of VRRP you can tell Wireshark to decode it as CARP.
Highlight a VRRP packet and Analyze > Decode As
Tell it to decode protocol 112 as CARP instead.
 -
Yeah 1pps is not a flood ;) But when you say your network - if you don't want to see these multicast packets elsewhere on the network. Then just filter it at your switch so only the ports connected to your pfsense interfaces pass this multicast traffic.
Then your client say on your lan would not see this multicast traffic - and wouldn't be flooded with that 1pps ;)
-
Be careful though. Any other switches on the same layer 2 will need to learn the CARP MAC from those advertisements.
My suggestion: don't hassle it.
-
Huh? Why would the downstream switches need to see these multicast carp packets?
The interface that is the master will be the one that would respond to the ARP for that IP with the virtual mac that is used by the carp setup.
Maybe I need to go refresh my memory on this, normally use hsrp.. But some downstream switch doesn't need to see the multicast protocol 112 packets that I am aware of..
-
That is the only way the CARP MAC address is learned by a bridge - seeing the CARP advertisement. They will not otherwise know where to send packets with that MAC address as the destination. So they would all be flooded.
The ARP response comes from the interface address with the CARP MAC inside the packet, not in the frame. Been looking at this pretty closely lately.
-
Still not getting you…
We have this.. (attached)
So filter so multicast only on the 2 ports connecting the 2 members of the carp.. This multicast is not sent down the trunk, or to any other ports on the top level switch.
The bottom switch will have in its arp table the mac of the 192.168.1.1 carp address virtual mac in its arp table... Same with the top switch he will learn that mac when someone looking for it.. He should only have the virtual mac on the port connected to the carp interface that is master.. Since this would be the only one answer arp for the 192.168.1.1 address.
The switch is not going to be looking in any carp 112 traffic for mac addresses to "learn" That would all be done via your typical arp..
-
A bridge does not look at ARP. It sees a frame to ff:ff:ff:ff:ff:ff and floods it out all ports except the one it was received on. It has no idea whether it is ARP, DHCP, etc (unless a higher-layer snooping is in effect). If your bridges are pulling MAC addresses out of ARP replies and adding them to the mac-address-table on that port then they probably shouldn't be doing what they are doing…
This is an ARP reply for a CARP VIP:
Ethernet II, Src: d2:a1:91:e7:2b:cb, Dst: 00:08:a2:09:99:ae
Destination: 00:08:a2:09:99:ae
Source: d2:a1:91:e7:2b:cb
Type: ARP (0x0806)
Address Resolution Protocol (reply)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: reply (2)
Sender MAC address: 00:00:5e:00:01:e4
Sender IP address: 172.25.228.17
Target MAC address: 00:08:a2:09:99:ae
Target IP address: 172.25.228.1Note the ethernet source address in the frame that will be added to the mac-address-table on the receiving port is the interface address (d2:a1:91:e7:2b:cb). The CARP MAC (00:00:5e:00:01:e4) will be added to the ARP table on 172.25.228.1 for layer 3 address 172.25.228.17. But the CARP MAC will not be added to the bridge's mac-address-table since it was not the source address of the received frame.
There is nothing there that will cause the CARP MAC to be added to a bridge's mac-address-table. That information only comes from the CARP advertisements:
Ethernet II, Src: 00:00:5e:00:01:e4, Dst: 01:00:5e:00:00:12
Destination: 01:00:5e:00:00:12
Source: 00:00:5e:00:01:e4
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 172.25.228.19, Dst: 224.0.0.18
0100 …. = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x10 (DSCP: Unknown, ECN: Not-ECT)
Total Length: 56
Identification: 0x0000 (0)
Flags: 0x02 (Don't Fragment)
Fragment offset: 0
Time to live: 255
Protocol: VRRP (112)
Header checksum: 0x0b06 [validation disabled]
[Header checksum status: Unverified]
Source: 172.25.228.19
Destination: 224.0.0.18
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Common Address Redundancy Protocol
Version 2, Packet type 1 (Advertisement)
Virtual Host ID: 228
Advertisment Skew: 100
Auth Len: 7
Demotion indicator: 0
Adver Int: 1
Checksum: 0xc368 [correct]
Counter: 7823498306054967289
HMAC: 0a33b43b1d6659e415c5786881438c6a0ca6bc5a@RFC3768:
Note: VRRP packets are transmitted with the virtual router MAC
address as the source MAC address to ensure that learning bridges
correctly determine the LAN segment the virtual router is
attached to. -
Ahhh!!!! Doh!! You are right the arp reply will have a source of the physical nic sending it on the frame, but the info for the vip will be contained in the reply!
Thanks! Brain fart on my end for sure ;) Your packet captures very helpful in knocking out the stupid fog ;)
But even with downstream switches you can filter the those multicast packets from end point ports, and only allow it to the switches and the carp interfaces if he is worried about the "flooding" ;)
-
Certainly. As long as the switches see the CARP MAC so they can stick it in the table for the ISL/trunk port. Else they will flood it every time they see it. My hunch is stuff would break.
-
Hi
One Packet per Second is normal?
i have 16 Packets per second(in one vlan so if i have 20 VLANS, i have 20*16 Packets only vor VRRP) and little netgear switches have problem to handle this traffic.
Is this a failure in my network or normal and i must filter it?
Thank you
-
i have 16 Packets per second(in one vlan so if i have 20 VLANS, i have 20*16 Packets only vor VRRP) and little netgear switches have problem to handle this traffic.
Ok, so that would be 320 pps. Even a cheap Netgear switch is rated at 1,448,000 pps. To repeat Derelict, don't hassle it.
-
If they are all different CARP VIPs then that is how your network is designed so you need switches that can deal with it.
I have a hard time believing that any switch (even a Netgear) can't handle 320 CARP advertisements per second.
-
The switch doesn't really care about the frames being CARP.
If it's a "dumb" switch it only looks at the lowest bit of the first byte of the destination-mac: the multicast-bit.
–> a dumb switch can't actually distinguish a broadcast-frame from a multicast-frame (broadcast is just a special multicast-address)The more intelligent switches can analyze the multicast frames a bit more in depth (usually on L3 with IGMP), but as long as it's still a L2 multicast frame it doesn't really care if it's CARP or something else.
-
If they are all different CARP VIPs then that is how your network is designed so you need switches that can deal with it.
No, and this is the only reason why i am so worried, i have multiple Packets (same VRID) in each VLAN per second and i read somthing about pfsense CARP send only one packet per second.
Can it be that this happening because i have a LCAP Lagg(1 pfsense has 4 cable connected on two Stack Switches, so both pfsense are connected on two switches with 8 cables)?Thank you
-
Post a packet capture of this please.
advbase 1 advskew 0, the default on a primary node, will result in about 1.003 seconds between advertisements PER CARP VIP.
If you are seeing more than that you are probably seeing some incorrect flooding or looping in your switching gear.
If you only capture CARP in Diagnostics > Packet Capture CARP will be decoded. If you need to subsequently vire a capture just select CARP again and click View Capture.
Wireshark can be coerced to decode CARP instead of VRRP. Right-click a VRRP packet and select Decode As. Set protocol 112 to CARP there.
-
Special, pfsense gives me one packet per second but if i capture with a client i have multiple packets per second.
I add both capture as attachment, i hope this helps. -
You don't happen to run 12 CARP IPs on the same ID do you?
