CARP WAN interfaces generating NBT UDP broadcast loop/storm?
-
So I've noticed that my CARP fail-over setup acts a bit weird.
When there's almost 0 traffic on the LAN, there are 100-150Mbps of synchronous traffic on the WAN interface of both boxes.
I went in and did a TCPdump and saw that each MAC was sending out tons of NBT UDP PACKET (UDP 137) broadcasts:
14:29:24.838189 00:25:90:c0:XX:XX (oui Unknown) > Broadcast, ethertype IPv4 (0x0800), length 110: 169.254.65.83.netbios-ns > 169.254.255.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
14:29:24.838192 00:25:90:c0:YY:YY (oui Unknown) > Broadcast, ethertype IPv4 (0x0800), length 110: 169.254.65.83.netbios-ns > 169.254.255.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCASTWhere the MAC addresses listed are the hardware (not CARP) addresses of each firewall.
I have BOGON and RFC1918 blocking enabled on the interfaces.
I've run TCPdump on the LAN side (I have several LAN/OPT internal interfaces) and have not seen matching traffic.
I'm wondering if there's a process (or kernel "feature") that is amplifying a single netbios-ns query into a storm.
If I reboot either (usually the secondary) firewall the storm goes away.
The network is two ports on one switch going to the firewalls, a 10GB uplink to another switch and then 1GB Fiber to my internet provider. The 3 interfaces (firewalls and internet) are isolated in their own VLAN, with no other ports enabled.
Any advise is greatly appreciated.
Thanks
-S -
It's not CARP, but probably an incorrect local rule with a source of * and a gateway set.
See https://redmine.pfsense.org/issues/2073
-
Thanks Jimp!
I think that's probably what's happened.
In my efforts to move groups of users over to the new WAN one group at a time, I was using wildcard source rules to route traffic for certain VLANS out the new WAN.
I've added proper rules to block the APIP space to hopefully avoid the situation in the future.
Thanks again!
-
we are seeing the same issue here. whenever we place a notebook (osx) on the loacal subnet with a non local ip (as this happens when users had their device at home with a static ip and forgot to switch back to dhcp) and if we then activate shared folder - after the notebook sent out the first nbt broadcast with the "wrong" ip, hell breaks loose betwwen the two firewall wan interfaces. we circumvent this now by strict checking for local ips, i.e. we block everything on lan interface which is not src lan subnet, but i'm curious why such broadcast packets getting ping-ponged and mass-amplified by the firewalls. any clues?
-
Because you tried to policy route broadcast traffic. See the earlier response from me on the ticket.
-
could you perhaps explain why the system behaves this way? why does it only happen with broadcasts from 'wrong' ips on the subnet ? j
-
Which version of pfsense are you running?
-
The symptoms you are describing is exactly what occurred at one of our customers.
What version PFSense were you running?Our customer site was running 2.2.6
