ARP reports bogons

johnpoz

@stephenw10 exactly... If you want to have some isolated networks that pfsense never sees sure ok.. Multihoming to get to these different networks seems odd, when you have a router right there at your disposal.. If you don't want the networks talking to each other simple enough to just not let them via firewall rules.

But if your going to do it that way - they for sure should be actually isolated at L2, and sure wouldn't suggest you use use overlapping IPs on them, etc.

deanfourie

Just out of curiosity,

What is a arp watch "flip-flop" event?

These are happening a lot. I have confirmed that they are not IP conflicts and that there is enough space in the DHCP pool.

Thanks

stephenw10

How does that actually appear in the logs?

I imagine it's a device that switches to a different IP/MAC and then immediately back but I don't think I've seen that myself.

deanfourie

@stephenw10 so this is still happening.

Multiple ARP replies for my gateway IP (pfSense) for a host MAC that is not on my network.

I am getting like 100 notifications for flip-flop for a this MAC, and I look complete LAN and internet access entirely.

How can I trace these ARP packets and where they are coming from?

01:31:51.651811 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:51.652989 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:31:53.136724 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:53.801919 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:31:54.690516 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:54.691952 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:31:56.145759 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:56.805442 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:31:57.723844 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:57.724819 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:31:59.160216 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:31:59.800668 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46
01:32:03.744058 ARP, Request who-has 172.16.101.2 tell 172.16.101.1, length 28
01:32:03.745421 ARP, Reply 172.16.101.1 is-at 50:06:ab:93:0d:c1, length 46

stephenw10

So 50:06:ab:93:0d:c1 is not a device on your network?
That's a Cisco MAC address.

Where did you run that pcap? Look at it in more details and check the source and destination MACs.

Those do not look like matched requests and replies. Unclear why you're not seeing the request for 172.16.101.1 there though.

johnpoz

@stephenw10 yeah those replies look like gratuitous arps.. from say the svi on the cisco device.

Those are clearly not replies to what was asked 101.1 is asking for mac of 101.2

stephenw10

The timing is suspect. But you're probably only seeing half the traffic because of where it's captured.

It seems very likely you have some address conflict somewhere. You just need to find and resolve it.

deanfourie

@stephenw10 so I just installed a new Cisco switch, with a truck port trunking the VLANs.

I do remember configuring a VLAN ip address on 172.16.101.1 as I was having issues getting the trunk up but even after unplugging that backbone they are still rolling in.

Plus, once I got the trunk talking, I issues a default interface GigabitEther x/x

So you think this is coming from the switch itself?

stephenw10

Probably. Is that the same IP pfSense is using in that subnet? Or any other device? That would certainly be a conflict if so.

deanfourie

@stephenw10 no, I know not to address a anything on that address obviously.

Except for that VLAN interface during testing but certainly nothing else is statically assigned on that IP.

The behavior does look a lot like a conflict, flip-floping between the two Macs fighting for that IP.

However, also a arp spoof would have the same behavior.

Just a flood of ARP packets advertising 172.16.101.1 is at xx:xx:xx:xx:xx, Any Mac could be here and this would cause issues, no?

stephenw10

Where did you run that pcap?

ARP requests are broadcast. Replies I'd expect to go back to the requester directly.
Those are probably gratuitous ARPs announcing the IP/MAC.

What is 172.16.101.2? The switch isn't seeing a response from it either by the looks of things since it keeps ARPing for it.

Steve

deanfourie

@stephenw10 OK, I think it was the switch VLAN which still had its IP on 172.16.101.1.

Since disabling that, all is good now.

How can I secure ARP more?
Like locking the ARP table on each device and disabling dynamic ARP updates? Is that possible?

Thank

stephenw10

I've never tried but you could add static ARP entries for everything on all devices. I can only imagine it being a complete nightmare though! You'd be chasing connectivity issues forever. Hard to recommend.