Hosts with DHCP address cannot ping each other but static entries can…

burnsl · Feb 25, 2018, 5:42 PM

So get this…

I have subnets of:
192.168.1.0
192.168.2.0
192.168.3.0
192.168.5.0

Any device that gets a DHCP address in .2 cannot ping another device on the .2 other than the default gateway (192.168.2.1).
This issue ONLY appears on the .2 network.

Here is the behavior matrix:

If I ping FROM a .2 device that has a static IP to any other subnet, I get a reply.
If I ping FROM a .2 device that has a static IP to any other device in the .2 subnet, I get a reply.
If I ping FROM a .2 device that received it's IP from DHCP to any other subnet, I get a reply.
If I ping FROM any other subnet, to any .2 device that received it's IP from DHCP , I get a reply.

I have checked the settings for the DHCP server it's handing out the same gateway and mask as my statically addresses hosts.

what devious magic is at work here??!

Here is the DHCP setting page for the .2 network…

Grimson · Feb 25, 2018, 5:31 AM

@burnsl:

Any device that gets a DHCP address in .2 cannot ping another device on the .2 other than the default gateway (192.168.2.1).
This issue ONLY appears on the .2 network.

Not a pfSense problem, as it can't filter traffic within the same network: https://doc.pfsense.org/index.php/Firewall_Rule_Troubleshooting#Unfilterable_Traffic

burnsl · Feb 25, 2018, 5:42 AM

@Grimson:

Not a pfSense problem, as it can't filter traffic within the same network: https://doc.pfsense.org/index.php/Firewall_Rule_Troubleshooting#Unfilterable_Traffic

Correct - it's not filtering it.

However any host with a static address in the .2 network can ping other hosts.

Any host that gets it's IP from PFSENSE DHCP cannot.

This is driving me NUTS

burnsl · Feb 25, 2018, 6:19 AM

ISSUE RESOLVED

it was a screwed up ARP Table for the network (.2) inside the Wireless Access Point

JKnott · Feb 25, 2018, 2:55 PM

@burnsl:

ISSUE RESOLVED

it was a screwed up ARP Table for the network (.2) inside the Wireless Access Point

That doesn't make sense. An access point wouldn't maintain an arp table entry for anything that was not directly communicating with. For example, it would have one for a computer being used to configure it, but it wouldn't have for any device that it was simply passing traffic for. In that respect, an access point is no different than a switch. Like a switch, however, access points will maintain a table of MAC addresses, so that it knows where to forward frames, but that has nothing to do with IP addresses. The arp table is used to map an IP address to a MAC address. Also, arp table entries have a fairly short lifetime, typically 10-20 minutes, so even bad entries should soon disappear. The MAC address table, used for forwarding also has a fairly short lifetime.

burnsl · Feb 25, 2018, 5:46 PM

@JKnott:

That doesn't make sense. An access point wouldn't maintain an arp table entry for anything that was not directly communicating with. For example…

Welp, okay.
You're right. So, why then when I rebooted my AP, the problem went away?

Ubiquiti access points (The one here is a Ubiquiti AP-AC-HD), it has tons of capability like a guest portal, etc.
I have all that disabled because Pfsense does all this, but are you SURE about it not being related to some IP/MAC ARP-like table?

Why would rebooting the AP restore function?
Why would it only be DHCP addresses given by Pfsense and not affect hosts with static addresses?

Grimson · Feb 25, 2018, 6:00 PM

@burnsl:

Why would it only be DHCP addresses given by Pfsense and not affect hosts with static addresses?

I think there is a big misunderstanding here, both of the above are addresses the client gets via DHCP. The only difference is that a static lease has a fixed address while the non-static client gets a random address from the DHCP pool, but both work via DHCP. A real static address would be configured directly on the client, without any involvement of pfSense.

Note: In static leases you can override other DHCP options, which can have an effect. But then you should know what you did different there. This also wouldn't be fixed by rebooting the AP.

burnsl · Feb 25, 2018, 6:36 PM

@Grimson:

@burnsl:

Why would it only be DHCP addresses given by Pfsense and not affect hosts with static addresses?

I think there is a big misunderstanding here, both of the above are addresses the client gets via DHCP.

Nope, wait…
There definitely is a misunderstanding here.

I know how the addresses are given out and I understand the reservation/random thing you refer to however...

I built the whole thing here end to end and I MANUALLY assigned the static addresses to the hosts that I say have them.

So...
hosts with static addresses can ping other hosts in the same subnet.
hosts with DHCP addresses can NOT ping other hosts in the same subnet.

I simply rebooted the Ubiquiti AP and my wireless and wired devices on this subnet were able to see each other again.

How would rebooting the AP return full function to the network?

I changed nothing else.

Grimson · Feb 25, 2018, 6:42 PM

Did you enable the "Guest Policy" on the AP, if yes that will prevent wireless clients from talking to other wireless devices in the network. If not your best bet would be the Ubiquiti forums, as this is still not a pfSense issue.

JKnott · Feb 25, 2018, 6:58 PM

You're right. So, why then when I rebooted my AP, the problem went away?

All that indicates is there may have been a problem with the AP. However, it wouldn't have anything to do with ARP. Perhaps you can do some research on what ARP is about.

On a local network, all traffic is carried using the MAC address, not IP. So, when a device wants to communicate with another, it looks up the IP address in it's ARP cache and then uses the corresponding MAC address. If it isn't there, then an ARP request is sent out, asking who has this IP address. The device with that address then responds and it's MAC address is then added to the ARP cache. Communication can now take place, using that MAC address. As long as it's being used, that IP/MAC pair will remain in the cache. Then, at some point after the last communication, the entry will be removed from the cache. So, unless a device has recently had direct communication with, not through, the AP, there will be nothing in the APs ARP cache. When traffic passed through the AP, the MAC address will be cached only for the purposes of forwarding, but it will not contain any IP address, as it is not relevant to the forwarding process.

https://en.wikipedia.org/wiki/Address_Resolution_Protocol

burnsl · Feb 25, 2018, 7:02 PM

@Grimson:

Did you enable the "Guest Policy" on the AP, if yes that will prevent wireless clients from talking to other wireless devices in the network. If not your best bet would be the Ubiquiti forums, as this is still not a pfSense issue.

So, I agree (mostly) that this isn't a Pfsense issue, but since Pfsense is handing out DHCP addresses and those are the only ones that cannot talk to each other on the same subnet, then the jury is still out on this.

I also found the "Guest policy" issue on the Ubiquiti forums, but i don't have that checked.
Also that wouldn't have caused the issue to resolve after a reboot of the AP, as that setting would still be there.

This is still a mystery.

burnsl · Feb 25, 2018, 7:09 PM

@JKnott:

You're right. So, why then when I rebooted my AP, the problem went away?

All that indicates is there may have been a problem with the AP. However, it wouldn't have anything to do with ARP. Perhaps you can do some research on what ARP is about.

On a local network, all traffic is carried using the MAC address, not IP….

Yes I do know this, and I learned a while back that there isn't ARP in play here.

However, since the issue was related to IP addresses handed out by PFsense and manually installed static addresses had no issue, I question where the issue truly is.

Was the issue in the Pfsense server, and the reboot of the AP reset the link on that NIC causing something in the server to clear/reset?
Was the issue in the AP and the reboot caused something in the AP to clear/reset?

If this happens again I certainly will disable and re-enable the interface the AP is connected to first to check.

On the Pfsense FW I reset / cleared the State table often and the issue remained, so that wasn't it.

I have never seen anything treat DHCP addressed devices differently (in this way) than statically addressed devices.

Grimson · Feb 25, 2018, 7:24 PM

@burnsl:

Yes I do know this, and I learned a while back that there isn't ARP in play here.

However, since the issue was related to IP addresses handed out by PFsense and manually installed static addresses had no issue, I question where the issue truly is.

Well if the devices received a correct IP and netmask, as you already confirmed, pfSense is out of the picture it's involvement stops right there. Traffic of devices on the same network segment doesn't even reach pfSense, this is all handled by the switch/ap responsible for this network.

JKnott · Feb 25, 2018, 7:27 PM

I have never seen anything treat DHCP addressed devices differently (in this way) than statically addressed devices.

If all the devices have a valid IP address, it's not a DHCP issue. The only relevant DHCP info, for communicating over the local network, would be IP address and subnet mask. On computers, you can check the ARP cache to see if a device you're trying to contact is in it. As I've mentioned many times, in these forums, capturing traffic can help understand the problem. Packet Capture on pfSense will only capture traffic for or passes through pfSense, as well as broadcasts. So, to capture traffic you'd need Wireshark and a means, such as port mirroring with a managed switch, to capture the traffic. You can also run Wireshark on any computer that's involved in the problem. Without knowing what's on the wire, we're largely guessing, in the absence of other information.