Very strange DHCP and DHCP6 issue with Apple devices since upgrading WiFi APs; roaming clients sometimes lose connectivity due to DHCP failure
-
This likely isn't a pfSense issue (at least I don't think it is) but I am hoping with all the smart and knowledgable people here someone might have some ideas about this, or even have run into (and solved) this issue.
I have a fairly complex home network but it works very well and performs excellently. I am experiencing one significant annoyance recently, which only started after I changed my WiFi setup.
Key points about my network:
-
Router is NetGate 6100 running software version 24.03.
-
Core network is wired 10 GbE (NetGear XS512EM switch) and 1 GbE (TP-Link SG3452X).
-
NetGate 6100 provides DHCP and DHCP6/RA services. The DHCP backend in use is ISC DHCP.
-
Most devices use DHCP[6] with static address mappings.
Originally my WiFi network was provided by 3x Apple AirPort Extreme WiFi 5 APs (all running in bridge mode not router mode). The backhaul was wired 1 GbE. All APs used the same SSID, encryption and password etc. With this setup, WiFi coverage was good and performance adequate but not exceptional. Mobile devices (in our case primarily iPhones, iPads and Apple Watches) could roam around the house and would switch APs pretty much seamlessly.
I recently replaced the 3x Apple AirPort Extreme APs with a NetGear Orbi 970 WiFi 7 mesh system (running the latest available firmware). I have a router unit and a satellite unit (so 2 APs) and the system is configured in AP mode not router mode (so there are no routing, dhcp, dns or firewall functions provided, they just act as a mesh AP). The backhaul now is wired 10 GbE. This works really well (apart from the issue I am about to describe). WiFi coverage is better than before and WiFi performance is much better. However, Apple devices roaming between the two APs seem to experience a strange, but intermittent, issue...
Sometimes when a device (most usually an iPhone) moves from a location where it is connected to the main ('router') AP to a location where it is connected to the secondary (satellite) AP, it loses connectivity. It is associated to the new AP but the issue is that it loses its IPv4 and IPv6 addresses. It seems to get stuck in a state where it is trying, but failing, to get IP addresses (via DHCP[6]) and eventually assigns itself an auto configured (169) address, which of course is of no use. This issue affects all our iPhones (all recent models running the very latest iOS version) and occasionally also our iPads. Other devices seem unaffected, though to be fair we don't have many other devices which 'roam'. Note that the issue never occurs when a device 'roams' from the satellite AP to the main AP. I have thoroughly tested the 10 GbE wired backhaul and it is spotless. If there was an issue there I would certainly notice other symptoms.
I did some packet capture and managed to catch an occurrence of this issue. From the packet capture it looks to me like iOS is not responding correctly during the DHCP exchange with the NetGate.
What I see is the following (NetGate is 10.0.200.1, iPhone static mapping is 10.0.200.75):
iOS: DHCPDISCOVER -> broadcast
NetGate: DHCPOFFER -> iOS (offers 10.0.200.75)
iOS: ARP -> router(MAC address) who has 10.0.200.1? Tell 10.0.200.75
NetGate: ARP 10.0.200.1 is at <router_LAN_MAC_Address>
<variable delay of up to 10 seconds>
iOS: ARP -> router(MAC address) who has 10.0.200.1? Tell 10.0.200.75
NetGate: ARP 10.0.200.1 is at <router_LAN_MAC_Address>
<another delay>
iOS: ARP -> router(MAC address) who has 10.0.200.1? Tell 10.0.200.75
NetGate: ARP 10.0.200.1 is at <router_LAN_MAC_Address>
<another delay>
iOS: ARP -> router(MAC address) who has 10.0.200.1? Tell 10.0.200.75
NetGate: ARP 10.0.200.1 is at <router_LAN_MAC_Address>It is almost like iOS is not correctly processing the ARP response from the router, though that seems extremely unlikely. It just sits in this state for ages repeating the ARP request (and getting a valid response) and then eventually goes back to the DHCPDISCOVER step and everything repeats.
If, while in this state, I manually connect the iPhone to a different WiFi network (totally different SSID hosted on different APs with a totally different backend network and DHCP server) then it connects to that straight away and gets (totally different) IPv4 and IPv6 addresses. If I then switch it back to to the main network the DHCP sequence then goes as expected, the device accepts the offered address (via DHCPREQUEST), the server confirms it (DHCPACK) and everything is happy.
To me this looks like an iOS issue perhaps, but if so it is strange how it has been exposed by the switch of WiFi AP hardware.
If anyone has any theories or suggestions I'd love to hear them.
-
-
Hi, to me it seems like you have a handoff problem within your mesh. I interpret your packet capture as a symptom of that problem. I would start by reviewing your settings and seeing what your manufacturer has to say about handoff and how to best fine-tune it.
-
@Uglybrian Thanks. There are no settings relating to handoff in the WiFi setup. I will open a case with NetGear support.