DHCP Server Assigns Same Address to Multiple Hosts

[jesse_cureton]

When multiple clients make DHCP requests almost simultaneously, the DHCP server will assign them all the same address. This leads to IP address collisions on the network. I can reliably reproduce this when spawning virtual machines programmatically, since they all come up and request an address at nearly the same time.

Here's a DHCP log from pfSense that shows what happens with two VMs - VM A in green, and VM B in blue. Log is reverse chronological.

VM A comes up, completes a lease from discover->release and goes down for reboot at 15:29:57. VM B does the same a couple seconds later at 15:29:59. Both are given the same address since it was released when A went down for reboot. A then comes back up from the reboot at 15:31:10, completes a negotiation and receives an ACK from pfSense for 192.168.2.183. A couple seconds later, B comes up, completes a negotiation, except dhcpd offers the same 192.168.2.183 address it just ACKed for A!

I've confirmed this occurs reliably up to 3 VMs simultaneously, but suspect it will be true at higher numbers as well. Not really sure where to go from here. The time granularity on the pfSense logs is pretty lacking, and a packet capture on a separate machine seems to miss all the pfSense DHCP responses.

My Setup:

[JKnott]

Why are you hiding MAC addresses?

[jesse_cureton]

Why are you hiding MAC addresses?

Mostly because I found it easier to read/parse with only the last octet visible, and the hidden info shouldn't be directly related to the problem. Happy to grab another log with them unhidden if its useful information

[jesse_cureton]

Found a solution.. I got caught up on why the last DCHPDISCOVER logged the hostname of A with the MAC of B.

This ended up being because of the client identifier in the DHCP request. The requests still had the proper, unique hostnames, but the Ubuntu template image I was deploying the VMs from actually send /etc/machine-id as the client ID. Since this was defined in the template, the DHCP server was identifying the machines as the same image with different MACs. This is the default behavior for Ubuntu 18.04 onward.

This blog post pointed out that zeroing out that file in the template (truncate -s 0 /etc/machine-id) will ensure a new ID is created when the cloned template boots. Alternatively, enabling the Ignore client identifiers option in pfSense's DHCP server settings will accomplish the same result.