Updating MAC for a reserved IP problem

pfpv

Can someone please educate me on this or see if there is a bug? I am on 23.09.1, use ISC DHCP after a short trip to KEA and use IP reservations for all my devices and static ARP. My IP pool is from .250 to .253 and there is a long list of DHCP Static Mappings outside of this pool. The pool essentially is not used because I set "Enable Static ARP entries" and "Allow known clients from only this interface". For all clients "Create an ARP Table Static Entry for this MAC & IP Address pair" is checked. Pretty restrictive.

I installed a 2.5Gb NIC into one of my PCs and wanted to switch to it from the builtin gigabit NIC. I changed MAC in the static mapping entry to that of the new NIC, saved and plugged the cable into the new NIC. No connection, did not get an IP. I plugged the cable back into the old NIC and got an IP immediately.

I wanted to delete the DHCP lease for the old NIC but I didn't get this option in the lease table probably because the lease is static. I restarted the DCHP server and re-plugged the cable into the new NIC, no IP/connection. Plugged back into the old NIC and got IP. Then I went into the ARP table and deleted the ARP cache entry for the old NIC, re-plugged the cable to the new NIC, still no IP/connection. Once I plug the cable into the old NIC I get IP.

I rebooted pfSense and while it was rebooting I plugged the cable into the new NIC but still did not get an IP/connection. I even tried to set that same static IP to the new NIC in Windows but somehow the NIC still got that auto-configured 169.x.x.x IP. That part I did not understand.

At some point I rebooted pfSense again. I still could not get an IP for the new NIC. But once I plug the cable into the old NIC I immediately get the static reserved IP for that NIC ... but that old MAC is no longer listed anywhere in the pfSense settings. This part I don't understand. The new NIC MAC is tied to that IP. I struggled for about an hour trying various things to no avail. At some point I unchecked "Enable Static ARP entries" and set "Allow all clients", and unchecked "Create an ARP Table Static Entry for this MAC & IP Address pair" for this particular mapping. Even after pfSense reboot the new NIC did not get an IP but the old one did.

In the end I set a different IP to the new NIC MAC and it could finally connect. About 12 hr later I set the old IP to the new NIC MAC and the PC finally got it.

Is all of the above an expected behavior? I suspect it had something to do with static ARP that I understand poorly. Why didn't reboots help? Is the table saved and reloaded after reboots? Now, when writing this I thought I didn't try releasing the IP from the client itself. Maybe this would help? What is the best course of actions to update MAC for a static IP reservation?

msswift

What were the lease times? The 12hr timing suggests that it worked as intended once a 12hr lease expired. DHCP does keep its lease table between restarts and boots. There's a button at the bottom Status -> DHCP Leases (/status_dhcp_leases.php) which will show you the full contents of /var/dhcpd/var/db/dhcpd.leases. When leases are expired there is a delete button (trash can) on leases. Maybe also when a client is offline for a while, I can't remember. In a pinch you can edit the leases file by hand with Diagnostics -> Edit File, including just discarding all leases (obviously restart the server after an edit).

When you were troubleshooting, if the pfsense DCHP system log wasn't helpful, you could have used pfsense packet tracing (Diagnostics -> Packet Capture) on the interface with the new NIC that wasn't getting a lease and examined the trace in pfsense or (for nicer looking output) wireshark to see whether the new NIC was sending DHCPDISCOVER (more precisely, whether pfsense was getting it -- to see if your client was sending it, you'd run wireshark on the client), and whether pfsense was sending DHCPOFFER, and if so what exactly was being offered. Looks like the network was working (not a firewall or NIC config issue) because it eventually worked, but something was hanging there between DHCPDISCOVER and DHCPOFFER that was keeping the negotation from getting to DHCPACK, at least held it up long enough that your client defaulted to a 169.* APIPA address. Guessing further, I think in the logs you would have seen pfsense fail to DHCPOFFER to the new NIC because the static IP you told it to offer was already assigned to the old NIC in an unexpired lease. Whether static ARP comes into it or not, I don't know, don't have much experience with that on pfsense, but one thing to remember when messing with arps is that your client as well as pfsense has an arp cache just like a DNS cache, which may need to be cleared too.

pfpv

@msswift, thanks for your reply. The default lease time is set to 24hr but I believe renewal happens at half this time. I didn't know that lease table survives reboot. Do you think releasing the lease from the client before disconnecting would have helped avoid all this trouble? I thought about it only later.

pfpv

Can anyone advise me how I can replace the NIC in a device on my network and avoid the trouble I described in the OP? I'd like it to be assigned the same reserved IP. Would releasing the lease from the client help? What if I can't do it? For example, if I find a way to release the lease from Synology I will lose the connection to it and won't be able to properly shutdown it. I have another headless client that would present the same problem. Is it possible to clear whatever reservations in pfSense that would help?

johnpoz

@pfpv you can always delete your reservation. Or just change its mac, and then the new client would get the IP.

Sounds like he was playing with static arp

something to do with static ARP that I understand poorly.

Not sure why he was playing with that if he doesn't understand what it does. A static arp locks and IP to specific mac address. Doesn't really have anything to do with dhcp. Its an arp cache thing.

But you can easy just change the mac in an reservation to hand out that IP to a client. Or change the IP of the reservation so that client will get the new IP.

pfpv

@johnpoz said in Updating MAC for a reserved IP problem:

@pfpv you can always delete your reservation. Or just change its mac, and then the new client would get the IP.

I thought so but it didn't happen. I described the details in the OP. I thought I'd change the MAC in the reservation, replace the NIC and the client will get the same IP but it could not get any IP, the NIC was autoassigned a 169.x.x.x IP.

Sounds like he was playing with static arp

something to do with static ARP that I understand poorly.

Not sure why he was playing with that if he doesn't understand what it does. A static arp locks and IP to specific mac address. Doesn't really have anything to do with dhcp. Its an arp cache thing.

But you can easy just change the mac in an reservation to hand out that IP to a client. Or change the IP of the reservation so that client will get the new IP.
I was exactly "playing" but we can call it that way because I am not a networking expert. My goal was to secure my network, so all clients have only reserved IPs and if someone manages to plug something in any of my switches that device won't get access to the network. Also, certain clients have certain restrictions based on their IPs. If those clients spoof their MACs, which is done easily, they won't get access to the network. That's why I have "Allow known clients from only this interface" and "Enable Static ARP entries" enabled, and each client has "Create an ARP Table Static Entry for this MAC & IP Address pair" checked.

I've used this configuration for a couple of years. When I get a new client I add a reserved IP based on its MAC. Just plugging it in into a switch doesn't grant access to the network. A couple of places I worked at used the same strategy.

I had a problem only when I needed to replace NICs in some clients when upgrading to 2.5Gb. Since I did what you suggested ("you can easy just change the mac in an reservation to hand out that IP to a client") and that client couldn't get any IP after that I have a feeling that something doesn't work right.

Because you said I was "playing" I have a feeling that this configuration isn't used often.

Can you look into this please? I described the configuration above. I replace the MAC for a new one in the DHCP static mapping, replace the NIC in a client, boot it up and it's can't get an IP. I place the old NIC back into the client, I don't change the static mapping back to the old MAC, so the old MAC is nowhere entered in pfSense anymore, I even reboot pfSense but the old NIC still gets the IP. That doesn't seem right. Right?

I can change the IP, I had to do it and it worked but I wanted that client to get the same old IP because some clients have specific restrictions based on their IPs. Why couldn't I achieve that?

One idea was to release the lease from the client before disconnecting it. Would it help? I didn't think about it before. But on some headless clients I won't be able to do that because I won't be able to shut them down properly without a connection.

I just noticed a checkbox "Do not record a unique identifier (UID) in client lease data if present in the client DHCP request". It's not checked. The description says "the resulting server behavior violates the official DHCP specification'. Would this help maybe? Or something is not working right?

pfpv

Can anyone please recommend me how I can replace network interface cards and keep the same IPs and not get those clients locked out?

johnpoz

@pfpv don't use static arp.. With a static arp, your telling pfsense IP 1.2.3.4 is mac abc..

Even if you had out 1.2.3.4 to mac xyz, its not going to work because pfsense arp cache for 1.2.3.4 is abc.. So he is not going to be able to talk to 1.2.3.4 at mac xyz.

If you want new nic at mac def to get IP 1.2.3.4, then set that up as a reservation.. There you go - done.. You can either do that by deleting the old reservation and creating a new one, or just by editing the reservation.

You prob want to flush the arp cache on pfsense, after you have deleted the static arp setting.. Atleast the arp chache entry for 1.2.3.4 on mac abc..

pfpv

@johnpoz, thanks. I am using reservations, too. Static ARP is in addition to that.

Without static ARP how do I achieve what I described - connected unknown devices and those with spoofed MACs (or those with manually assigned IPs) don't get access to the network?

johnpoz

@pfpv here...

I have a reservation for my pc.. at this mac B0-4F-13-0B-FD-16

I changed the reservation to a different mac B0-4F-13-0B-EC-15

I changed my clients mac to B0-4F-13-0B-EC-15 and bobs your uncle new mac on the old IP..

I just did it this way to simulate what your doing your changing the mac that gets that IP via reservation by using a new nic... I don't have a nic handy to change to, so just changed the mac of my current one to simulate a different nic.

pfpv

@johnpoz, thanks again. I see from your screenshot that you also use static ARP. Did you flush the ARP cache? How? There is an ARP Table in the Diagnostics tab. But I believe when I tried to delete the entry for the NIC I had already removed from the network it wasn't there. But the new NIC didn't get a new IP anyway.

johnpoz

@pfpv said in Updating MAC for a reserved IP problem:

also use static ARP.

Where are you seeing that - I have a reservation, that is different than a static arp..

A static arp is in the arp table of pfsense that say ip 1.2.3.4 is mac ABC.. that has nothing to do with dhcp specifically... You can have pfsense set a static arp entry from there, but its completely different things.. And last time I looked there were issues with it sticking anyway.. Here is some advice - if you do not understand completely what some option does that is not checked by default - do not check it ;)

pfpv

@johnpoz, there are checkmarks under Static ARP in the first column in the tables you posted earlier. I have them too but once I uncheck the option "Create an ARP Table Static Entry for this MAC & IP Address pair" for a reservation the checkmark disappears. How come you have them when this option is not selected in your case?

Below is my example. For the first one the option is on, for the second is off.

johnpoz

@pfpv oh your correct, that was left over from another test about static arps not sticking.. Thanks for pointing that out.. I forgot to uncheck those when I was done testing..

Now when you update the reservation, and you are using static arps, it should update the static arp entry to your new mac.. But if that doesn't happen you could have a bad day.

edit:
Do you want static arp? Do you know what static arp is? Do you have a need/want for them.. Most likely not.. Which is why it default to not using them.. It would be special use case where you would want/need to set those.. If you do not fully understand what they are, I would uncheck doing them..

pfpv

@johnpoz, It doesn't happen. So, something is wrong in pfSense then.

Can you tell me how to flush the static ARP cache? Maybe it will help.

johnpoz

@pfpv - look in your arp table.. for the IP..

What does it show?

Make sure you just uncheck static arp in the dhcp settings... And then look at your arp table, does it show an expire time?

You can flush the whole cache with -d -a

[23.09.1-RELEASE][root@sg4860.home.arpa]/root: arp -d -a
10.1.1.128 (10.1.1.128) deleted
10.1.1.129 (10.1.1.129) deleted
10.1.1.1 (10.1.1.1) deleted
192.168.110.110 (192.168.110.110) deleted
192.168.6.101 (192.168.6.101) deleted
192.168.4.76 (192.168.4.76) deleted
192.168.4.77 (192.168.4.77) deleted
192.168.4.78 (192.168.4.78) deleted
192.168.4.79 (192.168.4.79) deleted
192.168.4.72 (192.168.4.72) deleted
192.168.4.201 (192.168.4.201) deleted
192.168.4.73 (192.168.4.73) deleted
192.168.4.203 (192.168.4.203) deleted
192.168.4.71 (192.168.4.71) deleted
192.168.4.65 (192.168.4.65) deleted
192.168.4.61 (192.168.4.61) deleted
192.168.4.62 (192.168.4.62) deleted
192.168.4.63 (192.168.4.63) deleted
192.168.4.217 (192.168.4.217) deleted
192.168.4.57 (192.168.4.57) deleted
192.168.4.89 (192.168.4.89) deleted
192.168.4.58 (192.168.4.58) deleted
192.168.4.59 (192.168.4.59) deleted
192.168.4.53 (192.168.4.53) deleted
192.168.4.214 (192.168.4.214) deleted
192.168.4.55 (192.168.4.55) deleted
192.168.4.80 (192.168.4.80) deleted
192.168.4.209 (192.168.4.209) deleted
192.168.4.81 (192.168.4.81) deleted
192.168.4.49 (192.168.4.49) deleted
192.168.4.50 (192.168.4.50) deleted
192.168.7.10 (192.168.7.10) deleted
192.168.7.5 (192.168.7.5) deleted
192.168.7.99 (192.168.7.99) deleted
192.168.7.3 (192.168.7.3) deleted
192.168.7.98 (192.168.7.98) deleted
192.168.7.97 (192.168.7.97) deleted
192.168.7.96 (192.168.7.96) deleted
192.168.7.95 (192.168.7.95) deleted
192.168.7.94 (192.168.7.94) deleted
192.168.200.10 (192.168.200.10) deleted
192.168.3.10 (192.168.3.10) deleted
192.168.3.99 (192.168.3.99) deleted
192.168.3.32 (192.168.3.32) deleted
192.168.2.203 (192.168.2.203) deleted
192.168.2.200 (192.168.2.200) deleted
192.168.2.13 (192.168.2.13) deleted
192.168.2.2 (192.168.2.2) deleted
192.168.2.3 (192.168.2.3) deleted
192.168.2.198 (192.168.2.198) deleted
192.168.2.4 (192.168.2.4) deleted
192.168.2.221 (192.168.2.221) deleted
192.168.2.50 (192.168.2.50) deleted
209.122.48.1 (209.122.48.1) deleted
192.168.9.99 (192.168.9.99) deleted
192.168.9.98 (192.168.9.98) deleted
192.168.9.100 (192.168.9.100) deleted
192.168.9.10 (192.168.9.10) deleted
[23.09.1-RELEASE][root@sg4860.home.arpa]/root:

Or you can delete specific IP arp, etc. But I would make sure your not using static arp via dhcp first

pfpv

@johnpoz said in Updating MAC for a reserved IP problem:

Do you want static arp? Do you know what static arp is? Do you have a need/want for them.. Most likely not.. Which is why it default to not using them.. It would be special use case where you would want/need to set those.. If you do not fully understand what they are, I would uncheck doing them..

I explained earlier why I wanted them. So, unknown devices, those with spoofed MACs and manually assigned IPs don't get access to my network.

But I suppose if I want to change NICs I need to disable static ARP. Would it be sufficient to uncheck "Create an ARP Table Static Entry for this MAC & IP Address pair" for the affected clients only and uncheck "Enable Static ARP entries" in the main DHCP server section? I think I tried that. I think I need to flash the ARP cache properly.

johnpoz

@pfpv said in Updating MAC for a reserved IP problem:

those with spoofed MACs and manually assigned IPs don't get access to my network.

taht is not how static arps work.. Did you setup dhcp to deny unknown macs? If not its just going to get an IP out of dhcp pool..

Sure that could prevent someone from using same IP as some other device - but that mostly would cause issues with duplicate IP addresses and break stuff for the other client, etc. and is normally noticed very quickly.. Who is going to be spoofing mac/IP address pairs on your network??

pfpv

@johnpoz, I did setup dhcp to deny unknown macs. But not all my devices are up all the time. And static IPs can be assigned on devices that are not reserved in my pfSense.

As to who is going to spoof MACs. I read that some IP cameras start changing their MACs trying to call home. Hopefully I don't have such cameras. I plan to wire POE cameras outside and I don't want someone unplug my camera and plug in their device instead. And last, my kids may learn how to spoof MACs in order to go around the restrictions, unlikely though.

But it's not the point. This is a good security system. I worked for a few universities and they did just that. If you want to plug a new device, give its MAC to the sysadmins. Otherwise, no access to the network. The system works in my case and the geek in me wants to use it even if I don't probably really need it.

pfpv

@johnpoz said in Updating MAC for a reserved IP problem:

Make sure you just uncheck static arp in the dhcp settings... And then look at your arp table, does it show an expire time?

It's strange. I haven't unchecked anything yet but many entries in the ARP table show "Expires in xxxx seconds" where xxxx is 1200 maximum and below. Some show "Permanent". I don't see any correlation why some would be expiring and some permanent. All should be permanent. Is there any explanation to this?

And thank you for explaining how to flush the ARP cache.