ARP Tables...Most Static Addresses are No Longer Permanent

johnpoz

@ARAMP1 I agree it is odd, if shows static in the setting then it should be static - see my edit go in and click save on one of the entries that isn't showing static, but you have set too.. Does it switch to perm?

I just unchecked that one I had set as test and it went back to non static in the arp table. Maybe some weirdness in the upgrade process?

edit: maybe the device grabbed wrong IP and not its reservation - but if static entry was set then it wouldn't work, etc. But yeah it is odd if your setting shows it should create a static arp pair, and its not set.

edit2: So I looked through my reservations to see if I had previous set any to static - and I see the same thing.. This was previous set to create a static pair..

But as see its not - I recently updated to 23.05.1 so maybe something in the update the static pairing gets lost??

edit3: Well I do believe you have run into some sort of oddness for sure.. I went in and just hit save on the entry I had marked as static, and it didn't update the arp table. But unchecked static, saved, and then rechecked static and save and it switched to perm in the arp table..

Might want to file a redmine about it..

Let me look in redmine if I see anything related.

edit4: This seems relevant for sure

https://redmine.pfsense.org/issues/14374

Would have to reboot to see if happens then.. I could test on one of my VMs.. running 2.7, but there is a comment 3 days ago about seeing this in 2.7 as well..

ARAMP1

@johnpoz I went in and unclicked a few ARP static entry boxes, saved it, then went back in and clicked them. They're now showing up as permanent.

ARP Table 1.jpg

johnpoz

@ARAMP1 yeah see my edit this seems to be this bug

https://redmine.pfsense.org/issues/14374

I don't really feel like rebooting my box ;) I am working remote today.. But I could test it on my 2.7 vm..

ARAMP1

@johnpoz Just read through the link. Thanks.

I've rebooted a few times since upgrade and it didn't change anything.

johnpoz

@ARAMP1 so when you reboot the perm entries go away - that seems like what the bug is talking about.

ARAMP1

Ah...Yeah, just confirmed that's the case. Rebooted and all the permanent entries are gone (except for the VLAN interfaces themselves).

Well, thanks again @johnpoz for your help.

johnpoz

@ARAMP1 you might want to just chime in on that redmine, that you too are seeing it.. Or I can make a comment if you want, and link to this thread.

I made a comment and linked to this thread.

zkhcohen

I can also validate that Regression #14374 is present on my 3.7.0 installation.

I noticed that simply opening and saving each DHCP entry with static ARP assigned (without making any changes) will resolve the issue for that specific entry. Rebooting the DHCP service has no impact.

Another interesting note - when I create an XML backup of my pfSense config and inspect it, all the proper static ARP entries are present. In other words, the configuration is present, it's just not being initialized properly on reboot. Note that I haven't had a chance to test the secondary symptom -- the eventual erasure of static ARP entries for inactive devices.

Setting up an hourly cron job / shellcmd which sets "arp -s <ip> <mac>" for each entry is the best solution until a patch has been issued.

zkhcohen

Here's a script which leverages python3.11 to search for and set static arp entries:

Step 1. Create a shell script called "arp.sh" in the /usr/local/etc/rc.d/ directory:

#!/bin/sh

python3.11 - <<EOF
import xml.etree.ElementTree as ET
import subprocess

# Load the XML file
tree = ET.parse("/conf/config.xml")
root = tree.getroot()

# Extract the IP and MAC addresses using list comprehension
results = [{'mac': staticmap.find("mac").text, 'ip': staticmap.find("ipaddr").text}
           for staticmap in root.iter("staticmap")
           if staticmap.find("arp_table_static_entry") is not None]

# Execute the arp commands
for result in results:
    command = f"arp -s \"{result['ip']}\" \"{result['mac']}\""

    print("Command:", command)

    completed_process = subprocess.run(command, shell=True, executable="/bin/sh", capture_output=True, text=True)

    if completed_process.returncode == 0:
        print("Command executed successfully.")
        print("Output:", completed_process.stdout)
    else:
        print("Command execution failed.")
        print("Error:", completed_process.stderr)
EOF

Step 2. Run: chmod +x /usr/local/etc/rc.d/arp.sh
Step 3. In the pfSense UI, navigate to Services > Cron and create the following job:

0 	* 	* 	* 	* 	root 	/bin/sh /usr/local/etc/rc.d/arp.sh

Step 4. To execute the command manually, run /bin/sh /usr/local/etc/rc.d/arp.sh. Upon checking the pfSense UI Diagnostics > ARP Table, you should see that all of the entries are now "Permanent".

johnpoz

@zkhcohen so your script looks to what is in the arp table and makes them all static - ie perm? That would be a horrible idea

zkhcohen

@johnpoz No. It looks for a pre-existing permanent static arp mapping in the DHCP config and then reapplies it via the CLI.

zkhcohen

@johnpoz

A patch was just issued (extremely quickly, I might add), and I can confirm that it fixes the issue in my environment:

https://redmine.pfsense.org/projects/pfsense/repository/2/revisions/5082edf92795fe8266be49905fe4f07eb682449d

ARAMP1

The patch does not appear to work on 2.7.0.

johnpoz

@ARAMP1 I just tested this on my 2.7 vm and seems to be working..

So I created a static arp and it was there.

I then rebooted and it was gone.

I then applied the patch, and rebooted again and now its there.

ARAMP1

Must have been user error....I got it working. After a couple restarts and wondering what I did wrong, I reinstalled the patch and everything is working.

Appreciate the help, gents.

32G3LiQxu8

I’m seeing this behavior in 23.09. ARP table shows dynamic (expires in <time>) ARP entry instead of static (permanent).

If I uncheck, “Create an ARP Table Static Entry for this MAC & IP Address pair” and then save.

Then recheck “Create an ARP Table Static Entry for this MAC & IP Address pair” and then save.

Then go back to ARP table, almost all the dynamic entries are flipped to static (permanent)

Anyone else seeing this?

johnpoz

@32G3LiQxu8 there seems to be an issue with when clients update their lease - there is this thread.

https://forum.netgate.com/topic/184155/static-arp-in-dhcp-overwritten

And this redmine

https://redmine.pfsense.org/issues/14970

So yeah if it has flipped to dynamic, a resave of the dhcp page where you set it as static will turn it back to perm, but it seems when client updates its lease again it will go back to dynamic.. See the thread linked to above, where you can duplicate it doing it by just having your client release and renew its lease.

From the redmine it seems to be correct in the upcoming 24.03 and 2.8 CE releases

"This is not reproducible on the latest builds of 2.8.0 and 24.03"

32G3LiQxu8

@johnpoz Thank you for the information. I will wait for a fix.