SOLVED AND REVISED: 2.4.4-RELEASE arpresolve: can't allocate llinfo for $GATEWAY on interface0, DHCP MTU 576

[bfeitell]

I was having problems with interface brand/driver agnostic flapping on WAN using DHCP on RCN business "static" (see below).

KLUDGE:
I managed to workaround the problem by installing the shellcmd package and running 'arp -s $gateway-ip-address $gateway-macadress temp' in three shellcommand entries, early, regular, and on filter reload. This works, but the proper fix is now built into dhclient.

ACTUAL PROBLEM:
This problem relates to a line of bugs involving the setting of a too small MTU, in my case 576, by a bad DHCP server. This seems to break aspects of arp and/or aspects of pfSense's interface update scripts.
See: e.g. https://redmine.pfsense.org/issues/8507

SOLUTION:
In the "Lease Requirements and Requests" section for WAN DHCP in the field "Option modifiers" add the text without quotes: "supersede interface-mtu 0"
Cryptic references:
https://reviews.freebsd.org/D15484
https://svnweb.freebsd.org/base?view=revision&revision=334787

After applying this fix, I have removed my ARP kludge, the interface no longer flaps, there are no ARPRESOLVE LLINFO related errors, and I have a normal default MTU of 1500. This might make a good candidate for explicit inclusion in the documentation, as divining the solution is not intuitive, or otherwise well documented.

Other notes on symptoms. My buddy had complained that certain web sites which had been very snappy and quick to load under 2.4.3_p1 were completely unavailable or balky after the upgrade to 2.4.4-RELEASE. After noticing the very short MTU of 576, I considered whether packet fragmentation might be an issue. I usually run systems with "IP Do-Not-Fragment compatibility" enabled, and disabling this option brought back proper connectivity to the balky web sites. I have since re-enabled the option, as the MTU is now a healthy 1500.

KEYWORDS: WAN GATEWAY ARP ARPRESOLVE ALLOCATE LLINFO DHCP MTU 576 SUPERSEDE INTERFACE_MTU

-- original post below
Original Title: "Is there a proper way to add a static arp pair for the default gateway? Flapping WAN problem. (2.4.3_p1 works and 2.4.4-RELEASE is BROKEN)"

I am having problems with a WAN connection flapping on any adjustment of settings. The system in question worked fine under 2.4.3_p1, but is broken under 2.4.4-RELEASE The failure is accompanied by errors like:

arpresolve: can't allocate llinfo for $GATEWAY on interface0

The DHCP service from this ISP (RCN) is odd. It is sold as static and involves the provisioning of a defined seven address /29 subnet giving a "static" gateway that never changes, a "static" broadcast address that never changes, and five consecutive IP addresses available for hosts. Allowing traffic to RFC_1918 addresses is required for RCN DHCP to function, and block bogons (which used to cause trouble with DHCP in pfSense) is also disabled on the WAN. The wan has been tested with Intel (em), Realtek (re), and Intel (igb) interfaces, and the problem is hardware agnostic.

Having read what I can on the errors, I see the possibility of UDP checksum problems, and have disabled "Hardware Checksum Offloading" in advanced settings.

I have also explored adding -rxcsum and -txcsum to the ifconfig statement in /usr/local/sbin/pfSense-dhclient-script

I have also set and selected as default, a static default gateway that gets ignored when the WAN comes up on boot. The problem occurs with or without this static gateway.

In regular FreeBSD I would try the static arp pair strategy as described in the following post, but under pfSense I have no functioning rc.conf file to work with. Where can I place these settings in pfSense?

https://unix.stackexchange.com/questions/432146/freebsd-static-arp-entry-when-link-cycles