FreeNAS kernel arp attempts to modify permanent arp entry
-
I'm a little bit stumped on this one.
Setup -- pfSense 2.4.4 release p3 -- pfSense with IP address of 10.0.1.1
DNS Ranges 10.0.1.100-10.0.1.150, 10.0.1.165-10.0.1.200FreeNAS 11.2-U6
Within FreeNAS - FreeBSD jail using dhcp -- IP address of jail 10.0.1.15810.0.1.158 reserved as static IP address within pfSense with the freeBSD jail setup to obtain IP address through DHCP (however somehow when choosing this option a VNET is created).
Once jail is booted the arp table within the jail appears as such:
? (10.0.1.158) at 02:ff:60:ba:b5:82 on epair0b permanent [ethernet]Littered within the logs of the jail are the following:
freenas kernel: arp: ce:c0:9f:c5:9a:b1 attempts to modify permanent entry for 10.0.1.1 on epair0bI get this message repeated every 10-30 seconds
ce:c0:9f:c5:9a:b1 is the MAC address of the pfSense box
As an attempt to suppress the errors I tried adding the pfSense MAC address permanently into the arp cache of the jail.
sudo arp -s 10.0.1.1 ce:c0:9f:c5:9a:b1This however doesn't change the log message of:
freenas kernel: arp: ce:c0:9f:c5:9a:b1 attempts to modify permanent entry for 10.0.1.1 on
epair0bIs there anything I can do to prevent this from happening. Logs are filling up super quick.
-
I think you have it backward. There is currently a static ARP entry on FreeNAS for 10.0.1.1, pointing to something that isn't pfSense.
-
Thanks for getting back to me -- I really appreciate since I'm not sure exactly how to troubleshoot this issue.
OK I troubleshooted a few things and changed my available DHCP range from 10.0.1.100-10.0.1.150, 10.0.1.165-10.0.1.195
I'm no networking expert in creation of virtual LANs and jails. FreeNAS creates a VNET for each jail then associates it with an epair interface that is bridged to the main adapter.
On the main FreeNAS system I have these interfaces:
igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=2400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6> ether 0c:c4:7a:84:a5:94 hwaddr 0c:c4:7a:84:a5:94 inet 10.0.1.197 netmask 0xffffff00 broadcast 10.0.1.255 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect (1000baseT <full-duplex>) status: active igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 0c:c4:7a:84:a5:95 hwaddr 0c:c4:7a:84:a5:95 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect status: no carrier lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> groups: lo bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: iohyve-bridge-igb0 ether 02:f6:c7:64:02:00 nd6 options=1<PERFORMNUD> groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: vnet0:12 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 10 priority 128 path cost 2000 member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 9 priority 128 path cost 2000000 member: epair2a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 8 priority 128 path cost 2000 member: epair1a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 7 priority 128 path cost 2000 member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 6 priority 128 path cost 2000 member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 5 priority 128 path cost 2000000 member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 1 priority 128 path cost 20000 tap1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: iohyve-arch-on-FreeNAS-- options=80000<LINKSTATE> ether 00:bd:8d:56:f8:01 hwaddr 00:bd:8d:56:f8:01 nd6 options=1<PERFORMNUD> media: Ethernet autoselect status: active groups: tap Opened by PID 3915 epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 02:a8:d0:00:06:0a hwaddr 02:a8:d0:00:06:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair epair1a: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 02:a8:d0:00:07:0a hwaddr 02:a8:d0:00:07:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair epair2a: flags=8942<BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether 02:a8:d0:00:08:0a hwaddr 02:a8:d0:00:08:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: Attached to UbuntuMC options=80000<LINKSTATE> ether 00:bd:52:cc:f8:00 hwaddr 00:bd:52:cc:f8:00 nd6 options=1<PERFORMNUD> media: Ethernet autoselect status: active groups: tap Opened by PID 8225 vnet0:12: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: associated with jail: nextcloud as nic: epair0b options=8<VLAN_MTU> ether 02:ff:60:ba:b5:81 hwaddr 02:a8:d0:00:0a:0a nd6 options=1<PERFORMNUD> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active groups: epair
In this example I'm having issues with the 10.0.1.158 jail or the jail on net0:12 on epair0b -- meaning i get the error in the logs if the jail is running -- however when the jail is turned off, I don't get the error in the logs:
freenas kernel: arp: ce:c0:9f:c5:9a:b1 attempts to modify permanent entry for 10.0.1.1 on epair0b
In terms of arp entries -- (I've got three systems potentially I'm dealing with -- pfSense, FreeNas, Jail within FreeNas)
Arp caches are as follows:
pfSense:/root: arp -a ? (10.0.1.171) at d0:e7:82:bc:b7:0b on xn1 expires in 1195 seconds [ethernet] ? (10.0.1.10) at 00:e0:67:13:5b:7f on xn1 expires in 985 seconds [ethernet] ? (10.0.1.169) at 40:9f:38:26:e6:af on xn1 expires in 1002 seconds [ethernet] ? (10.0.1.9) at 74:83:c2:1e:ad:c5 on xn1 expires in 977 seconds [ethernet] ? (10.0.1.168) at 38:c9:86:1b:2b:2f on xn1 expires in 1071 seconds [ethernet] ? (10.0.1.174) at 38:8b:59:0e:b8:43 on xn1 expires in 974 seconds [ethernet] ? (10.0.1.173) at 8c:85:90:35:cc:e7 on xn1 expires in 1071 seconds [ethernet] ? (10.0.1.172) at 70:ef:00:b1:5a:92 on xn1 expires in 1186 seconds [ethernet] ? (10.0.1.3) at b4:fb:e4:b0:af:f8 on xn1 expires in 1160 seconds [ethernet] ? (10.0.1.226) at 18:b4:30:9b:b9:56 on xn1 expires in 1006 seconds [ethernet] ? (10.0.1.193) at 94:e9:6a:00:81:67 on xn1 expires in 1111 seconds [ethernet] pfSense.xxxxx.com (10.0.1.1) at ce:c0:9f:c5:9a:b1 on xn1 permanent [ethernet] ? (10.0.1.160) at 00:a0:98:c6:2b:6d on xn1 expires in 1185 seconds [ethernet] ? (10.0.1.224) at 18:b4:30:30:68:33 on xn1 expires in 977 seconds [ethernet] ? (10.0.1.167) at 5c:ad:cf:c9:dc:6c on xn1 expires in 1071 seconds [ethernet] ? (10.0.1.71) at d4:81:ca:5e:f7:58 on xn1 expires in 976 seconds [ethernet] ? (10.0.1.197) at 0c:c4:7a:84:a5:94 on xn1 permanent [ethernet] ? (10.0.1.228) at 18:b4:30:98:b3:e3 on xn1 expires in 977 seconds [ethernet] ? (10.0.1.132) at 64:52:99:7d:6b:60 on xn1 expires in 972 seconds [ethernet] ? (10.0.1.91) at c8:63:f1:cf:6a:c6 on xn1 expires in 1199 seconds [ethernet] ? (10.0.1.186) at 9c:ae:d3:1e:31:d2 on xn1 expires in 1178 seconds [ethernet] ? (10.0.1.121) at d4:81:ca:5e:de:04 on xn1 expires in 1022 seconds [ethernet] ? (10.0.1.120) at 00:5b:94:d8:c5:0f on xn1 expires in 1158 seconds [ethernet] ? (10.0.1.95) at 00:a0:98:4e:65:cf on xn1 expires in 990 seconds [ethernet] ? (10.0.1.222) at 18:b4:30:36:38:4b on xn1 expires in 1070 seconds [ethernet] ? (10.0.1.158) at 02:ff:60:ba:b5:82 on xn1 expires in 981 seconds [ethernet] ? (10.0.1.221) at 18:b4:30:3a:24:b1 on xn1 expires in 1053 seconds [ethernet] ? (10.0.1.220) at 18:b4:30:99:0f:95 on xn1 expires in 1195 seconds [ethernet] ? (10.0.1.211) at 64:16:66:9d:d6:b8 on xn1 expires in 969 seconds [ethernet] ? (10.0.1.146) at 7a:5b:b5:96:d0:af on xn1 expires in 993 seconds [ethernet] ? (10.0.1.210) at 18:b4:30:15:ab:50 on xn1 expires in 975 seconds [ethernet] ? (10.0.1.178) at 6c:40:08:94:5b:78 on xn1 expires in 971 seconds [ethernet] ? (10.0.1.177) at 00:9d:6b:bc:cc:68 on xn1 expires in 1088 seconds [ethernet] ? (10.0.1.176) at b0:ca:68:c5:06:11 on xn1 expires in 1114 seconds [ethernet] ? (10.0.1.212) at 64:16:66:9e:45:22 on xn1 expires in 1030 seconds [ethernet] ? (10.0.1.116) at 90:dd:5d:cf:ee:04 on xn1 expires in 976 seconds [ethernet] c-xx.xx.xx.xx.hsd1.il.comcast.net (xx.xx.xx.xx) at 2a:ec:f4:07:7c:72 on xn0 permanent [ethernet] ? (xx.xx.xx.xx) at 00:01:5c:64:f0:46 on xn0 expires in 780 seconds [ethernet
FreeNAS:
# arp -a ? (10.0.1.10) at 00:e0:67:13:5b:7f on igb0 expires in 1139 seconds [ethernet] ? (10.0.1.168) at 38:c9:86:1b:2b:2f on igb0 expires in 1154 seconds [ethernet] ? (10.0.1.173) at 8c:85:90:35:cc:e7 on igb0 expires in 1154 seconds [ethernet] pfSense.xxxxx.com (10.0.1.1) at ce:c0:9f:c5:9a:b1 on igb0 expires in 1171 seconds [ethernet] ? (10.0.1.6) at 64:a5:c3:5b:bb:90 on igb0 expires in 1182 seconds [ethernet] ? (10.0.1.5) at 34:12:98:03:ca:6a on igb0 expires in 1154 seconds [ethernet] ? (10.0.1.197) at 0c:c4:7a:84:a5:94 on igb0 permanent [ethernet] ? (10.0.1.186) at 9c:ae:d3:1e:31:d2 on igb0 expires in 1161 seconds [ethernet] ? (10.0.1.178) at 6c:40:08:94:5b:78 on igb0 expires in 848 seconds [ethernet]
and finally the jail at 10.0.1.158
? (10.0.1.158) at 02:ff:60:ba:b5:82 on epair0b permanent [ethernet] ? (10.0.1.178) at 6c:40:08:94:5b:78 on epair0b expires in 1198 seconds [ethernet]
Nothing looks amiss to me when I look at the arp cache tables. Hopefully this makes some sense to you. Problem goes away when I stop the jail, however I kind of need it running.
-
Curious, I'm not sure why it would only do that with the Jail active, but I'm not familiar enough with FreeNAS to say what it might be doing with the network stack inside the Jail. I haven't seen my plain FreeBSD jails do anything like that in the past.
This is unlikely to be anything related to pfSense, though. You'd probably have better luck asking on a FreeNAS-specific forum/subreddit
-
@jimp Just making sure however it probably isn't relative --- I have pfSense virtualized within xcp-ng (Citrix open source variant). I looked at the arp table for dom0 and it didn't have any other entries that conflicted either. I'm not overlooking a setting (promiscuous mode or some such) when I virtualize pfSense that would lead to this issue? FreeNas is not virtualized and runs on bare metal.
-
Doubtful. From what you've posted this would appear to be entirely contained somewhere inside FreeNAS. It's logging what it believes to be an alteration to something else on the FreeNAS system. The fact that the error mentions the actual IP address and MAC address of pfSense, and not something else, would appear to suggest that it's not a problem external to FreeNAS.
Slight possibility there is another device on the network also trying to be 10.0.1.1, but usually in that case you'd see an error logged on pfSense about another device attempting to use its MAC address.