pfSense doesn not respond to ARPs
-
Let me set the stage here
pfsense -- unifi switch -- att gateway
For a few months I've been troubleshooting connectivity issues with pfsense. Randomly it would drop network connectivity to the Internet and I am not able to access the firewall from the LAN. There is a thread i have out there that documented that whole fiasco but ultimately TL:DR , still don't know why its broken. I am running a 6100 and using the ix* interfaces for WAN seemed to be the culprit. When i move the WAN to the igc interfaces the random disconnects on the LAN/WAN stopped.
In between this time i moved my WAN connection to my switch so i can set up a port mirror to see whats happening on the wire. Also a cable modem swapped happen. All was well for about a month. I moved the WAN back to the ix interface and all was stable.
Today....it happened again. LAN access was not possible. Internet was very spotty. Randomly i would hit a website abut then nothing would would connect. DNS resolution to pfsense was timing out.
Access to my Unifi switch was possible and i did have a SPAN port from my ATT modem.
Packet captures show ARP requests from my firewall and the ATT gateway responding.
The ATT gateway is sending out a ARP requests and there is no ARP reply from pfsense.
Pfsense is for sure the culprit and i have pcap evidence.I moved my SPAN port to now mirror traffic off the pfsense WAN port. The condition is the same there. ATT modem is sending a ARP requests and pfsense is not responding. pfSense is sending a ARP requests and the ATT modem is responding.
This isn't a switch issue as this was happening way before connectivity went through a switch. The switch is only used to see mirrored traffic.
@stephenw10 Finally figured out the true culprit. I still think its a NIC failure on the ix side.
I can share the pcaps if you are curious.Has anyone seen or been through something like this on pfsense?
NIC failures cause these type of issues?Here is a snippet of the pcap. notice the HUMAX is responding. Silicom [pfsense] never responds to HUMAXs arp..
Those pub IPs will be changed tomorrow
-
Can we see an actual pcap file with the failed ARP? I assume the MAC addresses are correct?
And to be clear it resumes correct function after rebooting the 6100?
I forget what other tests we did there. Replugging the WAN cable? Resaving the interface? Either bring it back?
-
That is correct, 6100 is working again after a reboot.
Can you DM the link I can use to upload the pcaps? -
You can upload them here: https://nc.netgate.com/nextcloud/s/tMRseYCQ2HWzkKs
-
@stephenw10 files uploaded.
2x different capture points but same story.
-
@stephenw10
what do you think about the pcaps? -
@michmoor you don't have the ethernet filtering enabled do you? With that it is possible to block arp..
-
@johnpoz said in pfSense doesn not respond to ARPs:
@michmoor you don't have the ethernet filtering enabled do you? With that it is possible to block arp..
Turned off. Never used it actually
-
Interesting. Were they filtered?
I'm not sure why we see the ARP requests from pfSense in the modem mirror but not in the pfSense mirror?
Also interesting that we see ARP requests from the modem but from 192.168.1.254.
-
@stephenw10
What do you mean by filtered?
My theory right now is that the card is faulty. Although we see the modem responding it may not be processing within the card. I can’t think of anything else.
Cables have been swapped
Modems have been swapped
The only solution is the igc card and the problem goes away -
I mean when you ran the pcap was it capturing all traffic or was it filtering by just a limited set of MAC addresses or IP addresses for example?
