Proxmox: WAN NIC passthrough issue with 2.5G connection to fiber router
-
So the WAN gateway stops responding? What is the gateway monitoring set to?
It sounds like something loses an ARP entry. Does the gateway disappear from the ARP table in pfSense? (Diag > ARP).
If you run a pcap on the WAN when it fails, is it still sending the ping packets?
Steve
-
thanks for pointing me out to look at arp
i dont have gw monitoring. the issue remains when it s enabled
my arp table has 2 entries but internet stops when the entries have grown.
it goes up to 90+ entriesmy isp is using cgnat
the original 2 entries are still there when it breaks
-
Hmm, how are testing then? Where are you pinging from?
But, yes, run pcap and see what's being sent and if anything is coming back.
-
@stephenw10
the test is that i just lose internet
I did a pcap I can see traffic going out but nothing back
it seems that I get arp entries from every other customers on the ISP
I 'm trying to block them on the firewall but no luck
any suggestions on preventing other customers to flood me with arp? -
Seeing broadcast ARPs from other customers is only an indication of a poor setup at the ISP IMO. It should not affect your connectivity unless maybe here is a conflicting MAC address or IP. But pfSense would log that and complain loudly if there were.
You said it stops after ~2000 pings so I assumed you were running a ping test from somewhere against something?
We have seen similar reports of ISP gateways losing ARP entries and instead of ARPing to renew it just dropping traffic which does sound similar. However it normally comes back after some time in those situations. Are you seeing that?
A workaround for that is to set the pfSense ARP renewal time much shorter so it keeps the upstream device current. To test that run at the command line:
sysctl net.link.ether.inet.max_age=300 -
@lp-descamps said in Proxmox: WAN NIC passthrough issue with 2.5G connection to fiber router:
Has anyone encountered a similar issue with Intel I225-V NICs or passthrough configurations?
I have encountered similar behavior with multiple I225 class NICs, yes. No passthrough in my case. The only thing that showed up in the pfSense logs was repeating entries like:
kernel: arpresolve: can't allocate llinfo for <wan gateway ip > on igc0Like you, simply "reconfiguring" the interface without any changes would bring back the connection until next time it happened.
I had this problem with 3 different igc NICs, including one OEM Intel part and two cheap chinese parts.
I eventually gave up on igc and got a different NIC, in my case an X710-T4L.
-
@whosmatt said in Proxmox: WAN NIC passthrough issue with 2.5G connection to fiber router:
arpresolve
I do get this error too
kernel arpresolve: can't allocate llinfo for 192.168.1.1 on igc0but I don't use 192.168.1.0 network on pfsense?!?
thanks
-
i m pinging from a laptop to 1.1.1.1
I ve now enable the gw monitoring to ping 8.8.8.8 instead of ISP gw. will how it goes
the internet connection works again after the lease time of the arp expires and disappears from the table and I m back with 2 arp entries from the isp. the gw IP and wan interface
I will try sysctl net.link.ether.inet.max_age=300 if the gw monitoring didn't do the trick. what is the default value for max_age in case I want to roll this back?
thanks
-
@lp-descamps
lost internet againchanged the sysctl net.link.ether.inet.max_age from 1200 to 300
I didn't get kernel arpresolve: can't allocate llinfo for 192.168.1.1 on igc0 errors this time
-
@lp-descamps said in Proxmox: WAN NIC passthrough issue with 2.5G connection to fiber router:
but I don't use 192.168.1.0 network on pfsense?!?
Might be a clue there though. Your WAN is DHCP I'm assuming? Perhaps something else is giving it a lease with that IP as the gateway.
-
@stephenw10
there is a conflicting MACwhen internet is working I can see this in arp table,
WAN 100.68.0.1 42:9e:a4:3c:8e:2b Expires in 1156 seconds ethernet WAN 100.69.245.16 7c:83:34:bd:4b:21 Permanent ethernetwhen internet breaks, I get this
WAN 100.68.0.64 42:9e:a4:3c:8e:2b Expires in 1186 seconds ethernet WAN 100.68.0.32 42:9e:a4:3c:8e:2b Expires in 1186 seconds ethernet WAN 100.68.0.65 42:9e:a4:3c:8e:2b Expires in 1186 seconds ethernet WAN 100.69.245.16 7c:83:34:bd:4b:21 Permanent ethernet WAN 100.68.0.69 42:9e:a4:3c:8e:2b Expires in 1186 seconds ethernet WAN 100.68.0.31 42:9e:a4:3c:8e:2b Expires in 1186 seconds ethernet -
Which of those, if any, is the WAN gateway? If it is it's a long way from the WAN IP or a huge subnet.
That doesn't look like it's necessarily a conflict though. One MAC address could be associated with all those IPs.
-
@stephenw10
the wan gw is 100.68.0.1 42:9e:a4:3c:8e:2b
i can see it in the arp table before and after I lose internet
but sometimes I get incomplete instead of MAC -
Hmm, if it shows 'incomplete' then it's unable to get the gateway MAC which implies a layer2 failure.
Where is the gateway in this case? Remote, at the ISP?
It does seem like you might be hitting an issue with the NIC though.
-
@stephenw10
i believe the gw is remotely at the isp.
all I have at home is a fiber box that seems unmanaged
is there a way to set static arp?
how would I validate a hw nic failure? -
I've yet to see the i225 disconnect issues that are relatively well known but as I understand it they usually show as the NIC losing link which I don't think you're seeing?
I would run a pcap on it in the failed state and see if it's actually passing packets at all.
-
@stephenw10 that s one of the capture packetcapture-igc0-20241212183144.pcap
-
Hmm, so no incoming traffic at all.
Have you tried running it as not pass-through so pfSense just sees it as vtnet?
-
@stephenw10 i was non-passthrough and had issues hence moving to passthrough
-
What sort of issues? Similar connection failures?
