PfSense network interface sometimes hangs on Hyper-V
-
pfSense version 2.4.2 Release p1 running on Hyper-V 2012R2.
We have been running pfSense for years but after upgrading to this version we had 2 network interface hangs in the last 3 weeks. Only way to recover was a restart of the VM.
The console logs show these error message repeated many times.Feb 23 02:56:14 amsfw1 kernel: hn0: RXBUF ack failed
Feb 23 02:56:14 amsfw1 kernel: hn0: RXBUF ack failed
Feb 23 02:56:14 amsfw1 kernel: hn0: RXBUF ack failed
…. etc.While in this state the other network interfaces kept operating normal.
Here the log from the second time.
Mar 19 00:21:20 amsfw1 kernel: hn1: RXBUF ack retry
Mar 19 00:21:20 amsfw1 kernel: hn1: RXBUF ack failed
Mar 19 00:21:20 amsfw1 kernel: hn1: RXBUF ack failed
Mar 19 00:21:20 amsfw1 kernel: hn1: RXBUF ack failed
.... etc.Any suggestions how to solve this?
Should we go back to an older version?Lex
-
Maybe try removing the interfaces and adding them back to the VM? Maybe a driver changed or something and Hyper-V doesnt like it. I'd try this after hours sometime since it wont take more then a minute or two.
Just a thought…
-
There appears to be a problem in the freebsd hyperv driver.
This is the exact same problem we have:
https://www.mail-archive.com/freebsd-stable@freebsd.org/msg134578.htmlAfter noticing a rollback in the 2.4.3RC I upgraded to that version, so far it has been stable for 7 days.
We have to wait and see.
Lex
-
Unfortunately had the same problem this morning. One of the interfaces had the RXBUF ack error and did not pass traffic anymore. Had to restart the VM to get back online.
Lex
-
this seems like it might be very similar to what we're seeing with pfSense on Proxmox for a while (seems like early proxmox 3 did not have the issue but we're seeing it a lot more now on current releases)…
Connectivity just fails after a few days/weeks especially with high load.
Tell me - when your pfSense is running - can you run some speed tests and see if your pfSense can pass at [your ISP's rated speed]? I found that even when our pfSense is "running" - regardless of the NIC drivers I've used, I do not see reliable speeds anywhere near the 200Mbps rated bandwidth from our ISP - however, a laptop connected directly to our ISP does run at rated speed…
This makes me deduce that even before pfSense finally "crashes" and stops talking to [one of my] networks, that there seems to be some sort of timing or packet loss or ? other timing issue… and for me - heavy traffic seems to trigger the failure...
Also note that I don't see any errors in any of the logs so far...
-
The problem we had is probably solved in the 2.4.3 release. We are now running for 22 days without problems. Still have to wait but sofar it looks good.
As for speed, it runs near the LAN speed of 1 Gbs.
In our case problem was clearly visible in the logs.
Lex
-
Same problem in the 2.4.4-RELEASE-p3 ( FreeBSD 11.2-RELEASE-p10 ) running on Hyper-V 2012R2.
Every few days pfsens stops forvarding traffic on one of the interfaces with an log entry kernel: hn1: RXBUF ack failed
Any advice?
Piotrek
-
It looks like freebsd ntpd and hyper-v timesync conflict here.
Please read here: https://groups.google.com/forum/#!topic/muc.lists.freebsd.stable/q6uwGYmSWqQ
NTPD or timesync disabling is recommended. -
I got the same problem on Hyper-V of Windows Server 2019. I disabled timesync, is there anything else I can do?
-
The timesync disabling had no effect, it still crashes.
-
I really would like to have some sleep in the next days can somebody help me, the firewall is crashing like every day after midnight.
-
@dominikk said in PfSense network interface sometimes hangs on Hyper-V:
I really would like to have some sleep in the next days can somebody help me, the firewall is crashing like every day after midnight.
I have seen a number of posts lately about Hyper-V and FreeBSD, especially the newer 11.3/STABLE and 12.1/STABLE versions of FreeBSD. Since pfSense is based on FreeBSD, any FreeBSD problems are going to show up in pfSense virtual machines. I have seen posts and trouble reports for stalling, crashes, virtual NIC adapters, and more with pfSense/FreeBSD and Hyper-V. It seems the Hyper-V with Windows 2016 and higher has been especially problematic.
One big thing that recently changed with the latest pfSense version is the underlying FreeBSD OS was updated from 11.1/RELEASE to 11.3/STABLE. I doubt it's what you want to hear, but ESXi seems to work better with the newer FreeBSD versions. Is swapping over an option for you?
-
Thanks, swapping hypervisors is not really an option.
I will have to consider some non FreeBSD firewalls like IPFire then. Still they are not as great as pfsense sadly.
-
Next two, top best solution :
Use a vanilla Windows Pro : I was using pre-2.4.5 using FreeBSD 1.1 and FreeBSD 11.3 using pfSense 2.4.5-p1, and have not seens any issues. But notre that this is a @home setup - more or like a test bed install - as VM are ment to be.
The real setup shouldn't be discarded :
You could even hide it into the Win 2012 device if you have space constraints. No VM code here so many things that can't go wrong.
