Watchdog timeout on queue 0
-
@rulerof Has the issue popped again after doing what you outlined? I'm encountering this all of a sudden and seems like the time-span is random. Started occurring (it seems) after adding a 3rd NIC to the VM to connect to a DVR. I disabled the NIC (in pfSesne) and it seems like the issue subsided. That was about 10 days ago. Then, it just occurred again. Not sure what's going on. But, I'll look into implementing your steps.
-
@erutan409 it happened one more time in the last month, and just like before I really don't have any explanation as to why. Sorry :(
-
I resolved this issue, running pfSense as a VM in vSphere. I changed the NIC's assigned to the VM from VMXNET 3 to E1000e. Apparently, there's an issue with grouping more than two of the VMXNET 3 type of NIC's together in the same environment. Up and running for about a month without the issue reoccurring.
-
@Erutan409 if this happens to me again, I'll change my virtual NICs over to fully-emulated hardware. I hate to have to do that, but I'm glad it seems to be a viable fix!
-
Was there ever a fix for this? I have the latest version of pfsense running on an esxi host. I have a 4 port intel gigabit nic that was using two ports for a lan and wan. That would cause the watchdog timeout on queue 0 to happen every couple of hours. I have since switched to the onboard motherboard nic for LAN and using the quad port card for WAN. That seemed to have fixed it, but then I had it happen again today. Any reason why this is happening?
-
@clifford64 I mentioned in my last response what my fix was. If your environment is similar, I'd suspect it's the type of virtual adapter you're using. And I'm not sure it's something that is reconcilable within pfSense. It might be in the OS, itself.
-
So, I may have found a fix for my configuration - accounting for bufferbloat:
- vSphere - 6.7 - VMXNET3 on all 4 network adapters I have configured on the VM
- Internet speed - 250 Mbps/30 Mbps
- pfSense version - latest (2.4.4_3)
Following this guide, I seemed to have avoided getting this issue when (a reoccurring issue) I'd set a couple downloads going on my PS4 and max out the download speed during the night. This seems to be the best reproduction of this issue, without fail, when I'd wake up in the morning.
The Internet would get sluggish and eventually stop working all together. Then, I'd log into vSphere and see the issue in the VM console.
But, after following that guide, it seems like my Internet is overall stable. I'll update if it still happens in the future. Hoping this IS the fix.
-
It happened again. Took over a year for the problem to recur.
I've switched the problematic interface to the
e1000type. I'll reply back if I experience the issue again. -
So, I'm not sure where I found [potential] solution for this, but switching to those interfaces didn't seem to fix the problem, either. It seems that setting the VM to sync its time with the vSphere host was causing the issue as the host would start losing A LOT of time over the span of a month or two. We're talking like close to a minute.
I disabled the time sync setting and had pfSense keep itself synced, reverted back to VMXNET interfaces and I've been running fine for 4+ months now. I haven't updated pfSense for a couple years, FWIW.
Same Internet activity I've had, plus working from home (COVID forced me to), hasn't reproduced the issue.
-
@Erutan409 I've been running with it for a couple of days and discovered that my pfSense box has significantly increased CPU load, and services behind that particular interface feel throttled.
Again, I'm running on KVM and I don't think it has any such paravirtual time synchronization—I run NTP on the host and have pfSense update from the host ntpd once an hour.
In the meantime, I've managed to break my pfSense install by powering it off at the wrong point, so I'm going to reinstall the latest version from ISO and switch back to PV NICs. I'll update here again if I learn anything.