High interrupt CPU usage in v2.7.1
-
Same issue occurring with me. Fresh install of XCP-NG 8.2, single VM with pfsense, TCP offload disabled both in the XCP-ng host as well as pfsense interface.
Getting 60% interrupts at all times, goes down when loaded.
Interestingly, this seems to result in upload speeds of 300-400Mbit while downloads are 920Mbit on a 1Gb fiber WAN.
I really love the XCP-ng infrastructure but this is likely going to force me to switch to proxmox for these router + unifi controller builds. -
I can also confirm this is happening on XCP-ng 8.2 here is my
vmstat -iinterrupt total rate irq1: atkbd0 2 0 irq23: uhci0 16 0 cpu0:xen 143780 161 cpu1:xen 142710 160 irq2096: cpu0:r 298 0 irq2097: cpu0:itlb 6922 8 irq2098: cpu0:b 35713 40 irq2102: cpu1:r 115 0 irq2103: cpu1:itlb 7700 9 irq2104: cpu1:b 39100 44 irq2108: xen_et0:c0 86311 97 irq2109: xen_et0:c1 85965 96 irq2110: xenstore0 3020 3 irq2113: xbd0 11017 12 irq2114: xn0 819 1 irq2115: xn0 957 1 irq2116: xn0 1699 2 irq2117: xn0 4956 6 irq2118: xn1 1114 1 irq2119: xn1 130 0 irq2120: xn1 90 0 irq2121: xn1 418 0 irq2124: xn2 1200 1 irq2125: xn2 273 0 Total 574325 643 -
System referenced this topic on
-
Same issue with 2.7.1 on XCP-ng 8.2.1 Clean install for testing purposes, everything is default with the exception of DHCP service is disabled.
[2.7.2-RELEASE][admin@pfSense.home.arpa]/root: vmstat -i interrupt total rate irq1: atkbd0 128 0 irq15: ata1 1375 0 irq23: uhci0 16 0 cpu0:xen 176561 47 cpu1:xen 152210 41 cpu2:xen 180816 48 cpu3:xen 141422 38 irq2096: cpu0:r 274 0 irq2097: cpu0:itlb 6901 2 irq2098: cpu0:b 71712 19 irq2102: cpu1:r 249 0 irq2103: cpu1:itlb 7447 2 irq2104: cpu1:b 77524 21 irq2108: cpu2:r 230 0 irq2109: cpu2:itlb 7074 2 irq2110: cpu2:b 76266 20 irq2114: cpu3:r 101 0 irq2115: cpu3:itlb 7389 2 irq2116: cpu3:b 74840 20 irq2120: xen_et0:c0 61626 16 irq2121: xen_et0:c1 44576 12 irq2122: xen_et0:c2 44672 12 irq2123: xen_et0:c3 44854 12 irq2124: xenstore0 1437 0 irq2129: xbd0 30860 8 irq2138: xn2 5051 1 irq2139: xn2 11971 3 irq2140: xn2 47811 13 irq2141: xn2 22958 6 irq2142: xn3 1783 0 irq2143: xn3 1866 0 irq2144: xn3 8840 2 irq2145: xn3 1782 0 Total 1312622 351 -
@onu sorry, 2.7.2 for pfSense
-
just to be thorough, I installed Proxmox on the same system, then pfSense as a VM.
way different results.
[2.7.2-RELEASE][admin@pfSense.home.arpa]/root: vmstat -i interrupt total rate irq1: atkbd0 2 0 irq11: uhci0+ 20 0 irq15: ata1 449 0 cpu0:timer 19445 21 cpu1:timer 16727 18 cpu2:timer 14113 15 cpu3:timer 14768 16 irq27: virtio_pci1 12081 13 irq29: virtio_pci2 6783 7 irq30: virtio_pci2 1 0 irq32: virtio_pci3 194987 212 irq33: virtio_pci3 278 0 Total 279654 304 [2.7.2-RELEASE][admin@pfSense.home.arpa]/root: -
I have the same issue. Running pfSense CE 2.7.2 in a VM on a XCP-ng host (8.2.1). Interrupt CPU usage is between 60-80%.
-
Some more data:
Topon the vm:last pid: 23296; load averages: 0.08, 0.14, 0.14 up 0+18:46:31 10:20:03 286 threads: 2 running, 251 sleeping, 33 waiting CPU: 0.0% user, 0.0% nice, 0.0% system, 84.0% interrupt, 16.0% idle Mem: 85M Active, 142M Inact, 299M Wired, 56K Buf, 403M Free ARC: 171M Total, 46M MFU, 118M MRU, 4096B Anon, 841K Header, 5776K Other 143M Compressed, 278M Uncompressed, 1.94:1 Ratio Swap: 1024M Total, 1024M FreeTopOn the hypervisor:top - 10:16:34 up 9 days, 12:45, 1 user, load average: 651.11, 650.81, 649.76 Threads: 1763 total, 1 running, 1696 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.4 us, 0.7 sy, 0.0 ni, 98.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.3 st KiB Mem : 1329412 total, 39032 free, 892192 used, 398188 buff/cache KiB Swap: 1048572 total, 951548 free, 97024 used. 277052 avail Memvmstat -iinterrupt total rate irq1: atkbd0 1094 0 irq15: ata1 44975 1 irq23: uhci0 20 0 cpu0:xen 23137567 342 irq2102: xen_et0:c0 7093144 105 irq2103: xenstore0 8039 0 irq2105: xbd0 599431 9 irq2106: xn0 925434 14 irq2107: xn0 501940 7 irq2108: xn0 437785 6 irq2109: xn0 4301938 64 irq2110: xn1 2861373 42 irq2111: xn1 1348751 20 irq2112: xn1 1813507 27 irq2113: xn1 3541821 52 irq2114: xn2 44963 1 irq2115: xn2 15997 0 irq2116: xn2 50798 1 irq2117: xn2 133775 2 irq2121: xn3 6 0 Total 46862358 693 -
I am seeing the same thing, though it looks like it might be an accounting bug and not a real issue - see: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277231
Regardless, it does not seem to be related to the network cards or the version of Xen.
I see the same thing with only a physical Intel X520-DA2 passed through to the VM.
I also tested on XCP-ng 8.3-beta2 with Xen 4.13 as well as with Xen 4.17; all show the same issue. -
@Raghnall Thank you for finding the bug report! I did search on the FreeBSD forums but couldn't find anything.
-
Same thing here, running pfSense 2.7.2 on XCP-ng 8.3.0
It appears that this is just a bug on pfSense reporting itself because the VM stats show everything peacefully working.
-
In top it shows high interrupt
-
S stephenw10 referenced this topic on