disable sound driver

wickeren

I have some hosted PFsense VM's running on KVM showing permanent high CPU usage from IRQ11:

12 root -96 - 0B 272K WAIT 0 337:42 51.95% [intr{irq11: pcm0 uhci0+*}]
11 root 155 ki31 0B 32K RUN 1 323:01 50.98% [idle{idle: cpu1}]
11 root 155 ki31 0B 32K RUN 0 256:02 37.99% [idle{idle: cpu0}]

This turn out to be a (virtual) soundcard:

cat /dev/sndstat
Installed devices:
pcm0: <AudioPCI ES1370> (play/rec) default

As I have no control over the hosted KVM hypervisor itself I cannot control or disable the virtual hardware from there.
Is there a way to disable this in PFsense itself?

I tried already:

snd_driver_load="NO"

in loader.conf.local, but this doesn't seem to do the trick.
Any other ideas?

jimp

That is a bit odd, I don't have that on any of my KVM VMs.

Here is what the VM config looks like on nearly all my KVM VMs in Proxmox:

I know you don't have any control over it, but can you at least have them show you the VM setup?

You might be able to disable it via device hints. Edit /boot/device.hints and at the end, add:

hint.pcm.0.disabled="1"

wickeren

Wish it was Proxmox, then I had all control about virtual hardware needed, but the hoster just tells this is the way it is...
But I can test in my lab Proxmox if I can disable it the way you suggested. Tnx for the hint!

wickeren

Yes, that did the trick! No more unnecessary IRQs eating up CPU cycles.

Tnx a lot!

jimp

Now that you know it works, move that exact same line over to /boot/loader.conf.local which wouldn't be clobbered by updates. Test it again after, but that should work the same way.

/boot/device.hints will get overwritten eventually, even if it is the "proper" location for those hints.

wickeren

@jimp

Bit too early... while the PCM0 device is gone, the same IRQ11 is still using a lot of CPU cycles, this time used by virtio_pci0:

12 root -92 - 0B 272K WAIT 0 65:52 69.97% [intr{irq11: virtio_pci0*}]
11 root 155 ki31 0B 32K CPU1 1 80:35 35.99% [idle{idle: cpu1}]
11 root 155 ki31 0B 32K RUN 0 65:12 21.97% [idle{idle: cpu0}]
12 root -92 - 0B 272K WAIT 1 13:45 14.99% [intr{irq10: virtio_pci2*}]

I don’t know how to calm this down, the traffic is almost idle while seeing this. This machine has a total of 4 Virtio Interfaces, but I only see IRQ11 en 10 used and pci0 and pci2.
Looks like an issue specific to IRQ11 but have no clue why.
Any help appreciated!

jimp

Look at pciconf -lvb and see what is reported as being on that device.

Also look at dmesg | grep 'irq 11' to see what is on that IRQ.

The last time I saw some baseline weird CPU usage on the VM, it turned out to be related to the console. Switching to the SPICE console fixed it. Though IIRC it might have been something in the virtualized mouse driver or along those lines. SPICE is a much better console than the default so I didn't dig too deep.

wickeren

@jimp

pciconf -lvb output:

hostb0@pci0:0:0:0: class=0x060000 card=0x11001af4 chip=0x12378086 rev=0x02 hdr=0x00
vendor = 'Intel Corporation'
device = '440FX - 82441FX PMC [Natoma]'
class = bridge
subclass = HOST-PCI
isab0@pci0:0:1:0: class=0x060100 card=0x11001af4 chip=0x70008086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82371SB PIIX3 ISA [Natoma/Triton II]'
class = bridge
subclass = PCI-ISA
atapci0@pci0:0:1:1: class=0x010180 card=0x11001af4 chip=0x70108086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82371SB PIIX3 IDE [Natoma/Triton II]'
class = mass storage
subclass = ATA
bar [20] = type I/O Port, range 32, base 0xc260, size 16, enabled
uhci0@pci0:0:1:2: class=0x0c0300 card=0x11001af4 chip=0x70208086 rev=0x01 hdr=0x00
vendor = 'Intel Corporation'
device = '82371SB PIIX3 USB [Natoma/Triton II]'
class = serial bus
subclass = USB
bar [20] = type I/O Port, range 32, base 0xc180, size 32, enabled
intsmb0@pci0:0:1:3: class=0x068000 card=0x11001af4 chip=0x71138086 rev=0x03 hdr=0x00
vendor = 'Intel Corporation'
device = '82371AB/EB/MB PIIX4 ACPI'
class = bridge
vgapci0@pci0:0:2:0: class=0x030000 card=0x11001af4 chip=0x11111234 rev=0x02 hdr=0x00
class = display
subclass = VGA
bar [10] = type Prefetchable Memory, range 32, base 0xfd000000, size 16777216, enabled
bar [18] = type Memory, range 32, base 0xfebd0000, size 4096, enabled
virtio_pci0@pci0:0:3:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio network device'
class = network
subclass = ethernet
bar [10] = type I/O Port, range 32, base 0xc1a0, size 32, enabled
bar [14] = type Memory, range 32, base 0xfebd1000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe000000, size 16384, enabled
virtio_pci1@pci0:0:4:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio network device'
class = network
subclass = ethernet
bar [10] = type I/O Port, range 32, base 0xc1c0, size 32, enabled
bar [14] = type Memory, range 32, base 0xfebd2000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe004000, size 16384, enabled
virtio_pci2@pci0:0:5:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio network device'
class = network
subclass = ethernet
bar [10] = type I/O Port, range 32, base 0xc1e0, size 32, enabled
bar [14] = type Memory, range 32, base 0xfebd3000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe008000, size 16384, enabled
virtio_pci3@pci0:0:6:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio network device'
class = network
subclass = ethernet
bar [10] = type I/O Port, range 32, base 0xc200, size 32, enabled
bar [14] = type Memory, range 32, base 0xfebd4000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe00c000, size 16384, enabled
virtio_pci4@pci0:0:7:0: class=0x020000 card=0x00011af4 chip=0x10001af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio network device'
class = network
subclass = ethernet
bar [10] = type I/O Port, range 32, base 0xc220, size 32, enabled
bar [14] = type Memory, range 32, base 0xfebd5000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe010000, size 16384, enabled
pcm0@pci0:0:8:0: class=0x040100 card=0x4c4c4942 chip=0x50001274 rev=0x00 hdr=0x00
vendor = 'Ensoniq'
device = 'ES1370 [AudioPCI]'
class = multimedia
subclass = audio
bar [10] = type I/O Port, range 32, base 0xc000, size 256, enabled
virtio_pci5@pci0:0:9:0: class=0x010000 card=0x00021af4 chip=0x10011af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio block device'
class = mass storage
subclass = SCSI
bar [10] = type I/O Port, range 32, base 0xc100, size 128, enabled
bar [14] = type Memory, range 32, base 0xfebd6000, size 4096, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe014000, size 16384, enabled
virtio_pci6@pci0:0:10:0: class=0x00ff00 card=0x00051af4 chip=0x10021af4 rev=0x00 hdr=0x00
vendor = 'Red Hat, Inc.'
device = 'Virtio memory balloon'
class = old
bar [10] = type I/O Port, range 32, base 0xc240, size 32, enabled
bar [20] = type Prefetchable Memory, range 64, base 0xfe018000, size 16384, enabled

dmesg | grep 'irq 11' output:

uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xc180-0xc19f irq 11 at device 1.2 on pci0
virtio_pci0: <VirtIO PCI Network adapter> port 0xc1a0-0xc1bf mem 0xfebd1000-0xfebd1fff,0xfe000000-0xfe003fff irq 11 at device 3.0 on pci0
virtio_pci1: <VirtIO PCI Network adapter> port 0xc1c0-0xc1df mem 0xfebd2000-0xfebd2fff,0xfe004000-0xfe007fff irq 11 at device 4.0 on pci0
virtio_pci4: <VirtIO PCI Network adapter> port 0xc220-0xc23f mem 0xfebd5000-0xfebd5fff,0xfe010000-0xfe013fff irq 11 at device 7.0 on pci0
pcm0: <AudioPCI ES1370> port 0xc000-0xc0ff irq 11 at device 8.0 on pci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xc180-0xc19f irq 11 at device 1.2 on pci0
virtio_pci0: <VirtIO PCI Network adapter> port 0xc1a0-0xc1bf mem 0xfebd1000-0xfebd1fff,0xfe000000-0xfe003fff irq 11 at device 3.0 on pci0
virtio_pci1: <VirtIO PCI Network adapter> port 0xc1c0-0xc1df mem 0xfebd2000-0xfebd2fff,0xfe004000-0xfe007fff irq 11 at device 4.0 on pci0
virtio_pci4: <VirtIO PCI Network adapter> port 0xc220-0xc23f mem 0xfebd5000-0xfebd5fff,0xfe010000-0xfe013fff irq 11 at device 7.0 on pci0
pcm0: <AudioPCI ES1370> port 0xc000-0xc0ff irq 11 at device 8.0 on pci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xc180-0xc19f irq 11 at device 1.2 on pci0
virtio_pci0: <VirtIO PCI Network adapter> port 0xc1a0-0xc1bf mem 0xfebd1000-0xfebd1fff,0xfe000000-0xfe003fff irq 11 at device 3.0 on pci0
virtio_pci1: <VirtIO PCI Network adapter> port 0xc1c0-0xc1df mem 0xfebd2000-0xfebd2fff,0xfe004000-0xfe007fff irq 11 at device 4.0 on pci0
virtio_pci4: <VirtIO PCI Network adapter> port 0xc220-0xc23f mem 0xfebd5000-0xfebd5fff,0xfe010000-0xfe013fff irq 11 at device 7.0 on pci0
pcm0: <AudioPCI ES1370> port 0xc000-0xc0ff irq 11 at device 8.0 on pci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xc180-0xc19f irq 11 at device 1.2 on pci0
virtio_pci0: <VirtIO PCI Network adapter> port 0xc1a0-0xc1bf mem 0xfebd1000-0xfebd1fff,0xfe000000-0xfe003fff irq 11 at device 3.0 on pci0
virtio_pci1: <VirtIO PCI Network adapter> port 0xc1c0-0xc1df mem 0xfebd2000-0xfebd2fff,0xfe004000-0xfe007fff irq 11 at device 4.0 on pci0
virtio_pci4: <VirtIO PCI Network adapter> port 0xc220-0xc23f mem 0xfebd5000-0xfebd5fff,0xfe010000-0xfe013fff irq 11 at device 7.0 on pci0

So looks like USB on same interrupt might be the culprit?
About the console, it’s definitely not spice but have no control about the console type either, as I would have in e.g Proxmox.

jimp

Could be. I'm not sure what may be connected to that, you can try usbconfig to see what USB devices show up.

If you don't see anything, or it just shows the hub, try adding this to loader.conf.local:

hint.uhci.0.disabled="1"

If you see a keyboard or mouse attached you may need USB for console access.

wickeren

@jimp

usbconfig output:
ugen0.1: <Intel UHCI root HUB> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=SAVE (0mA)
ugen0.2: <QEMU QEMU USB Tablet> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)

I don’t see any benefit in a QEMU USB Tablet for something without a gui, so going to try your hint. If the console might fail I can still connect SSH.

Tnx again!

jimp

I think that is the console issue I was thinking of. There is a virtual machine option "Use tablet for pointer" which leads to this problem with the default console.

If you could change that option on the VM, that would also likely fix it, though hopefully disabling the device is sufficient.

There is no need for a mouse/pointer on pfSense anyhow so it should be safe.

wickeren

After setting hint.uhci.0.disabled="1" a bit better, but IRQ11 still quite high:

11 root 155 ki31 0B 32K RUN 1 1:48 67.97% [idle{idle: cpu1}]
11 root 155 ki31 0B 32K CPU0 0 1:45 56.98% [idle{idle: cpu0}]
12 root -92 - 0B 272K WAIT 0 1:20 32.96% [intr{irq11: virtio_pci0*}]
12 root -92 - 0B 272K WAIT 1 0:21 5.96% [intr{irq10: virtio_pci2*}]

jimp

Did the output of the commands above change at all due to the hints?

Not likely, but worth checking.

There may not be much more you can do in the guest without making changes to the VM config.

wickeren

@jimp

Yeah, it did, at least partially:

cat /dev/sndstat output:
No devices installed.
No devices installed from userspace.

usbconfig output:
No device match or lack of permissions.

so setting pcm0 en uhci0 to diabled helped with that. But it did not lower the CPU load from IRQ11 and the device are still visible with pciconf -lvb

Only thing I can do is contact the hosting company. I have linked them to this this thread, maybe that will help to convince them to change the KVM config for Pfsense VM's.