Remove VMware MSI-X from the PCI blacklist.

kiokoman

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203874
https://reviews.freebsd.org/rS355889

First reported against ESXi 5.0, PCI passthrough was not working due to
MSI-X issues. However, this issue was fixed via patch releases against
ESXi 5.5 and 6.0 in 2016. Given ESXi 5.5 and earlier have been EOL, this
patch removes the VMware MSI-X blacklist entries in the quirk table.

@jimp is this something that will automatically apply to pfsense in the future or do I need to open a feature request?

https://github.com/pfsense/FreeBSD-src/blob/devel-12/sys/dev/pci/pci.c

because I've noticed this error on my pfsense

vmx0: <VMware VMXNET3 Ethernet Adapter> port 0x5000-0x500f mem 0xfd3fc000-0xfd3fcfff,0xfd3fd000-0xfd3fdfff,0xfd3fe000-0xfd3fffff irq 19 at device 0.0 on pci4
vmx0: Using 512 TX descriptors and 512 RX descriptors
vmx0: Using 4 RX queues 4 TX queues
vmx0: failed to allocate 5 MSI-X vectors, err: 6
vmx0: Using an MSI interrupt

and I need to use

hw.pci.honor_msi_blacklist=0

in /boot/loader.conf.local

jimp

It looks like that change was never MFC'd back to 12. I only see it in the master branch on FreeBSD.

You can open a feature request pointing at it, we may be able to poke upstream and see if they can merge it. You can also comment on that FreeBSD PR and mention it.

Cool_Corona

@kiokoman said in Remove VMware MSI-X from the PCI blacklist.:

hw.pci.honor_msi_blacklist=0

Have you tried hw.pci.honor_msi_blacklist=1 ??

kiokoman

@Cool_Corona
it work ok with hw.pci.honor_msi_blacklist=0 the problem is that there should be no more need for it

asche

@kiokoman said in Remove VMware MSI-X from the PCI blacklist.:

@Cool_Corona
it work ok with hw.pci.honor_msi_blacklist=0 the problem is that there should be no more need for it

Just to chime in here: setting it to 0 did help a lot in my ESXi deployment by bringing down IRQ CPU utilisation by ca factor 10 (with passed through Intel NICs).

It should be made the default or at least added to the tuning section in the docs not only for vmxnet NICs (already in there since some weeks ago) but also intel NICs passed through to a virtual pfsense.